Top Banner
LINEAR ALGEBRA Sourav Sen Gupta
75

LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

Feb 09, 2019

Download

Documents

nguyentuyen
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

LINEAR ALGEBRASourav Sen Gupta

Page 2: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding
Page 3: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

PLAN OF ACTION

Part I — Geometry of Multivariate Data

1. Vector space modelling of multivariate data

2. Understanding Matrices as linear operators

3. Subspaces and Singular Value Decomposition

Part II — Applications in Multivariate Analysis

1. SVD and Principal Component Analysis

2. Eigenvalues and Eigenvectors in PageRank

3. Factor Analysis and SVD in Recommenders

Page 4: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

MULTIVARIATE DATA

Page 5: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

MULTIVARIATE DATA

p variables/features

n samples/observations

X1 X2 X3 X4S1 5.1 3.5 1.4 0.2S2 4.9 3 1.4 0.2S3 4.7 3.2 1.3 0.2S4 4.6 3.1 1.5 0.2S5 5 3.6 1.4 0.2

S146 6.7 3 5.2 2.3

S147 6.3 2.5 5 1.9

S148 6.5 3 5.2 2

S149 6.2 3.4 5.4 2.3

S150 5.9 3 5.1 1.8

Page 6: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

MULTIVARIATE DATA — HISTOGRAMS

Page 7: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

MULTIVARIATE DATA — BOXPLOTS

Page 8: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

MULTIVARIATE DATA — CORRPLOT

Page 9: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

MULTIVARIATE DATA — SCATTERPLOT

Page 10: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

MULTIVARIATE DATA — SCATTERPLOT

Page 11: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

MULTIVARIATE DATA — SCATTERPLOT

Page 12: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

MULTIVARIATE DATA — SCATTERPLOT

Page 13: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

MULTIVARIATE DATA — SCATTERPLOTS

Page 14: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

MULTIVARIATE DATA — SCATTERPLOTS

Page 15: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

MULTIVARIATE DATA — 3D SCATTERPLOT

Page 16: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

MULTIVARIATE DATA — 3D SCATTERPLOT

Page 17: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding
Page 18: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

LET’S START WITH 2D

Page 19: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

2D GEOMETRY

Page 20: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

2D GEOMETRY

(x,y)

x - units

y - units

X

Y

Page 21: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

SIZE

X

Y

(x,y)

Page 22: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

SIZE

X

Y

(x,y)

size of (x,y) = sqrt(x2 + y2)L2 Norm

Page 23: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

SIZE

X

Y

(x,y)

size of (x,y) = abs(x)+ abs(y)L1 Norm

Page 24: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

SIZE

X

Y

(x,y)

size of (x,y) = max(abs(x), abs(y))L∞ Norm

Page 25: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

UNIT CIRCLE

1

2

Page 26: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

ANGLE

X

Y

(x,y)

(u,v)

Page 27: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

ANGLE

X

Y

(x,y)

(u,v)

(x,y).(u,v) = |(x,y)| |(u,v)| cos ADot Product

A

Page 28: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

ANGLE

X

Y

(x,y)

(u,v)

(x,y).(u,v) = |(x,y)| |(u,v)|Dot Product

Page 29: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

ANGLE

X

Y

(x,y)

(u,v)

(x,y).(u,v) = 0Dot Product

Page 30: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

SIZE AND ANGLE CHEATSHEET

Notation for dot product : or =

L2 norm of a vector :

Distance between two vectors :

Do you see any connection with Statistics?

v · w vTwnX

i=1

viwi

vT v =nX

i=1

v2i = ||v||22

||v � w||2 =

vuutnX

i=1

(vi � wi)2

Page 31: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

ANOTHER LOOK AT 2D

Page 32: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

2D GEOMETRY

(x,y)

x - units

y - units

X

Y

Page 33: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

2D GEOMETRY

(u,v)

u - units

v - units

U

V

Page 34: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

2D GEOMETRY

U

V

X

Y

U = aX + bY

V = cX + dY

Page 35: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

2D GEOMETRY

U

V

X

Y

UV

�=

a bc d

� XY

Page 36: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

TRANSFORMATIONS

Page 37: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

TRANSLATIONpq

Page 38: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

ROTATIONcos ✓ sin ✓� sin ✓ cos ✓

Page 39: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

SCALING↵ 00 �

Page 40: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

GENERICa bc d

Page 41: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

ACTION OF A MATRIX

Page 42: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

MATRIX-VECTOR MULTIPLICATION

x =

M v w

Page 43: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

LINEAR MAP

Mv w

Page 44: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

LINEAR MAP

Mv w

R5 R4

Page 45: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

Rn RmMm⇥n

Domain of the function? — Where does the input originate from?

Codomain of the function? — What are possible values of output?

Range of the function? — What are the actual values of output?

Kernel of the function? — Which inputs map to zero in the range?

Page 46: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

SUBSPACES

Page 47: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

RANGE

+ + + + =

Must be the linear combination of Columns

Page 48: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

KERNEL

=

Must be the orthogonal to all Rows

Page 49: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

Rn RmMm⇥n

Column Space of the Matrix = Linear span of all Columns = Range

Row Space of the Matrix = Linear span of Rows = Non-zero output

Null Space of the Matrix = Map to zero = Orthogonal to Row Space

Page 50: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

Rn RmMm⇥n

ColSpaceRowSpace

Null Space

Page 51: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

LINEAR EQUATIONS

Page 52: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

SOLVABILITY OF LINEAR EQUATION Mx = b

x b

Page 53: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

SOLVABILITY OF LINEAR EQUATION Mx = b

x bSolution always exists.

Solution is unique as Mx = b and My = b implies (x-y) = 0.

Page 54: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

SOLVABILITY OF LINEAR EQUATION Mx = b

xb

Page 55: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

SOLVABILITY OF LINEAR EQUATION Mx = b

xbu

v

Infinitely many solutions.

If Mu = b, then M(u+v) = b, for any v from Null Space.

Page 56: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

SOLVABILITY OF LINEAR EQUATION Mx = b

xb

Page 57: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

SOLVABILITY OF LINEAR EQUATION Mx = b

xb

y

e

If Mx = y, then Mx ~ b + e, with irreducible error e.

Solution does not exist!

Page 58: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

xb

y

e

Rn RmMm⇥n

T

Page 59: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

xb

y

e

Rn RmMm⇥n

T

MTMx = MTb = MTy provides solution x = (MTM)-1MTy

Page 60: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

SOLVABILITY OF LINEAR EQUATION Mx = b

Unique Infinite

Infinite None

Page 61: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

SINGULAR VALUES

Page 62: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

ACTION OF A MATRIX

M

Page 63: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

ACTION OF A MATRIX

M

Page 64: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

ACTION OF A MATRIX

M

u1v1

M v1 = d1 u1

Page 65: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

ACTION OF A MATRIX

M

u1v1

u2

v2

M v1 = d1 u1

M v2 = d2 u2

Page 66: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

SINGULAR VECTORS

M v1 u1v2 v3 v4 v5 u2 u3 u4 u5

d1

d2

d3

d4

d5

x x=

Page 67: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

SINGULAR VECTORS

M

v1

u1

v2

v3

v4

v5

u2 u3 u4 u5

d1

d2

d3

d4

d5

xx=

Page 68: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

SINGULAR VALUE DECOMPOSITION

M

v1

u1

v2

v3

v4

v5

u2 u3 u4

d1

d2 xx=

M = U⌃V T

Page 69: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

M

v1

u1

v2

v3

v4

v5

u2 u3 u4

d1

d2 xx=

ColSpaceRowSpace

v1

v2

v3

v4

v5

u1

u2

u3

u4

M = U⌃V T

Page 70: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

M

v1

u1

v2

v3

v4

v5

u2 u3 u4

d1

d2 xx=

M

Page 71: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

M

v1

u1

v2

v3

v4

v5

u2 u3 u4

d1

d2 xx=

VT

rotation

M

v1

v2

e1

e2

Page 72: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

M

v1

u1

v2

v3

v4

v5

u2 u3 u4

d1

d2 xx=

VT ⌃

rotation scaling

M

e1

e2

d1e1

d2e2

Page 73: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

M

v1

u1

v2

v3

v4

v5

u2 u3 u4

d1

d2 xx=

VT U⌃

rotation rotationscaling

M

d1e1

d2e2

d1u1

d2u2

Page 74: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

M

v1

u1

v2

v3

v4

v5

u2 u3 u4

d1

d2 xx=

VT U⌃

rotation rotationscaling

M

Page 75: LINEAR ALGEBRA - isical.ac.inacmsc/WMSFMLT2016/LinearAlgebra.pdf · PLAN OF ACTION Part I — Geometry of Multivariate Data 1. Vector space modelling of multivariate data 2. Understanding

VT U⌃

rotation rotationscaling

M

REFERENCESGilbert Strang — Lectures in Linear Algebra — MIT OCW 18.06 (or YouTube)Gilbert Strang — The Fundamental Theorem of Linear Algebra — AMM 1993Lloyd Trefethen and David Bau — Numerical Linear Algebra — SIAM 1997