Top Banner
. TOPIC- ORTHOGONAL SET & BASIS THE GRAM- SCHMIDT PREPARED BY RAJESH GOSWAMI
21

Orthogonal basis and gram schmidth process

Jan 22, 2018

Download

Engineering

Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Orthogonal basis and gram schmidth process

.TOPIC- ORTHOGONAL SET & BASIS THE GRAM-SCHMIDT

PREPARED BY RAJESH GOSWAMI

Page 2: Orthogonal basis and gram schmidth process

Outline

Orthogonal Sets Orthogonal basis The Gram-Schmidt Orthogonalization Process

Page 3: Orthogonal basis and gram schmidth process

Slide 6.2- 3 © 2012 Pearson Education, Inc.

ORTHOGONAL SETS Proof: If for some scalars c1,…,cp, then

because u1 is orthogonal to u2,…,up.

Since u1 is nonzero, is not zero and so .

Similarly, c2,…,cp must be zero.

1 10 u up pc c= + +L

1 1 1 2 2 1

1 1 1 2 2 1 1

1 1 1 2 2 1 1

1 1 1

0 0 u ( u u u ) u

( u ) u ( u ) u ( u ) u

(u u ) (u u ) (u u )

(u u )

p p

p p

p p

c c c

c c c

c c c

c

= = + + +

= + + +

= + + +

=

Lg gLg g gLg g g

g

1 1u ug 1 0c =

Page 4: Orthogonal basis and gram schmidth process

Slide 6.2- 4 © 2012 Pearson Education, Inc.

ORTHOGONAL SETS

Thus S is linearly independent. Definition: An orthogonal basis for a subspace W

of is a basis for W that is also an orthogonal set.

Theorem 5: Let {u1,…,up} be an orthogonal basis for a subspace W of . For each y in W, the weights in the linear combination

are given by

1 1y u up pc c= + +L

y u

u uj

j

j j

c =gg

( 1, , )j p= K

Page 5: Orthogonal basis and gram schmidth process

Slide 6.2- 5 © 2012 Pearson Education, Inc.

ORTHOGONAL SETS

Proof: The orthogonality of {u1,…,up} shows that

Since is not zero, the equation above can be solved for c1.

To find cj for , compute and solve for cj.

1 1 1 2 2 1 1 1 1y u ( u u u ) u (u u )p pc c c c= + + + =Lg g g

1 1u ug

2, ,j p= K y u jg

Page 6: Orthogonal basis and gram schmidth process

.

Orthogonal basis: A basis consisting of orthogonal vectors in an inner product space.

Page 7: Orthogonal basis and gram schmidth process

ORTHOGONAL BASIS If S ={v1, v2 , … , vn} is an orthogonal basis of W, then for any w ∈ W,

where

are called the Fourier coefficients.

So the coordinate vector of w,

rw =

⟨ rw,

rvi ⟩

⟨rvi ,

rvi ⟩i=1

n

∑ rvi =

⟨ rw,

rv1⟩

⟨rv1,

rv1⟩

rv1 +

⟨ rw,

rv2 ⟩

⟨rv2 ,rv2 ⟩

rv2 + ...

⟨ rw,

rvn ⟩

⟨rvn ,

rvn ⟩

rvn ,

rwS = (

rw)S =

⟨ rw,

rv1⟩

⟨rv1,

rv1⟩

,⟨ rw,

rv2 ⟩

⟨rv2 ,

rv2 ⟩

, ... ,⟨ rw,

rvn ⟩

⟨rvn ,

rvn ⟩

.

⟨ rw,

rv1⟩

⟨rv1,rv1⟩

,⟨ rw,

rv2 ⟩

⟨rv2 ,rv2 ⟩

, ... ,⟨ rw,

rvn ⟩

⟨rvn ,

rvn ⟩

Page 8: Orthogonal basis and gram schmidth process

8

How to Find the Coordinate Vector with Respect to a Given Orthogonal Basis?

Example : Compute the coefficients and determine the coordinate vectors in Example 1 for u = (10,3).

From Example 1, we have v1 = (5,0), v2 = (0,-3) and

In this case, the coefficients are:

⟨ ru,

rv1⟩

⟨rv1,

rv1⟩

=ru ⋅ r

v1rv1

2 =(10)(5) + (3)(0)

52=

50

25= 2

⟨ ru,

rv2 ⟩

⟨rv2 ,

rv2 ⟩

=ru ⋅ r

v2rv2

2 =(10)(0) + (3)(−3)

32=

−9

9= −1

rv1 = 5, and

rv2 = 3.

Page 9: Orthogonal basis and gram schmidth process

How to Find the Coordinate Vector with Respect to a Given Orthogonal Basis?

So the coordinate vector of u,

We can see that a nice advantage of working with an orthogonal basis is that the coefficients in any basis representation for a vector are immediately known; they are called Fourier coefficients.

ruS = (

ru)S =

⟨ ru,

rv1⟩

⟨rv1,rv1⟩

,⟨ ru,

rv2 ⟩

⟨rv2 ,

rv2 ⟩

= (2,−1).

Page 10: Orthogonal basis and gram schmidth process

Properties of orthogonal matrices:

If is orthogonal, thenn nQ ×∈ ℜ

1

2 2

( ) The column vectors of form an orthonormal

basis for .

( )

( )

( ) , , preserve inner product

( ) preserve norm

( ) preserve angle

n

i Q

ii Q Q I QQ

iii Q Q

iv Qx Qy x y

v Qx x

vi

Τ Τ

Τ −

ℜ= =

== ←

= ←

Page 11: Orthogonal basis and gram schmidth process

Gram-Schmidt Orthogonalization Process

Page 12: Orthogonal basis and gram schmidth process

Cram-Schmidt Orthogonalization Process

Question: Given an ordinary basis ,

how to transform them into an orthonormal

basis ?

{ }1 2, ,..., nx x x

{ }1 2, ,..., nu u u

Page 13: Orthogonal basis and gram schmidth process

Given ,Clearly

Clearly,

Similarly,

Clearly, We have the next result

1 nx x

1 11

1u x

x 1 1{ } { }span u span x=

1 2 1 1 2 2 12 1

1, , ( )p x u u u x p

x p= −

1 2 1 2 1 2& { , } { , }u u span x x span u u⊥ =

2 3 1 1 3 2 2

3 3 23 2

, ,

1( )

p x u u x u u

and u x px p

= +

−−

3 1 3 2 1 2 3 1 2 3, & { , , } { , , }u u u u span x x x span u u u⊥ ⊥ =

1u

1p

2x

Page 14: Orthogonal basis and gram schmidth process

Theorem: (The Gram-Schmidt process)

H. (i) Let be a basis for an inner

product space .

(ii)

C. is an orthonormal basis.{ }1 nu u

{ }1 nx x

V

( )

1 11

1 11

11

1,

1, 1, , 1

,

K K KK K

K

K K j jj

u xx

u x p K nx p

where p x u u

+ ++

+=

=

= − = −−

= ∑

Page 15: Orthogonal basis and gram schmidth process

Example: Find an orthonormal basis for with

inner product given by

, where

Sol: Starting with a basis

3P

),()(,3

1i

ii xgxPgP ∑

=

=

.1&0,1 321 ==−= xxx

{ }2,,1 xx

{ }1 2 1

1 2

11 1 1 11 1

Let , ,..., be the projection vectors defines in Thm. 5.6.1, and

let , ,..., be the orthonormal basis of ( ) derived from the

Gram-Schmidt process.

Define

n

n

kk

p p p

q q q R A

r a a r q

r

= ⇒ = ⋅

=

1 for 2,...,

and for 1,..., 1 by the Gram-Schmidt process.

k k

Tik i k

a p k n

r q a i k

− == = −

Page 16: Orthogonal basis and gram schmidth process

Theorem: (QR Factorization)

If A is an m×n matrix of rank n, then A

can be factored into a product QR, where Q

is an m×n matrix with orthonormal columns

and R is an n×n matrix that is upper triangular

and invertible.

Page 17: Orthogonal basis and gram schmidth process

Proof. of QR-Factorization

{ }1 2 1

1 2 1

11 1

Let , ,..., be the projection vectors defined in Thm.5.6.1,

and let , ,..., be the orthonormal basis of ( ) derived from

the Gram-Schmidt process.

Define

n

n

kk k k

p p p

q q q R A

r a

r a p

=

@ 1

1 11 1

2 12 1 22 2

1 1

and for 1,... -1 for 2,...,

By the Gram-Schmidt process,

Tik i k

n n

r q a i kk n

a r q

a r q r q

a r q

= ==

== +

= +

M M

... nn nr q+

Page 18: Orthogonal basis and gram schmidth process

Proof. of QR-Factorization

1 2

11 12 1

22 2

If we set ( , ,..., ) and define to be the upper triangular matrix

0 ,

0 0

then the th column of the product wi

n

n

n

nn

Q q q q R

r r r

r rR

r

j QR

=

=

M M O M

1 1 2 2

1 2

ll be

... for 1,... .

Therefore,

( , ,..., )

j j j jj j j

n

Qr r q r q r q a j n

QR a a a A

= + + + = =

= =

Page 19: Orthogonal basis and gram schmidth process

Theorem:

If A is an m×n matrix of rank n, then the

solution to the least squares problem

is given by , where Q and R are the

matrices obtained from Thm.5.6.2. The solution

may be obtained by using back substitution to solve .

Ax b=vv

1x̂ R Q b− Τ=vv

x̂v

ˆRx Q bΤ=vv

Page 20: Orthogonal basis and gram schmidth process

Proof. of Thm

ˆLet be the solution to the leaset squares problem

ˆ

ˆ

ˆ ( ) ( ) ( )

ˆ ( )

TAT T

T T

T T

I

x

Ax b

A Ax A b

QR QRx QR b QR Factorization

R Q Q R

×

=

=

⇒ =

⇒ = −

v

vv

vv

vv

v

1

ˆ ( is invertible)

ˆ ˆ or

T T

T T T

T

x R Q b

R Rx R Q b R

Rx Q b x R Q b− Τ

=

⇒ =

⇒ = =

v

vv

v vv v

Page 21: Orthogonal basis and gram schmidth process

Example : Solve

By direct calculation,

1

2

3

1 2 1 1

2 0 1 1

2 4 2 1

4 0 0 2bA

x

x

x

− − − = − −

v

R

Q

QRA

==200

140

125

1

2

2

4

2

4

1

2

4

2

2

1

5

1

1

1

2

Q bΤ−

= −

v

The solution can be obtained from

5 2 1 1

0 4 1 1

0 0 2 2

⇒− −

− −