Euclidean Structure - Clemson Universitykevja/COURSES/Math853/NOTES-LAX/s7... · 2018. 6. 14. · Theorem Let A : X !X be an invertible linear map of Euclidean space into itself.

Euclidean Structure

Kevin James

Kevin James Euclidean Structure

Definition

Let X be any n-dimensional Euclidean space (-i.e. vector space over R).

1 Given x ∈ X , we define the length of x denoted ||x || to be thedistance from x to 0. By the Pythagorean theorem we have||x || =

√x21 + · · ·+ x2n .

2 Given x , y ∈ X , we define the inner (or dot) product of x and y as

(x , y) =∑n

i=1 xiyi .

Note

1 ||x ||2 = (x , x),

2 The dot product is symmetric and bilinear,

3 ||x + y ||2 = ||x ||2 + 2(x , y) + ||y ||2,

4 ||x − y ||2 = ||x ||2 − 2(x , y) + ||y ||2. (Note that the last two areindependent of coordinates (-i.e. there is a geometric description).)

5 cos(θ) = (x,y)||x||·||y || .


Definition

A Euclidean Structure in a real vector space is endowed by aninner product, which is symmetric bilinear form with the additionalproperty that (x , x) ≥ 0 with equality if and only if x = 0.

Assumption

Throughout we will assume that X is an n-dimensional real inner-productspace.

Definition

The norm of x ∈ X is ||x || = (x , x)1/2, and the distance betweenx , y ∈ X is ||x − y ||.

Theorem (Cauchy-Schwartz)

For all x , y ∈ X , |(x , y)| ≤ ||x || · ||y ||.

Corollary

For any x , y ∈ X , −1 ≤ (x,y)||x||·||y || ≤ 1.


Definition

Given x , y ∈ X , we define the angle between x and y to be

θ = cos−1(

(x,y)||x||·||y ||

).

Theorem

||x || = max{(x , y) | ||y || = 1}.

Theorem (Triangle Inequality)

For all x , y ∈ X , ||x − y || ≤ ||x ||+ ||y ||.


Definition

Two vectors x , y ∈ X are orthogonal if (x , y) = 0. We write this asx ⊥ y .

Remark (Pathagorean Theorem)

If x ⊥ y , then ||x + y ||2 = ||x ||2 + ||y ||2.

Definition

A subset {x1, · · · , xn} ⊆ X is orthogonal if whenever i 6= j we have(xi , xj) = 0.An orthogonal set is called an orthonormal set if each vector has length 1.

Fact

Any orthogonal set is linearly independent.


Theorem (Gram-Schmidt process)

Suppose W ≤ X has a basis {x1, . . . , xp}. Then,

y1 = x1 and v1 =1

||y1||y1

y2 = x2 − (x2, v1)v1 and v2 =1

||y2||y2

y3 = x3 − (x3, v1)v1 − (x3, v2)v2 and v3 =1

||y3||y3

...

yp = xp − (xp, v1)v1 − · · · − (xp, vp−1)vp−1 and vp =1

||yp||yp

is an orthonormal basis for W .Also,

Span(v1, . . . , vk) = Span(x1, . . . , xk)

for 1 ≤ k ≤ p.


Remark

An orthogonal matrix is a square matrix whose columns and rows areorthogonal unit vectors (i.e., orthonormal vectors), i.e.

QTQ = QQT = I .

In particular, an orthonormal basis {u1, · · · , un} for Rn forms anorthogonal matrix

[u1u2 · · · un].


Remark

Any real square matrix A may be decomposed as A = QR, where Q is anorthogonal matrix and R is an upper triangular matrix.

This is called the QR-factorization of A.

In particular, the Gram-Schmidt theorem gives a QR-factorization ofA = [x1x2 · · · xn], where x1, · · · , xn is a basis of X ,

[x1x2 · · · xn] = [u1u2 · · · un]

(x1, u1) (x2, u1) (x3, u1) · · ·

0 (x2, u2) (x3, u2) · · ·... 0 (x3, u3) · · ·...

... 0. . .


Remark

Any n-dimensional Euclidean space with an inner product is isomorphicto Rn with the standard dot product.

If x1, · · · , xn is an orthonormal basis, then for any x ∈ X we can write

x =n∑

j=1

ajxj ,

where aj = (x , xj). Moreover, if y =n∑

k=1

bkxk , then

(x , y) =

n∑j=1

ajxj ,n∑

k=1

bkxk

=n∑

j=1

(ajxj , bjxj) =n∑

j=1

ajbj .

The mapping X → Rn, x 7→ (a1, · · · , an), where aj = (x , xj), is anisomorphism that carries the inner product of X to the standard dotproduct of Rn.


Theorem

Every linear function ` ∈ X ′ can be written as `(x) = (x , y) for somey ∈ X . The mapping ` 7→ y is an isomorphism of the Euclidean space Xwith its dual.

Proof.

Let x1, · · · , xn be an orthonormal basis, and let bk = `(xk). Put

y =n∑

k=1

bkxk .

One can check for all 1 ≤ k ≤ n, we have `(xk) = (xk , y), since

(xk , y) =

(xk ,

n∑k=1

bkxk

)= bk = `(xk).

If two linear functions have the same value for all vectors that form abasis, they have the same values for all vectors x .


Definition

Let Y be a subspace of X . The orthogonal complement of Y is the set

{x ∈ X : (x , y) = 0 for any y ∈ Y }. We denote it by Y⊥.

Remark

This is consistent with our previous definition of

Y⊥ = {` ∈ X ′ : (`, y) = 0 for any y ∈ Y }.

Theorem

For any subspace Y of X , we have X = Y ⊕ Y⊥.

Definition

The map PY : X → X , x = y + y⊥ 7→ y , is called theorthogonal projection of x into Y .


Theorem

PY is linear and idempotent (i.e., P2Y = PY ).

Theorem

Let Y ⊂ X be a subspace. Then,

||x − PY (x)|| = min{||x − w || : w ∈ Y }.

Proof.

Suppose that x ∈ X .Write x = y + y⊥ (-i.e. y = PY (x) ).Then for any w ∈ Y , we have x − w = (y − w) + y⊥.Applying the Pythagorean theorem, we have

||x − w ||2 = ||y − w ||2 + ||y⊥||2 ≥ ||y⊥||2

with equality when w = y = PY (x).Thus ||x − PY (x)|| = ||y⊥|| = min{||x − w || : w ∈ Y }.


Definition

Let A : X → U be a linear map between Euclidean spaces. For anyu ∈ U, `(x) = (Ax , u) is a linear function X → R. By our previoustheorem, for some y ∈ X , `(x) = (x , y) = (Ax , u).

The vector y ∈ X depends on u ∈ U, i.e., for some function A∗ : U → X ,y = A∗u. Thus, we have

(x ,A∗u) = (Ax , u).

We call A∗ the adjoint of A.

Note

If A ∈ Rm×n, and X = Rn,Y = Rm, and we identify X ′ and Y ′ withR1×n and R1×m respectively, we have A∗ =T A.


Theorem

1 If A,B : X → U are linear, then (A + B)∗ = A∗ + B∗.

2 If A : X → U, C : U → V are linear, then (CA)∗ = A∗C∗.

3 If A : X → X is 1− 1, then (A−1)∗ = (A∗)−1.

4 (A∗)∗ = A.

Proof.

Let x ∈ X , u ∈ U, v ∈ V .

1 ((A + B)x , u) = (Ax , u) + (Bx , u) = (x ,A∗u) + (x ,B∗u)= (x , (A∗ + B∗)u).

2 (CAx , v) = (Ax ,C∗v) = (x ,A∗C∗v).

3 I = I ∗ = (A−1A)∗ = A∗(A∗)−1.

4 (Ax , u) = (u,Ax) = (A∗u, x) = (u, (A∗)∗x) = ((A∗)∗x , u).


Definition

Let A be an m × n matrix, where m > n. Let p = (p1, · · · , pm), wherep1, · · · , pm are the measured values. The system of equations given byAx = p where the number m of measurements exceeds the number n ofquantities in general does not have a solution. Such a system ofequations is overdetermined.

Remark

In such a situation, we seek that vector x that comes closest to satisfyingall the equations in the sense that makes ||Ax − p||2 as small as possible.This is called a least square problem.


Theorem

Let A be an m× n matrix, m > n, and suppose that A has only the trivialnullvector ~0. The vector x that minimizes ||Ax − p||2 is the solution z of

A∗Az = A∗p.

Proof.

Take p = PRA(p).

Then ∃z such that Az = p and z minimizes ||Ax − p||.Also (p − p) ⊥ RA.Thus ∀x ∈ X , we have 0 = ((p − p),Ax) = (A∗(p − Az), x).Thus A∗p − A∗Az = 0.This proves that the solution z to A∗Az = A∗p minimizes ||Ax − p|| asdesired.Suppose that z ′ is another solution to the above equation.Then, taking y = z − z ′ we have that A∗Ay = 0Thus 0 = (A∗Ay , y) = (Ay ,Ay) = ||y ||2.Thus z − z ′ = y = 0.


Theorem

If PY is the orthogonal projection onto Y , the

P∗Y = PY .

Proof.

For all w , y ∈ Y , we have

(y ,P∗Y (w)) = (PY (y),w) = (y ,w) = (y ,PY (w)).

Thus, P∗Y (w) = PY (w) for all w ∈ Y .

Definition

A function M : X → X is an isometry if for all x , y ∈ X ,||Mx −My || = ||x − y ||. (“Distance-preserving.”)

Example

Any translation Mx = x + a is an isometry.


Theorem

Let M be an isometric mapping of a Euclidean space into itself such thatM(0) = 0. Then,

1 M is linear

2 M∗M = I . Conversely if M∗M = I then M is an isometry.

3 M is invertible and M−1 is an isometry.

4 det(M) = ±1.

Definition

A matrix that maps Rn onto itself isometrically is called orthogonal.The orthogonal matrices (for fixed n) form a group under matrixmultiplication called the orthogonal group denoted O(n).The subgroup of matrices with determinant 1 is called thespecial orthogonal group, denoted SO(n).


Proposition

A matrix M is orthogonal iff its columns form an orthonormal set.

Definition

We define the norm of a map A : X → U as ||A|| = sup||x||=1 ||Ax ||.

Theorem

Let A : X → U be a mapping of Euclidean spaces. Then,

1 ||Az || ≤ ||A||||z || for all z ∈ X .

2 ||A|| = sup||x||=1;||v ||=1(Ax , v).


Theorem

Suppose we have linear maps A,B : X → U, C : U → U of Euclideanspaces. Then,

1 ||kA|| = |k |||A||,2 ||A + B|| ≤ ||A||+ ||B||,3 ||CA|| ≤ ||C || · ||A||,4 ||A∗|| = ||A||.

Theorem

Let A : X → X be an invertible linear map of Euclidean space into itself.Suppose B : X → X has the property that ||A− B|| < 1

||A−1|| . Then B is

invertible.

Note

We have assumed throughout the proof that dim(X ) <∞. However, theresult also holds when dim(X ) =∞.


Definition

A sequence of vectors xk in a Euclidean space X converges to the limit xwritten limk→∞ xk = x if ||x − xk || → 0 as k →∞.

Definition

A sequence of vectors {xk} in a Euclidean space X is called aCauchy sequence if ||xk − xj || → 0 as k , j →∞.A sequence {xk} of vectors in a Euclidean space X is called bounded if||xk || ≤ R for all k where R is some real number.

Theorem

Let X be a finite dimensional Euclidean space.

1 X is complete (-i.e. all Cauchy sequences converge in X ).

2 X is locally compact (-i.e. every bounded sequence contains aconvergent subsequence).


Remark

We defined ||A|| = sup{||Ax || | ||x || = 1} but in this case its justmax{||Ax || | ||x || = 1}. Take a subsequence {xk} (with ||xk || = 1) forwhich ||Axk || → ||A||. by theorem 7.16 this sequence has a subsequence{xki} that converges to some x ∈ X . Now, ||Ae|| ≤ ||Ax || for all unitvectors e ∈ X . So, ||A|| = ||Ax ||.

Theorem

If a Euclidean space X is locally compact then dimX <∞.


Definition

A sequence {Ak} of maps X → U converges to a limit A iflimk→∞ ||An − A|| = 0.

Proposition

If dimX <∞, then An → A iff Anx → Ax for all x ∈ X .

Remark

This is not true if dimX =∞.


Definition

Suppose that X is a finite dimensional vector space over C. Forz ,w ∈ X , we define (z ,w) =T wz =:H yx .

Proposition

1 Linear in x (-i.e. (kx , y) = k(x , y)).

2 Skew linear in y (-i.e. (x , ky) = k(x , y)).

3 Skew symmetric: (x , y) = (y , x).

4 (x , x) > 0 for all x 6= 0. (positivity)

5 ||x + y ||2 = ||x ||2 + 2Re((x , y)) + ||y ||2.

Definition

The adjoint of a linear map A : X → U is defined to be the mapA∗ : U → X such that (x ,A∗u) = (Ax , u), ∀x ∈ X ; u ∈ U.


Definition

1 We define the adjoint as A∗ =H A =T(A).

2 We say the set {qk} is orthonormal if Hqiqj = δij .

3 An isometry of a complex Euclidean space is a map M : X → Xsuch that ||Mx −My || = ||x − y ||, ∀x , y ∈ X (as before). Such amap is also called unitary.

Proposition

If M : X → X is unitary, then

1 M∗M = I ,

2 M−1 is unitary,

3 The set of untiary maps of X forms a group,

4 | det(M)| = 1.

Definition

The norm of a linear map between complex spaces is defined as in thereal case: ||A|| = sup||x||=1 ||Ax ||.


Euclidean Structure - Clemson Universitykevja/COURSES/Math853/NOTES-LAX/s7... · 2018. 6. 14. · Theorem Let A : X !X be an invertible linear map of Euclidean space into itself.

Documents