E 600 Chapter 1: Introduction to Matrix Algebra · Chapter 1: Introduction to Matrix Algebra Simona Helmsmueller August 9, 2017. PreviewIntroduction The vector space Mn m Matrices

Preview Introduction The vector space Mn×m Matrices and systems of linear equations Linear functions and matrices

Chapter 1: Introduction to Matrix Algebra

Simona Helmsmueller

August 9, 2017

Goals of this lecture:

• Be able to apply basic algebraic operations and to name

special matrices

• Be able to apply the GaußJordan algorithm

• Know how to calculate the determinant of a square matrix

and why it is important in the context of linear systems of

equations

• Have a (albeit vague) geometric intuition for the rank

condition

• Know the definition of definiteness of a matrix

• Know the definition of eigenvectors and -values

Following lectures (both in this class and other

courses) will assume these goals have been reached!

Introduction

x1 + x2 + x3 = 6

x2 − x3 = 0

x1 + x2 + x3 = 3

2x1 = 6− 2x2 − 2x3

Introduction

x1 + x2 + x3 = 6

x2 − x3 = 0

x1 + x2 + x3 = 3

2x1 = 6− 2x2 − 2x3

Introduction

x1 + x2 + x3 = 6

x2 − x3 = 0

x1 + x2 + x3 = 3

2x1 = 6− 2x2 − 2x3

1. Does a solution exist?

2. Is the solution unique / how many solutions are there?

3. How can we (or our computer) efficiently derive at the (set

of) solutions?

An×m =

a1,1 a1,2 · · · a1,ma2,1 a2,2 · · · a2,m

......

. . ....

an,1 an,2 · · · an,m

= (ai ,j)i=1,...,n;j=1,...,m

Definition(Equality of Matrices)

Two matrices are said to be equal if and only if:

(i) they have the same dimension

(ii) their corresponding elements are equal

In symbols, if An×m = (ai ,j)i=1,...,n;j=1,...,m and

Br×s = (bi ,j)i=1...r ,j=1...s , then

A = B ⇔ (n = r, m = s, and, ∀ i = 1, ...,n(= r),

j = 1, ...,m(= s) ai,j = bi,j).

Definition(Addition of Matrices)

For two matrices of identical dimension, we define addition of

matrices in terms of addition of their corresponding elements. In

symbols, if An×m = (ai ,j)i=1,...,n;j=1,...,m and

Bn×m = (bi ,j)i=1,...,n;j=1,...,m, then their sum, denoted by A + B, is

A + B = (ai ,j + bi ,j)i=1,...,n;j=1,...,m.

Definition(Multiplication of a Matrix by a Scalar)

Let λ ∈ R. Then to multiply a matrix by this scalar, we multiply

each element of the matrix by this scalar. In symbols, let

An×m = (ai ,j)i=1,...,n;j=1,...,m be any matrix, then:

λA := (λai ,j)i=1,...,n,j=1,...,m

Theorem( Vector Space of Matrices)

The set of all n ×m matrices, Mn×m together with the algebraic

operations matrix addition and multiplication with a scalar as

defined above defines a vector space.

Definition(Row and Column Vectors) Matrices containing only one row

(n = 1) are called row vectors, and similarly, matrices containing

only one column (m = 1) are called column vectors. In matrix

algebra, if not specified otherwise, by vector we conventionally

mean a column vector.

Definition(Square Matrix)

A matrix of size n ×m is said to be a square matrix if and only if

n = m.

Definition(Diagonal Matrix)

A square matrix is said to be a diagonal matrix is all of its

off-diagonal elements are zero.

Definition(Upper and Lower Triangular Matrix)

An upper triangular matrix is square and has characteristic

elements ai ,j equal to zero whenever i > j and a lower triangular

matrix is square and has characteristic elements ai ,j equal to zero

whenever i < j . Less formally, in the same way that we call a

matrix diagonal if non-diagonal entries are restricted to be null, we

call upper- (lower-) diagonal a matrix whose entries that do not

belong to the part above (below) the diagonal are restricted to be

Definition(Symmetric Matrix)

A square matrix is said to be a symmetric matrix if and only if

ai ,j = aj ,i∀1 ≤ i , j ≤ n.

Definition(Identity Matrices)

The n × n identity matrix, denoted In, is a diagonal matrix with all

its diagonal elements equal to 1.

Definition(Zero Matrices)

The n × n zero matrix, denoted 0n×n, is a square matrix with all

its elements equal to 0.

Definition(Transpose of a matrix)

The transpose of a matrix is obtained by reflecting all elements of

a matrix over its main diagonal. In symbols, let

An×m = (ai ,j)i=1,...,n;j=1,...,m be any matrix, then its transpose,

denoted A′m×n or ATm×n, is such that:

A′m×n := (aj ,i )j=1...m,1=1...n

Definition(Product of Matrices)

For two conformable matrices, their product is defined as the

matrix with ijth element equal to the inner product of the ith row

of the first matrix and the jth column of the second matrix. In

symbols, if An×m = (ai ,j)i=1,...,n;j=1,...,m is an n ×m matrix and

Bm×k = (bi ,j)i=1...m,j=1...k is an m × k matrix, the product of

An×m and Bm×k is the n × k matrix Cn×k with characteristic

element:

ci ,j =m∑l=1

ai ,lbl ,j

Theorem( Associativity and Distributivity of the Product)

The product for matrices is:

(i) Associative: (AB)C = A(BC)

(ii) Distributive over matrix addition: A(B + C) = AB + AC and

(A + B)C = AC + BC

Theorem( Transposition, sum, and product)

(i) If A and B are n ×m matrices, then:

(A + B)′ = A′ + B′

(ii) If A is an n ×m matrix and B is an m × k matrix, then:

(AB)′ = B′A′

(iii) If A is 1× 1 matrix, then A is actually a scalar and A′ = A.

Definition(Inverse Matrix)

Let A be an n × n matrix. If there exists a matrix A−1 such that:

A−1A = AA−1 = In

Then A is said to be invertible and A−1 is called the inverse matrix

TheoremA square matrix A can have at most one inverse.

TheoremLet An and Bn be invertible. Then the following holds:

1. (AT )−1

= (A−1)T

2. AB is invertible and (AB)−1 = B−1A−1.

3. for any scalar λ 6= 0, λA is invertible and (λA)−1 =1

λA−1.

TheoremIf a square matrix An is invertible, then the unique solution to the

system of linear equations Anx = b is x = A−1b.

The Gauß- Jordan algorithm

DefinitionLet

An,n =

a1,1 a1,2 · · · a1,na2,1 a2,2 · · · a2,n

......

. . ....

an,1 an,2 · · · an,n

detA =n∑

a1,jC1,j

Where C1,j := (−1)1+jA1,j is known as the (1, j)th cofactor of A,

and A1,j, known as the (1, j)th first minor, is the determinant of

the (n− 1)× (n− 1) formed out of A by deleting the first row and

the jth column.

Theorem( Determinant of the Product)

For any two n× n matrices A and B we have

det(AB) = det(A)det(B).

TheoremLet A be a square matrix. Then

A−1 exists ⇔ det(A) 6= 0.

Definition(Rank of a Matrix)

Let A be an n ×m matrix. Then, the rank of A, Rank(A), is

defined as the number of linearly independent columns of A. If

n = m and Rank(A)=n, then we say that A has full rank.

Theorem( Rank condition)

If A has full rank1, then for any b in Rn, the system Ax = b has a

unique solution.

Definition(Column Space)

The subset of Rn spanned by the columns of a matrix

A = [a1, ..., an] is called the column space of A:

Col(A) = Span(a1, ..., an).

Theorem

dimCol(A) = rankA

Definition(Column Space)

The subset of Rn spanned by the columns of a matrix

A = [a1, ..., an] is called the column space of A:

Col(A) = Span(a1, ..., an).

Theorem

dimCol(A) = rankA

TheoremLet A be n ×m matrix. Then:

1. The system of linear equations represented by Ax = b has a

solution for a particular b ∈ Rn if and only if b ∈ Col(A).

2. The system of linear equations represented by Ax = b has a

solution for every b ∈ Rn if and only if rankA = n.

3. If the system of linear equations represented by Ax = b has a

solution for every b ∈ Rn, then:

n = rankA ≤ number columns of A = m.

Definition(Linear Function)

Let X and Y be two vector spaces. A function f : X→ Y is said to

be linear if and only if:

n∑i=1

λi f (xi ) ∀n ∈ N, λi ∈ R, and xi ∈ X .

Definition(Definiteness of a Matrix)

A (symmetric) n × n matrix A is:

(i) positive semidefinite if and only if x ′Ax ≥ 0 for all x ∈ Rn.

(ii) negative semidefinite if and only if x ′Ax ≤ 0 for all x ∈ Rn.

(iii) positive definite if and only if x ′Ax > 0 for all x ∈ Rn.

(iv) negative definite if and only if x ′Ax < 0 for all x ∈ Rn.

Definition((Leading) principal minors of a matrix)

Let A be an n × n matrix.

(i) Any k × k submatrix of A formed by deleting n − k rows and

the corresponding n − k columns is called a kth order principal

submatrix of A. The determinant of the kth order principal

submatrix of A is called a kth order principal minor of A.

(ii) The k × k submatrix of A formed by deleting the last n − k

rows and columns of A is called the kth order leading principal

submatrix of A, denoted by Ak . Its determinant is called the kth

order leading principal minor of A, denoted |Ak |.

Theorem( Definiteness of a matrix)

Let A be an n × n symmetric matrix. Then,

(i) A is positive semidefinite if and only if all its principal minors

are non negative.

(ii)A is positive definite if and only if all its n leading principal

minors are strictly positive.

(iii)A is negative semidefinite if and only if, for all k , its kth order

principal minor have the same sign as (−1)k or are 0. (Put in

other words: A is negative semidefinite if and only if every

principal minor of odd order is ≤ 0 and every principal minor of

even order is ≥ 0.)

(iv) A is negative definite if and only if, for all k, its kth order

leading principal minor have the same sign as (−1)k .

Definition(Eigenvectors and -values)

Let A be a square n× n matrix, then, a vector x ∈ Rn is said to be

an eigenvector of A if and only if there exists a λ in R such that:

Ax = λx

λ is then called an eigenvalue of A.

Theorem( Definitness of a Matrix)

Let A be an n × n symmetric matrix. Then:

(i) A is positive definite if and only if all its n eigenvalues are

strictly positive.

(ii) A is negative definite if and only if all its n eigenvalues are

strictly negative.

(iii) A is positive semidefinite if and only if all its n eigenvalues are

non-negative.

(iv) A is negative semidefinite if and only if all its n eigenvalues are

non-positive.

E 600 Chapter 1: Introduction to Matrix Algebra · Chapter 1: Introduction to Matrix Algebra Simona Helmsmueller August 9, 2017. PreviewIntroduction The vector space Mn m Matrices

Documents

Modern Matrix Algebra

Introductory matrix algebra

Chapter 5 MATRIX ALGEBRA: DETEMINANT, REVERSE, EIGENVALUES

Chapter 10 Review: Matrix Algebra Introduction to MATLAB 7.....

Chapter 1 Matrix Algebra Review - University of...

LINEAR ALGEBRA, VECTOR ALGEBRA AND ANALYTICAL GEOMETRY ·.....

Chapter 7: Matrix Algebra 1.(7.1) Matrix Arithmetic...

Chapter 1 Matrix Algebra Review - University of...

Chapter 2 A short review of matrix algebra

2 Matrix algebra: a summary - Université de...

FP1: Chapter 4 Matrix Algebra

Advanced Matrix Algebra

Chapter 6 - Matrix Algebra

GG 313 Lecture 11 Chapter 3 Linear (Matrix) Algebra Sept 27,...

Chapter 6: Linear Equations and Matrix Algebra The All...

Chapter 1 Matrix Algebra. Deﬁnitions and...