Refresher: Vector and Matrix Algebra

Refresher:Refresher:Vector and Matrix AlgebraVector and Matrix Algebra

Mike KirkpatrickMike Kirkpatrick

Department of Chemical EngineeringDepartment of Chemical Engineering

FAMU-FSU College of EngineeringFAMU-FSU College of Engineering

OutlineOutline

Basics:Basics:• Operations on vectors and matricesOperations on vectors and matrices

Linear systems of algebraic equationsLinear systems of algebraic equations• Gauss eliminationGauss elimination• Matrix rank, existence of a solutionMatrix rank, existence of a solution• Inverse of a matrixInverse of a matrix• DeterminantsDeterminants

Eigenvalues and EigenvectorsEigenvalues and Eigenvectors• applicationsapplications• diagonalizationdiagonalization

moremore

Outline cont’Outline cont’

Special matrix propertiesSpecial matrix properties• symmetric, skew-symmetric, and symmetric, skew-symmetric, and

orthogonal matricesorthogonal matrices• Hermitian, skew-Hermitian, and unitary Hermitian, skew-Hermitian, and unitary

matricesmatrices

MatricesMatrices A matrix is a rectangular array of numbers (or A matrix is a rectangular array of numbers (or

functions). functions).

The matrix shown above is of size mThe matrix shown above is of size mxxn. Note that n. Note that this designates first the number of rows, then the this designates first the number of rows, then the number of columns.number of columns.

The elements of a matrix, here represented by the The elements of a matrix, here represented by the letter ‘a’ with subscripts, can consist of numbers, letter ‘a’ with subscripts, can consist of numbers, variables, or functions of variables.variables, or functions of variables.

mnm

n

aa

aa

aaa

1

2221

11211

VectorsVectors

A vector is simply a matrix with either one row or A vector is simply a matrix with either one row or one column.one column. A matrix with one row is called a row A matrix with one row is called a row vector, and a matrix with one column is called a vector, and a matrix with one column is called a column vector.column vector.

Transpose: A row vector can be changed into a Transpose: A row vector can be changed into a column vector and vice-versa by taking the column vector and vice-versa by taking the transposetranspose of that vector. e.g.: of that vector. e.g.:

5

4

3

543 TAthenAif

Matrix AdditionMatrix Addition

Matrix addition is only possible between Matrix addition is only possible between two matrices which have the same size.two matrices which have the same size.

The operation is done simply by adding The operation is done simply by adding the corresponding elements. e.g.:the corresponding elements. e.g.:

87

57

13

26

74

31

Matrix scalar multiplicationMatrix scalar multiplication

Multiplication of a matrix or a vector Multiplication of a matrix or a vector by a scalar is also straightforward:by a scalar is also straightforward:

3520

155

74

31*5

Transpose of a matrixTranspose of a matrix

028

573

641

,

056

274

831TAthenAif

Taking the transpose of a matrix is similar Taking the transpose of a matrix is similar to that of a vector:to that of a vector:

The diagonal elements in the matrix are The diagonal elements in the matrix are unaffected, but the other elements are unaffected, but the other elements are switched. A matrix which is the same as switched. A matrix which is the same as its own transpose is called its own transpose is called symmetricsymmetric, and , and one which is the negative of its own one which is the negative of its own transpose is called transpose is called skew-symmetricskew-symmetric..

Matrix MultiplicationMatrix Multiplication The multiplication of a matrix into another matrix not The multiplication of a matrix into another matrix not

possible for all matrices, and the operation is possible for all matrices, and the operation is not not commutativecommutative::

AB ≠ BA in generalAB ≠ BA in general In order to multiply two matrices, the first matrix must In order to multiply two matrices, the first matrix must

have the same number of columns as the second have the same number of columns as the second matrix has rows.matrix has rows.

So, if one wants to solve for C=AB, then the matrix A So, if one wants to solve for C=AB, then the matrix A must have as many columns as the matrix B has rows.must have as many columns as the matrix B has rows.

The resulting matrix C will have the same number of The resulting matrix C will have the same number of rows as did A and the same number of columns as did rows as did A and the same number of columns as did B.B.

Matrix MultiplicationMatrix Multiplication The operation is done as follows: The operation is done as follows:

using index notation:using index notation:

for example:for example:

lk

n

l jljk BAC

1

4518

4716

3811

6*05*91*02*9

6*25*71*22*7

6*35*41*32*4

61

52

09

27

34

AB

Linear systems of equationsLinear systems of equations One of the most important application of One of the most important application of

matrices is for solving linear systems of matrices is for solving linear systems of equations which appear in many different equations which appear in many different problems including electrical networks, problems including electrical networks, statistics, and numerical methods for statistics, and numerical methods for differential equations.differential equations.

A linear system of equations can be written:A linear system of equations can be written:

aa1111xx11 + … + a + … + a1n1nxxnn = b = b11

aa2121xx11 + … + a + … + a2n2nxxnn = b = b22

::

aam1m1xx11 + … + a + … + amnmnxxnn = b = bmm

This is a system of m equations and n This is a system of m equations and n unknowns.unknowns.

The system of equations shown on the The system of equations shown on the previous slide can be written more previous slide can be written more compactly as a matrix equation:compactly as a matrix equation:

Ax=bAx=b where the matrix A contains all the where the matrix A contains all the

coefficients of the unknown variables from coefficients of the unknown variables from the LHS, x is the vector of unknowns, and the LHS, x is the vector of unknowns, and b a vector containing the numbers from b a vector containing the numbers from the RHSthe RHS

Linear systems cont’Linear systems cont’

Gauss eliminationGauss elimination

Although these types of problems can be Although these types of problems can be solved easily using a wide number of solved easily using a wide number of computational packages, the principle of computational packages, the principle of Gaussian elimination should be Gaussian elimination should be understood.understood.

The principle is to successively eliminate The principle is to successively eliminate variables from the equations until the variables from the equations until the system is in ‘triangular’ form, that is, the system is in ‘triangular’ form, that is, the matrix A will contain all zeros below the matrix A will contain all zeros below the diagonal.diagonal.

A very simple example:A very simple example:

-x + 2y = 4-x + 2y = 4

3x + 4y =383x + 4y =38

first, divide the second equation by -2, then first, divide the second equation by -2, then add to the first equation to eliminate y; the add to the first equation to eliminate y; the resulting system is:resulting system is:

-x + 2y = 4-x + 2y = 4

-2.5x = -15-2.5x = -15 x = 6x = 6

y = 5y = 5

Gauss elimination cont’Gauss elimination cont’

Matrix rankMatrix rank

The rank of a matrix is simply the number The rank of a matrix is simply the number of independent row vectors in that matrix.of independent row vectors in that matrix.

The transpose of a matrix has the same The transpose of a matrix has the same rank as the original matrix. rank as the original matrix.

To find the rank of a matrix by hand, use To find the rank of a matrix by hand, use Gauss elimination and the linearly Gauss elimination and the linearly dependant row vectors will fall out, leaving dependant row vectors will fall out, leaving only the linearly independent vectors, the only the linearly independent vectors, the number of which is the rank.number of which is the rank.

Matrix inverseMatrix inverse

The inverse of the matrix A is denoted as AThe inverse of the matrix A is denoted as A-1-1

By definition, AABy definition, AA-1-1 = A = A-1-1A = I, where I is the A = I, where I is the identity matrix.identity matrix.

Theorem: The inverse of an nTheorem: The inverse of an nxxn matrix A n matrix A exists if and only if the rank A = n.exists if and only if the rank A = n.

Gauss-Jordan elimination can be used to find Gauss-Jordan elimination can be used to find the inverse of a matrix by hand. the inverse of a matrix by hand.

DeterminantsDeterminants

Determinants are useful in eigenvalue Determinants are useful in eigenvalue problems and differential equations.problems and differential equations.

Can be found only for square matrices.Can be found only for square matrices. Simple example: 2Simple example: 2ndnd order determinant order determinant

54*37*174

31det A

33rdrd order determinant order determinant

The determinant of a 3The determinant of a 3XX3 matrix is 3 matrix is found as follows:found as follows:

The terms on the RHS can be The terms on the RHS can be evaluated as shown for a 2evaluated as shown for a 2ndnd order order determinant.determinant.

3231

222113

3331

232112

3332

232211

333231

232221

131211

detaa

aaa

aa

aaa

aa

aaa

aaa

aaa

aaa

A

Some theorems for determinantsSome theorems for determinants

Cramer’s: If the determinant of a Cramer’s: If the determinant of a system of n equations with n system of n equations with n unknowns is nonzero, that system unknowns is nonzero, that system has precisely one solution.has precisely one solution.

det(AB)=det(BA)=det(A)det(B)det(AB)=det(BA)=det(A)det(B)

Eigenvalues and EigenvectorsEigenvalues and Eigenvectors

Let A be an nxn matrix and consider the Let A be an nxn matrix and consider the vector equation:vector equation:

Ax = Ax = xx A value of A value of for which this equation has a for which this equation has a

solution x≠0 is called an eigenvalue of the solution x≠0 is called an eigenvalue of the matrix A.matrix A.

The corresponding solutions x are called The corresponding solutions x are called the eigenvectors of the matrix A.the eigenvectors of the matrix A.

Solving for eigenvaluesSolving for eigenvaluesAx=Ax=xx

Ax - Ax - x = 0x = 0

(A- (A- I)x = 0I)x = 0 This is a This is a homogeneoushomogeneous linear system, linear system,

homogeneous meaning that the RHS are homogeneous meaning that the RHS are all zeros.all zeros.

For such a system, a theorem states that a For such a system, a theorem states that a solution exists given that det(A- solution exists given that det(A- I)=0.I)=0.

The eigenvalues are found by solving the The eigenvalues are found by solving the above equation.above equation.

Solving for eigenvalues cont’Solving for eigenvalues cont’ Simple example: find the eigenvalues for Simple example: find the eigenvalues for

the matrix:the matrix:

Eigenvalues are given by the equation Eigenvalues are given by the equation det(A-det(A-I) = 0:I) = 0:

So, the roots of the last equation are -1 So, the roots of the last equation are -1 and -6. These are the and -6. These are the eigenvalueseigenvalues of of matrix A.matrix A.

22

25A

674)2)(5(

22

25)det(

2

IA

EigenvectorsEigenvectors For each eigenvalue, For each eigenvalue, , there is a , there is a

corresponding eigenvector, x.corresponding eigenvector, x. This vector can be found by substituting This vector can be found by substituting

one of the eigenvalues back into the one of the eigenvalues back into the original equation: Ax = original equation: Ax = x : for the x : for the example:example: -5x-5x11 + 2x + 2x22 = = xx11

2x2x11 – 2x – 2x22 = = xx22

Using Using =-1, we get x=-1, we get x22 = 2x = 2x11, and by , and by arbitrarily choosing xarbitrarily choosing x11 = 1, the eigenvector = 1, the eigenvector corresponding to corresponding to =-1 is:=-1 is:

and similarly,and similarly,

2

11x

1

22x

Special matricesSpecial matrices

A matrix is called A matrix is called symmetricsymmetric if: if:

AATT = A = A A skew-symmetric matrix is one for A skew-symmetric matrix is one for

which:which:

AATT = -A = -A An An orthogonalorthogonal matrix is one whose matrix is one whose

transpose is also its inverse:transpose is also its inverse:

AATT = A = A-1-1

Complex matricesComplex matrices

If a matrix contains complex (imaginary) If a matrix contains complex (imaginary) elements, it is often useful to take its elements, it is often useful to take its complex conjugatecomplex conjugate. . The notation used for The notation used for the complex conjugate of a matrix A is: the complex conjugate of a matrix A is:

Some special complex matrices are as Some special complex matrices are as follows:follows:

Hermitian: Hermitian: TT = A = A

Skew-Hermitian:Skew-Hermitian: TT = -A = -A

Unitary:Unitary: TT = A = A-1-1