Linear Algebra - University of Stavanger · Linear Algebra B A Ødegaard 15 November 2018 1 Introduction, why do we need to know linear algebra ...

Linear Algebra

B A Ødegaard

15 November 2018

1 Introduction, why do we need to know linear algebra?I prefer to think of Linear Algebra as a very convenient shorthand for mathematical operations on data.

Traditionally, much of introductory courses in linear algebra were concerned with the actual mechanicsof calculation of some simple properties of matrices, but may never have given “the big picture” in terms ofunderstanding linear algebra by analogy with the real numbers.

These days, the impetus to know and use linear algebra comes from convenience when one want to donontrivial calculations on a computer. A tool implementing numerical linear algebra operations in a compactway has turned out to be very convenient for working on problems in finance and economics.

It is also very hard to do any kind of econometrics without knowing the basics of linear algebra. Iwill also claim that it is almost impossible to understand the intuition behind the results without a basicunderstanding of linear algebra.

I first go over some of the basic results of linear algebra. I am not going to go into proofs, but the resultsshould be known. I will first go over the theoretical results at high speed, and then slow down, go back, andillustrate these results using actual numbers and the computer for calculations.

2 Theoretical summary of Linear AlgebraI now summarize the theoretical concepts of linear algebra. My main goal here is to illustrate the similaritiesbetween the operations in linear algebra and the standard operations that are used on real numbers.

While I do not want to get to formal, it will not hurt to see the abstract definitions of a number ofconcepts.1 They are put here mainly for reference.

2.1 FieldsA set of elements e1, e2, · · · are said to belong to a field (F ), if they are closed under the operations ofaddition (ej + ei) and multiplication, (eiej), that is, the sum and products of elements of F also belong toF , and satisfy the following conditions. (A for addition, M for multiplication).

(A1) ei + ej = ej + ei (commutative law)(A2) ei + (ej + ek) = (ei + ej) + ek (associative law)(A3) For any two elements ei, ej ,

there exists an element eksuch that ei + ek = ej

1These definitions are from Rao (1973).

1

This condition implies that there is an element e0 such that ei + e0 = ei for all i. The element e0 is like zeroof the (real) number system.

(M1) eiej = ejei (commutative law)(M2) ei(ejek) = (eiej)ek (Associative law)(M3) ei(ej + ek) = eiej + eiek (Distributive law)(M4) For any two elements ej and ej

such that ei 6= e0, the zero elementthere exists an element ek such that eiek = ej

(M4) implies that there is an element e1 such that eie1 = ei for all i. This element e1 is like 1 (unity) of thenumber system.Example

The set of real numbers, denoted by R, is the best known example of a field.

2 + 0 = 2

2 + 3 = 5 = 3 + 2

2 · 1 = 2

2 · 3 = 6 = 3 · 2

2.2 Linear Vector Spaces.For each of these definitions, see if you can see the connection to what you know about real numbers.

Consider a collection of elements X,Y, Z, . . ., finite or infinite, which we choose to call vectors.2. Assumea set of scalar numbers c1, c2, . . . , that constitute a field (Like the ordinary numbers with the operations ofaddition, subtraction, multiplication and division suitably defined). To define a linear space we lay downcertain rules for combining the elements.

Vector Addition. The operation of addition indicated by + is defined for any two vectors leading toa vector in the set and is subject to the following rules.

(i) X + Y = Y +X Commutative Law(ii) X + (Y + Z) = (X + Y ) + Z Associative Law

Null Element. There exists an element in the set, denoted by 0, such that

(iii) X + 0 = X, for all X

Inverse (negative) Element. For any given element X, there exist a corresponding element ψ such that

(iv) X + ψ = 0

Scalar Multiplication. The multiplication of a vector X by a scalar c leads to a vector in the set,represented by cX, and is subject to the following rules.

(v) c(X + Y ) = cX + cY (Distributive law for vectors)(vi) (c1 + c2)X = c1X + c2X (Distributive law for scalars)(vii) (c1(c2X) = (c1c2)X (Associative law)(viii) 1X = X 1 is the scalar unit element.

2Note that this definition is not limited to what we usually think of as vectors, a set of n real numbers, X = (x1, x2, · · · , xn).(The Euclidean Linear Space). It allows for more abstract elements, but you may want to think of the Euclidean case forintuition

2

A collection of elements (with the associated field of scalars F ) satisfying axioms (i) to (viii) above is calleda linear vector space.

Some further useful definitions.Dependence. A set of vectors X1, · · · , Xn is said to be linearly dependent if there exists scalars

c1, · · · , cn, not all zero, such thatc1Xn + c2X2 + · · ·+ cnXn = 0

Otherwise it is independent.Basis. A linearly independent subset of vectors in a vector space, generating or spanning the vector

space, is called a basis.Inner product. The inner product, denoted (·, ·), is a (complex or real-valued) function of two vectors,

satisfying the following conditions

(i) (X,Y ) = (Y,X), the complex conjugate of (X,Y )(ii) (X,X) > 0 if X 6= 0,= 0 if X = 0(iii) (cX, Y ) = c(X,Y ) ( c is a scalar )(iv) (X + Y, Z) = (X,Z) + (Y,Z)

Norm. The square root of (X,X) is called the norm of X and denoted by ‖ X ‖=√(X,X). The

function ‖ X − Y ‖ satisfies the postulates of a distance in a metric space.Orthogonality. Two vectors X and Y are said to be orthogonal if their inner product, (X,Y ) = 0.Angle. The angle θ, between to non-null vectors X and Y is denoted by

cos θ =(X,Y )

‖ X ‖ · ‖ Y ‖

Orthogonal basis. A set of non-null vectors X1, · · · , Xn, orthogonal in pairs, is necessarily independent.A set of orthogonal vectors spanning a vector space is called an orthogonal basis of the vector space.Example

Let us look at the usual (Euclidean) vector space

x =

(21

)

y =

(12

)Addition:

x+ y =

(21

)+

(12

)=

(2 + 11 + 2

)=

(33

)=

(1 + 22 + 1

)=

(12

)(21

)= y + x

Null element

x+ 0 =

(21

)+

(00

)=

(21

)= x

Additive inverse element:

x+

(−2−1

)= 0

Scalar multiplication

2x =

(2 · 22 · 1

)=

(42

)The two vectors (

22

),

(11

)

3

are dependent, because

1

(22

)+ (−2)

(11

)=

(00

)= 0

Inner product (dot product) (22

)·(

11

)= 2 · 1 + 2 · 1 = 2 + 2 = 4

Norm ∥∥∥∥( 22

)∥∥∥∥ =

√(22

)·(

22

)=√2 · 2 + 2 · 2 =

√4 + 4 =

√8 = 2.8284

Orthogonal basis.For the 2-dimensional case, we can generate all vectors from the two vectors(

10

),

(01

)Why?

These are orthogonal, because (10

)·(

01

)= 0(

10

),

(01

)is thus an orthogonal basis.

2.3 Theory of MatricesA matrix A is a collection of pq elements arranged in p rows and q columns

a11 a12 · · · a1qa21 a22 · · ·...

. . .ap1 · · · apq

We will usually write a matrix by bold capitals (A). If necessary, will write explicitly the matrix as the

collection of elements [aij ] where i indexes rows, and j columns.The following operations are defined on matrices.Addition. If A and B are two matrices with the same dimension p × q, then the sum of A and B is

defined to be the matrix A+B = [aij + bij ], of the same order p× q. You may want to verify that

A+B = B+A

A+ (B+C) = (A+B) +C = A+B+C

Example

A =

[1 22 4

]B =

[1 11 1

]Addition

A+B =

[2 33 5

]= B+A

4

Scalar multiplication. As for vectors, a matrix may be multiplied by a scalar c, the result being amatrix of the same form

cA = [c · aij ]

Given this definition, it is easy to verify that

(c+ d)A = cA+ dA

c(A+B) = cA+ cB

Example

2A =

[2 · 1 2 · 22 · 2 2 · 4

]=

[2 44 8

]Matrix Multiplication. The matrix product AB is defined only when the number of columns in A is

the same as that of the rows in B. The result is a matrix with the (i, j) entry as the (inner) product of thei ’th row vector of A with the j ’th column vector of B.

If A = [aij ] and B = [bij ], then the product is obtained as

AB = [cij ] cij =∑r

airbrj

If A is p× q and B is q × t, then C = AB is p× q.Example

AB =

[1 22 4

]·[

1 11 1

]=

[1 · 1 + 2 · 1 1 · 1 + 2 · 12 · 1 + 4 · 1 2 · 1 + 4 · 1

]=

[3 36 6

]Example

As a more involved example, consider

[−1 2 35 1 2

] 1 2 04 1 33 1 2

=

[16 3 1215 13 7

]

Note that the commutative law does not hold for matrix multiplication. In general AB 6= BA. It mayalso be that AB is defined but not BA. The associative law is however satisfied,

A(BC) = (AB)C = ABC

In the cases where the matrices are compatible, the distributive law also holds,

A(B+C) = AB+AC

(A+B)(C+D) = A(C+D) +B(C+D)

Null matrix. A null matrix, denoted by 0, is a matrix with all elements equal to zero.

0 =

0 · · · 0

0. . . 0

0 · · · 0

If 0 is the same dimension as A

A+ 0 = A

5

Identity matrix. The identity matrix , denoted by I, is a square matrix with ones along the diagonal, andzeros elsewhere. If we want to be explicit about the dimension, we will use the symbol Iq of the identitymatrix with dimension q × q.

I =

1 0 0 · · · 0 00 1 0 · · · 0 0...

. . .0 0 1

If the operations are defined

IA = A

AI = A

Thus the matrices 0 and I behaves like 0 and 1 in the (usual) number system, but remember that theoperations may not always be defined.

Transpose of a matrix. The matrix A′ (also denoted AT ), obtained by interchanging the rows andcolumns of A is called the transpose of A. This means that the (i, j)’th entry of A is the same as the (j, i)’thentry of A′. A useful property of transposes is that

(AB)′ = B′A′

Note: if X and Y are column vectors, the inner product of these vectors is equal to

X′Y = Y′X

Also, if A is a matrix, and X a column vector, then the product AX is a linear combination of the columnsof A.

Rank of a matrix. A matrix may be considered a collection of (row or column) vectors. The rank of thematrix is the number of linearly independent rows in the matrix, which is equal to the number of independentcolumns in the matrix.Example

rank[

1 11 2

]= 2

rank[

1 12 2

]= 1

Inverse of a matrix. A square matrix of order (m × m) is said to be nonsingular if its rank is m.Otherwise it said to be singular. If a matrix A is nonsingular, there exists a unique matrix A−1, called theinverse of A, such that

AA−1 = A−1A = Im

A useful result about inverses is that(AB)−1 = B−1A−1

Example

A =

[1 11 2

]A−1 =

[2 −1−1 1

]Let us check this

A · (A−1) =[

1 11 2

]·[

2 −1−1 1

]=

[1 · 2 + 1 · (−1) 1 · (−1) + 1 · 11 · 2 + 1 · (−1) 1 · (−1) + 2 · 1

]=

[1 00 1

]

6

Generalized inverse of a matrix. The inverse is only defined for square nonsingular matrices. A muchmore general concept is the g-inverse. If A is a m × n matrix of any rank, there exist a matrix A− suchthat X = A−Y is a solution of the equation AX = Y for any Y in the span of A. Note that the g-inversedefined this way is not necessarily unique.

Idempotent matrix. A matrix that multiplied with itself does not change we call idempotent. A isidempotent if

AA = A

The Determinant of a square matrix A, of order m, written |A|, is a real valued function of the elementsaij defined by:

|A| =∑±a1ia2j · · · amp

where the summation is taken over all permutations (i, j, · · · , p) of (1, 2, · · · ,m), and the sign is + if thepermutation is odd, − if it is even.

Some properties of determinants:

• |A| = 0 only if rank(A) < m. (Hence if |A| 6= 0, then A is invertible.)

• if c is a constant,|cA| = |c|m · |A|

• if A is diagonal or triangular, |A| is the product of the diagonal elements.

Example

A =

[1 11 2

]|A| = (1 · 2− 1 · 1) = 1

Square Root of Matrix. For a matrix A, if there exist another matrix A12 such that

A12 ·A

12 = A

we call this the “square root” matrix.

2.4 Systems of Linear Equations.Let A1, A2, · · ·An be vectors. Consider the linear equation in the scalars x1, x2 · · ·xn

A1x1 +A2x2 + · · ·Anxn = 0 (1)

A necessary and sufficient condition that this has a nontrivial solution (that is, not all equal to zero) isthat A1 · · ·An be dependent.

Claim: If there is more than one solution to a set of linear equations, then there is an infinite number ofsolutions.

Proof: Let X = {x1, · · ·xn} and Y = {y1, · · · , yn} solve the linear equation, X 6= Y . Then

A1x1 +A2x2 + · · ·+Anxn = 0

A1y1 +A2y2 + · · ·+Anyn = 0

Since both equations equal zero, we can multiply each equation with a scalar without changing the sum. Letλ be some real number.

λ (A1x1 +A2x2 + · · ·Anxn) = 0

(1− λ) (A1y1 +A2y2 + · · ·+Anyn) = 0

7

Add the two equations:

A1(λx1 + (1− λ)y1) + · · ·+An(λxn + (1− λ)yn) = 0

Since this hold for any real λ, there is an infinite number of solutions to the set of linear equations.Consider next the non-homogeneous equation

A1x1 +A2x2 + · · ·Anxn = An+1 (2)

This equation 2 admits a solution if and only if An+1 is dependent on A1 · · ·An.A system of equations is most easily written using matrices. If the vectors are the usual (Euclidean) case,

the system is written as

a11x1 + a12x2 + · · ·+ a1nxn = 0

a21x1 + a22x2 + · · ·+ a2nxn = 0

......

am1x1 + am2x2 + · · ·+ amnxn = 0

Define the matrices

A =

a11 a12 · · · a1na21 a22 · · · a2n

. . .am1 am2 · · · amn

X =

x1x2...xn

The system of equations is then compactly written as

AX = 0

Similarly, to write

a11x1 + a12x2 + · · · a1nxn = b1

a21x1 + a22x2 + · · · a2nxn = b2...

...am1x1 + am2x2 + · · · amnxn = bn

in matrix notation, we define the column vector

B =

b1b2...bn

and define the system of equations as

AX = B

8

The results above about solvability is then expressed as conditions on the ranks of the matrices.The homogeneous equation:

AX = 0

A is an n×m matrix of rank r ≤ min(n,m). The set X of all solutions is a linear subspace of rank m− r.If A is square, and the rank r equals the number of equations, the equation only admits the trivial

solution X = 0.The non-homogeneous equation:

AX = B

only admits a solution the rank of the matrix A is equal to the rank of the matrix defined by augmentingA with the column vector B.

The simplest case is when A is a square matrix, say n× n. Then the solution of AX = B has a uniquesolution if rank(A) = n, (the matrix is non-singular) and the solution is

X = A−1B

9

Using the computer for linear algebra, Matrix toolsA large number of computing environmenents now is available for dealing with linear algebra on actual

numbers. Such an environment is the next logical step when spreadsheets become too limited, as you canat low cost deal with complicated calculations. Most of these tools tend to look very much alike, as long asyou can get data into matrices you will manupulate the matrices using commands. In this chapter we willuse one particular program to illustrate with, octave. This has the benefit of being freely available. Similarfreely available tools are scilab and R. Commercial offerings include matlab and S.

3 Matrix theory using a computer toolWe will now go over these basic matrix operations and results with actual computer examples.

3.1 Starting the programThis is left as an exercise to the reader. It depends on what operating environment you are working on. Forusers on a Windows type system you will find it on the menu, if properly installed. If on the other hand youare on a UNIX workstation with X windows installed you will then start octave from an xterm window.Type octave at the prompt, and the program starts, you get the prompt, which we will show here as

>>

which means that the program is ready to receive commands. .In the text output of octave will be shown typewritten as:

>> A = [1, 2, 3> 4, 5, 6]

This particular command defines a matrix A, and octave responds with printing the matrix that wasjust defined:

A =1 2 34 5 6

3.2 Basic matrix operations3.2.1 Defining variables

The basic variables in octave are numbers, vectors and matrices. The type of each variable is determinedas you define it:

>> a=1a = 1>> b=2b = 2>> c=3c = 3>> y=[1;2;3]y =

123

>> x=[1,2,3]x =

10

1 2 3>> A=[1 2 3;4 5 6]A =

1 2 34 5 6

As you see, a space or a comma means a new number, a semicolon a new row. To suppress the printing ofwhat you just defined, end the line with a semicolon:

>> A=[1,2,3,4];>> A=[1,2,3,4]A =

1 2 3 4>>

You can also use defined variables to define new variables, as long as the dimensions make sense. For example,given the above definitions:

>>B=[c x]B =

3 1 2 3>> C = [A;x]C =

1 2 34 5 61 2 3

>> D = [A y]error: number of rows must matcherror: evaluating assignment expression near line 22, column 3

If the dimensioning is wrong, you get an error message, and the variable is not defined.To see what is in a variable, tell octave to print the value by giving the name:

>> aa = 1>> bb = 2>> AA =

1 2 34 5 6

>> BB =

5 1 2 3

Note that octave is case-sensitive, both A and a are defined.

3.2.2 Arithmetic Matrix Operations

We now get to the important parts of octave, namely, its built–in matrix arithmetic.Addition and subtraction.Numbers

>> a=1;>> b=2;

11

>> c=3;>> a+bans = 3>> b+cans = 5

Vectors

>> x=[1 2 3 4 ]x =

1 2 3 4>> y=[4 3 2 1]y =

4 3 2 1>> x+yans =

5 5 5 5>> y-xans =

3 1 -1 -3

We can also take linear combinations of vectors:

>> a*x + b*yans =

9 8 7 6>> 0.5*x -1.5*yans =

-5.50000 -3.50000 -1.50000 0.50000

Matrices

>> A=[1 2 3; 4 5 6];>> B=[6 5 4; 3 2 1];>> A+Bans =

7 7 77 7 7

>> A-Bans =

-5 -3 -11 3 5

>> a*A + b*Bans =

13 12 1110 9 8

3.2.3 Matrix Multiplication

When multiplying matrices, you need to be more careful about dimensions, but otherwise it is just like vectormultiplication.

12

>> A=[1 2 3;4 5 6]A =

1 2 34 5 6

>> B = [1 2;3 4; 5 6]B =

1 23 45 6

>> A*Bans =

22 2849 64

>> B*Aans =

9 12 1519 26 3329 40 51

For these matrices, both AB and BA are defined operations, but note that the results are different, in fact,even the dimension of the product is different.

If we let B be a 2× 2 matrix, then multiplying AB is an error.

>> B=[1 2;3 4]B =

1 23 4

>> A*Berror: nonconformant matrices (op1 is 2x3, op2 is 2x2)>> B*Aans =

9 12 1519 26 33

3.2.4 Matrix Operations

TransposeThe transpose of a matrix a is a’:

>> AA =

1 2 34 5 6

>> A’ans =

1 42 53 6

Null and identity matrices

>> null = zeros(3,3)null =

0 0 0

13

0 0 00 0 0

>> ident = eye(3,3)ident =

1 0 00 1 00 0 1

Rank:

>> AA =

1 2 34 5 6

>> rank(A)ans = 2

Inverse

>> D = [3 4;4 6]D =

3 44 6

>> inverse(D)ans =

3.0000 -2.0000-2.0000 1.5000

To make sure that this is the inverse, multiply D and inverse(D):

>> D * inverse(D)ans =

1 00 1

Idempotent matrixSo that you can see that there really exists idempotent matrices, consider

F =

[12

12

12

12

]

>> F = [0.5 0.5; 0.5 0.5]F =

0.50000 0.500000.50000 0.50000

>> F * Fans =

0.50000 0.500000.50000 0.50000

Determinant

>> BB =

1 2

14

3 4>> det(B)ans = -2

A Warning. Numerical errors.One thing you should be aware of when using a tool like octave is that the underlying tools are floating

point number routines. All these are subject to numerical round–off errors, that may aggregate, and giveweird results. This is also a problem in doing numerical linear algebra, if a matrix is ill-conditioned. As anexample, consider the matrix

>> AA =

1 2 33 2 11 1 1

This matrix is singular, which you see if you take 14 times the sum of the first two rows, which is equal to

the third row. Since the matrix is singular, it does not have an inverse, and should have determinant equalto zero. Let us check.

>> det(A)ans = -4.4409e-16>> inverse(A)ans =

-2251799813685248 -2251799813685246 90071992547409894503599627370496 4503599627370494 -18014398509481980-2251799813685248 -2251799813685248 9007199254740992

>> rank(A)ans = 2

As you see, because of numerical inaccuracies, octave does not “catch” that the determinant of A is actuallyequal to zero, and tries to invert A giving this extremely wrong estimate of the inverse. The algorithm forfinding the rank of the matrix, on the other hand, is accurate enough to find the correct rank of 2. Takingdeterminants is thus useful as a diagnostic tool.3

Solving linear equations.Remember the basic linear equation

Ax = b

We saw that this equation had a defined solution if the rank of A was the same as the rank of [A|b]. If Ais nonsingular, we solve the linear equation by finding the unique solution

x = A−1b

Exercise 1.Consider the linear equation

3x1 + 4x2 = 5

4x1 + 6x2 = 8

Solve this system of equations using a matrix tool.Solution to Exercise 1.

3There are actually better tools available to check the condition of a matrix, which you may want to look at if you are doingimportant computations. These problems are not due to low quality of the matrix routines, which are actually the best USgovernment money can buy. (The basic matrix operations are done using LAPACK, a set of efficient and accurate FORTRANroutines financed by the US government.)

15

Write this in matrix form by defining

A =

[3 44 6

]b =

[58

]Let us first check that this system is solvable

>> A = [3 4;4 6]A =

3 44 6

>> b=[5;8]b =

58

>> rank(A)ans = 2>> rank ([A b])ans = 2

Note how I create the augmented matrix [A|b] by [A b]. The rank of the two is the same. Since A is square, we cancalculate the solution as

>> inverse(A)ans =

3.0000 -2.0000-2.0000 1.5000

>> x = inverse(A) * bx =

-12

The solution to the system of equations is

x =

[−12

](In this case we calculated the solution by finding the inverse. But you should be aware that solving the system of

equations by calculation of the inverse is not the numerically most stable way of doing the calculation. Matlab/Octavehas built in a direct linear matrix solver, which is invoked by the left division operator

>> x = A\bx =

-12

This solves the system of equations directly, and it is usually the way to do this operation, unless one needs the inverse forother purposes.)

4 Concluding comments.We have now covered a lot of ground in linear algebra. I hope you realize that linear algebra is a very usefultool for a number of purposes, and that a matrix tool takes a lot of the pain out of using linear algebra.

Readings You need the manual for your chosen computer program.

16

Further reading. Look at (Davidson and MacKinnon, 1993, ch 1.5) for a short discussion of problems ofnumerical calculations on a computer.

I prefer to think about this as the difference between menus and command lines. Menus are much easierthe first time you do something, but they get increasingly annoying when you need to do the same operationover and over. They are also limiting in the sense that everything has to be thought of beforehand. Anycomand line oriented interaction is much easier to extend, it is merely a matter of writing code for thecommand.

Be that as it may, for this course you need to understand such a matrix oriented tool because many ofthe examples I show you have been solved in such a tool, and you need to be able to read my solutions.

I also hope that you will start using such a tool for this and other classes, if only for preparing yourselvesfor industry practice.Exercise 2.

Define the following variables:Real numbers r = 1, s = 10, u = 2.Vectors

a =

123

b =

321

c =

111

x =

x1x2x3

Matrices

A =

1 1 12 2 23 3 3

B =

3 2 32 4 51 1 1

C =

1 1 11 1 11 1 1

I3 =

1 0 00 1 00 0 1

Calculate

ra+ sb

sA+ uB

B′

Calculate the determiants of the three matricesWhich of the three matrices are invertible?Calculate

B−1

Solve the linear systemBx = a

Solution to Exercise 2.Defining the variables

>> r=1r = 1>> s=10s = 10>> u=2u = 2>> a=[1;2;3]a =

123

17

>> b=[3;2;1]b =

321

>> c=[1;1;1]c =

111

>> A=[1 1 1; 2 2 2 ; 3 3 3 ]A =

1 1 12 2 23 3 3

>> B=[3 2 3; 2 4 5; 1 1 1]B =

3 2 32 4 51 1 1

>> C= ones(3,3)C =

1 1 11 1 11 1 1

>> I3=eye(3)I3 =

1 0 00 1 00 0 1

Doing the calculations

>> r*a+s*bans =

312213

>> s*A+u*Bans =

16 14 1624 28 3032 32 32

Matrix transpose and inverse

>> B’ans =

3 2 12 4 13 5 1

>> inv(B)ans =

0.33333 -0.33333 0.66667-1.00000 0.00000 3.000000.66667 0.33333 -2.66667

18

Calculating determinants

>> det(A)ans = 0>> det(B)ans = -3.0000>> det(C)ans = 0

To see which matrices are invertible we calculate the ranks of the three matrices. It is only when the matrices are squareand have ranks equal their dimensions that they are invertible.

>> rank(A)ans = 1>> rank(B)ans = 3>> rank(C)ans = 1

This could alternatively have been seen from the determinants, only when determinants are nonzero can the system besolved.

Solving the linear systemBx = a

>> x=inv(B)*ax =

1.66678.0000

-6.6667

Exercise 3.Calculate the sum 1 + 2 + 3 + · · ·+ 99 + 100.

Solution to Exercise 3.

>> [1:100]*ones(100,1)ans = 5050

Exercise 4.Let

A =

1 1 10 1 10 0 1

1. Calculate A2 and A3.

2. What is your guess about the form of An?

Solution to Exercise 4.

>> A=[1 1 1 ; 0 1 1; 0 0 1]A =

1 1 10 1 10 0 1

>> A^2

19

ans =1 2 30 1 20 0 1

>> A^3ans =

1 3 60 1 30 0 1

Guessing An: Let us try n = 10:

>> A^10ans =

1 10 550 1 100 0 1

>> [1:10]*ones(10,1)ans = 55

Certainly looks like the matrix is

An =

1 n∑n

j=1 j

0 1 n0 0 1

20

5 Finance examples of linear algebra usageLet us now look at some examples of linear algebra usage taken from financial economics.Exercise 5.

Consider two one-period economies, {PA, DA} and {PB , DB}, each with N = 3 assets with payoffs acrossS = 3 states. Their prices and payoff matrices DS×N (each row represents a different state) are given by

PA =

0.451.751.30

PB =

0.451.851.60

DA = DB = D =

0 1 30 0 21 4 1

D−1 =

−4 5.5 11 −1.5 00 0.5 0

1. What is the present value of the cash flow Y = (3, 3, 3)

′, and what is the riskless rate of return in the twoeconomies?

2. Are the prices consistent with no arbitrage?

Solution to Exercise 5.

1. Finding prices

A bit careful here, note that the states are on the rows, so we need to transpose D below:

>> D=[0 1 3;0 0 2;1 4 1]D =

0 1 30 0 21 4 1

>> Pa=[0.45;1.75;1.3]Pa =

0.450001.750001.30000

Pb=[0.45;1.85;1.6]Pb =

0.450001.850001.60000

We want to find the vector of prices Q that solves

PA = D′QA =⇒ QA = (D′)−1PA

PB = D′QA =⇒ QB = (D′)−1PB

21

>> Qa=inv(D’)*PaQa =

-0.0500000.5000000.450000

>> Qb=inv(D’)*PbQb =

0.0500000.5000000.450000

the risk free rates are found by

Rf =1

[1 1 1]Q

>> Rfa=1/([1 1 1]*Qa)Rfa = 1.1111>> Rfb=1/([1 1 1]*Qb)Rfb = 1.00000

To check that the prices are correct, see whether we get the P ’s back

>> D’*Qaans =

0.450001.750001.30000

>> D’*Qbans =

0.450001.850001.60000

Value of cash flow

>> [3 3 3]*Qaans = 2.7000>> [3 3 3]*Qbans = 3.0000

22

Exercise 6.There are four states, with state prices

φ =

0.250.20.150.10

Consider a firm. The firm will last for one more period and then be liquidated. The firms cash flow in each stateis

C =

100110200500

The Modigliani Miller theorems are statements about the indifference of the value of the firm to changes in thedebt/equity mix.

How does debt and equity fit into this picture?Debt: A claim on the bond principal F . If the firm can not pay the principal, it is bankrupt, and the bond

holder becomes the residual claimant.Equity: A claim on the residual value of the firm, after all the firm’s debt is paid out.

1. Consider first the case of the firm being all equity financed. Determine the value of the firm and the valueof equity.

2. Suppose now the firm has debt with face value F = 100 payable in period 1. Determine the current valueof the debt and equity in the company. Is the debt risk free? What is the sum of debt and equity?

3. Suppose instead the firms debt has a face value value F = 200. Determine the current value of the debtand equity in the company. Is the debt risk free? What is the sum of debt and equity?

Solution to Exercise 6.

1. Then the value of the firms equity is equal to the current value of the cashflows in the different states.Let V be the firm’s value and E and D the market values of debt and equity.

V = E = C′φ = 127

> C=[100 110 200 500]’> phi =[ 0.25 0.2 0.15 0.10 ]’> C’*phians = 127

2. Now consider the case of the firm having debt with face value F = 100 payable in period 1.This debt is risk free, the firm always has cash flows to pay the bond holders the full principalCash flows to debt is

CFD

100100100100

D = φ′CFD =

0.250.20.150.10

′

100100100100

= 70

An alternative way to calculate this is to use the risk free rate of interest.

Rf =1∑ω φω

=1

0.25 + 0.2 + 0.15 + 0.1=

1

0.7= 42.86%

23

Since the debt is riskless, it can be discounted at the risk free rate of interest.

D =100

1.4286≈ 70

Now, what about equity? The cash flows to equity is what is left over after debt has been paid.

CFE = C − CFD =

100110200500

−

100100100100

=

010100400

The current value of the firms equity is then

E = φ′CFE = φ′

010100400

= 57

The value of the firm is thenV = D + E = 70 + 57 = 127,

the same as before.

> CF_D = [100;100;100;100]> D = phi’* CF_DD = 70> phi’*ones(1,4)’ans = 0.70000> 1.0/0.7ans = 1.4286> 100/1.4286ans = 69.999> CF_E = C-CF_D

010

100400

> E=phi’*CF_EE = 57> V=D+EV = 127

3. Consider next a firm with bonds outstanding with face value F = 200.

CFD = min(200, C) =

min(200, 100)min(200, 110)min(200, 200)min(200, 500)

=

100110200200

CFE = C − CFD =

100− 100110− 110200− 200500− 200

=

000

300

The values of debt and equity are

D(F = 200) = φ′CFD = 97

E(F = 200) = φ′CFE = 30

V (F = 200) = D + E = 97 + 30 = 127

No matter what the mix of debt and equity, the value of the firm stays at 127.

24

> CF_D = min(200,C)100110200200

> CF_E = C-CF_D000

300> D=phi’*CF_DD = 97> E=phi’*CF_EE = 30> V=D+EV = 127

25

Exercise 7.You observe the three risk free bonds A, B and C:

Cashflow in periodBond Price 1 2 3A 95 100 0 0B 90 10 110 0C 85 10 10 110

1. What is the current value of receiving one dollar at time 3?

2. What are the interest rates (with annual compounding) implied in these prices?

Another risk free bond D is traded, with the following cash flows:

time: 1 2 3D 20 20 520

2. What is the current price of bond D?

Solution to Exercise 7.

1. One way to think of this is using state prices. Want to calculate the price of receiving one dollar for certain in period(state) t.If

B =100 0 010 110 010 10 110

is the payoffs, and

P =959085

is the current prices of the bonds, find the current state prices d (discount factors) from the relation:

P = Bd

d = B−1P

d = inv(B)*P0.950000.731820.61983

The current value of receiving one dollar at time 3 is thus 0.61983.

2. To find the interest rates we solve the expressions

dt =1

(1 + r)t

wrt r. Doing everything at once:

>> t=[1;2;3];>> r=d.^(-1./t)-1r =

0.0526320.1689570.172847

26

The interest rates are increasing with time. To check that these are correct, recalculate

>> (1./(1+r)).^tans =

0.950000.731820.61983

3. To price bond D we use the “discount factors” d:

> D =20 20 520

> d =0.95000 0.73182 0.61983

D*d’ = 355.95

Bond D has a price equal to 355.95.

27

Readings. The basic definitions of vector spaces are taken from Rao (1973)Theil (1971) 1.1, (1.2), 1.3, (1.4), (1.6) are basic sources on the use of linear algebra in econometrics.

Further Reading Gale (1960) is a good introduction, although somewhat dated for the rest of the material,to the use of linear algebra in economics. The same can be said of appendix B of Dorfman, Samuelson, andSolow (1987)

Any econometrics textbook will have an appendix with the basic linear algebra results, in more or lesscomprehensive form. The Davidson and MacKinnon (1993) appendix is useful, but terse.

Mathematical text, see Sydsæter and Øksendal (1977) or Hadley (1961).

Advanced Sources For a serious development of the mathematics of this material, the obvious first placeto look is Rao (1973).

ReferencesRussel Davidson and James G MacKinnon. Estimation and Interference in Econometrics. Oxford University Press, 1993.

Robert Dorfman, Paul A Samuelson, and Robert M Solow. Linear Programming and Economic Analysis. Dover, 1987. Originallypublished: MacGraw-Hill, 1958.

David Gale. The Theory of Linear Economic Models. University of Chicago Press, 1960.

G Hadley. Linear Algebra. Addison-Wesley, 1961.

C Radhakrisna Rao. Linear Statistical Inference and its applications. Wiley, Second edition, 1973.

Knut Sydsæter and Bernt Øksendal. Lineær Algebra. Universitetsforlaget, 1977.

Henri Theil. Principles of econometrics. Wiley, 1971.

28