6 Linear Algebraic Equations.pdf

Linear Algebraic Equations

Cheng-Liang Chen

PSELABORATORY

Department of Chemical EngineeringNational Taiwan University

CL Chen PSE LAB NTU 1

Linear Algebraic Equations and Gauss Elimination

−x + y + 2z = 2 (1)

3x− y + z = 6 (2)

−x + 3y + 4z = 4 (3)

(3)−(1)−→ 2y + 2z = 2 (4)(2)+3(1)−→ 2y + 7z = 12 (5)

(5)−2(4)−→ 5z = 10 (6) (→ z = 2)z=2 in (4)−→ y = −1 (7)

z=2,y=−1 in (1)−→ x = 1 (8)


1. Equation (1) is the pivot equation

2. Multiply it by −1 and add the result to (3)to obtain 2y + 2z = 2 (equivalent to y + z = 1)

−x + y + 2z = 2 (1)3x− y + z = 6 (2)

−x + 3y + 4z = 4 (3)

3. Next multiply (1) by 3 and add the result to (2)to obtain 2y + 7z = 12

Thus we have a new set of two equations in twounknowns (y, z):

y + z = 1 (4)2y + 7z = 12 (5)

4. Equation (4) is the new pivot equation.

5. Multiply (4) by −2 and add the result to (5)to obtain 5z = 10 (or z = 2)

6. Substitute z = 2 into (4) to obtain y + 2 = 1 (or y = −1)

7. Then substitute y = −1 and z = 2 into (1)to obtain −x− 1 + 4 = 2 (or x = 1)


Test Your Understanding

T6.1-1Solve the following equations using Gauss elimination:

6x− 3y + 4z = 41

12x + 5y − 7z = −26

−5x + 2y + 6z = 14

(Answer:x = 2, y = −3, z = 5.)


Singular and Ill-Conditioned Problems

3x− 4y = 5

6x− 10y = 2

⇒ unique solution, x = 7, y = 4

3x− 4y = 5

6x− 8y = 10

⇒ singular, infinite # solution

3x− 4y = 5

6x− 8y = 3

⇒ singular, no solution


0 1 2 3 4 5 6 7 8 9 10−5

0

5

10

x

yUnique solution

Line 1Line 2

0 1 2 3 4 5 6 7 8 9 10−5

0

5

10

x

y

Singular, No unique solution

Line 1Line 2

0 1 2 3 4 5 6 7 8 9 10−5

0

5

10

x

y

Singular, No solution

Line 1Line 2


x = [0 : 0.25 : 10];y_1 = (3*x-5)/4;y_2 = (6*x-2)/10;subplot(3,1,1)plot(x,y_1,x,y_2,’o’),...xlabel(’x’),ylabel(’y’),...title(’Unique solution’),...legend(’Line 1’,’Line 2’)y_3 = (6*x-10)/8;subplot(3,1,2)plot(x,y_1,x,y_3,’*’),...xlabel(’x’),ylabel(’y’),...title(’Singular, No unique soln’),...legend(’Line 1’,’Line 2’)y_4 = (6*x-3)/8;subplot(3,1,3)plot(x,y_1,x,y_4,’d’),...xlabel(’x’),ylabel(’y’),...title(’Singular, No solution’),...legend(’Line 1’,’Line 2’)

0 1 2 3 4 5 6 7 8 9 10−5

0

5

10

x

y

Unique solution

Line 1Line 2

0 1 2 3 4 5 6 7 8 9 10−5

0

5

10

x

y

Singular, No unique solution

Line 1Line 2

0 1 2 3 4 5 6 7 8 9 10−5

0

5

10

x

y

Singular, No solution

Line 1Line 2


Homogeneous Equations (zero RHSs)

6x + ay = 0 (1)

2x + 4y = 0 (2)

(1)−3(2)−→ (a− 12)y = 0

Case 1: y = 0 only if a 6= 12y=0 in (1)−→ x = 0

Case 2: 0y = 0 if a = 12

⇒ x = −2y infinite # of solutions


Ill-Conditioned Equations

An Ill-Conditioned set of equations is a set that is close to being

singular

3x− 4y = 5

6x− 8.002y = 3

⇒ y =3x− 5

4⇒ x = 4668

y =3x− 1.54.001

y = 3500

But if we had carried only two significant figures

⇒ We would have rounded the denominator of the latter

expression to 4.0

⇒ Two expressions for y have the same slope (parallel)

⇒ No solution ⇒ ill-conditioned status (soln dep.s on accuracy)



T6.1-2Show that the following set has no solution.

−4x + 5y = 10

12x− 15y = 8

T6.1-3For what value of b will the following set have a solution in which

both x and y are nonzero? Find the relation between x and y.

4x− by = 0

−3x + 6y = 0

(Answer: If b = 8, x = 2. If b 6= 8, x = y = 0.)


Matrix Methods for Linear Equations

2x1 + 9x2 = 5

3x1 − 4x2 = 7[2 93 −4

]︸︷︷︸

A

[x1

x2

]︸︷︷︸

x

=

[57

]︸︷︷︸

b

⇒ Ax = b

a11x1 + a12x2 + · · ·+ a1nxn = b1

a21x1 + a22x2 + · · ·+ a2nxn = b2

· · · · · ·am1x1 + am2x2 + · · ·+ amnxn = bm

a11 a12 · · · a1n

a21 a22 · · · a2n

· · · · · ·am1 am2 · · · amn

︸︷︷︸

A

x1

x2

...

xn

︸︷︷︸

x

=

b1

b2

...

bm

︸︷︷︸

b

Ax = b


Determinant and Singular Problems

A =

∣∣∣∣∣∣∣∣3 −4 1

6 10 2

9 −7 8

∣∣∣∣∣∣∣∣A = [3,-4,1; 6,10,2; 9,-7,8];det(A)

ans =8


Determinants and Singular Problems

Singular set: 3x− 4y = 5

6x− 8y = 10

⇒

3 −4

6 −8

︸︷︷︸

A

x

y

=

5

10

︸︷︷︸

b

|A| =

∣∣∣∣∣∣ 3 −4

6 −8

∣∣∣∣∣∣ = 3(−8)− (−4)(6) = 0

|A| indicates that the equation set is singular


Left-division Method With Three Unknowns

Example: Use the left-division method to solve the following set:

3x + 2y − 9z = −65

−9x− 5y + 2z = 16

6x + 7y + 3z = 5

Solution:

A =

3 2 −9

−9 −5 2

6 7 3

We can use MATLAB to check the determinant of A to see

whether the problem is singular.


A = [3, 2, -9; -9, -5, 2; 6, 7, 3];b = [-65; 16; 5];det_A = det(A),...soln = A\b ,... % Ax=b

A*soln

det_A =288

soln =2.0000-4.00007.0000

ans =-65.000016.00005.0000


Example: An Electrical-resistance Network

The circuit shown in the following Figure has five resistances and applied voltages.Assuming that the positive directions of current flow are in the directions shown inthe figure, Kirchhoff’s voltage law applied to each loop in the circuit gives

−v1 + R1i1 + R4i4 = 0

−R4 + R2i2 + R5i5 = 0

−R5i5 + R3i3 + v2 = 0

Conservation of charge applied at each node in the circuit gives

i1 = i2 + i4

i2 = i3 + i5

You can use these two equations to eliminate i4 and i5 from the first three


equations. The results is:

(R1 + R4)i1 −R4i2 = v1

−R4i1 + (R2 + R4 + R5)i2 −R5i3 = 0

R5i2 − (R3 + R5)i3 = v2

Thus we have three equations in three unknowns: i1, i2, and i3.

Write a MATLAB script file that uses given values of the applied voltages v1 and

v2 and given values of the five resistances to solve for the currents i1, i2, and i3.

Use the program to find the currents for the case R1 = 5, R2 = 100, R3 = 200,

R4 = 1504, R5 = 250 kΩ, v1 = 100, and v2 = 50 volts. (Note that 1

kΩ = 1000Ω)


Solution:

% File resist.m % Solvers for the currents i_1, i_2, i_3R = [5, 100, 200, 150, 250]*1000;v1 = 100; v2 = 50;A1 = [R(1) + R(4), -R(4), 0];A2 = [-R(4), R(2) + R(4) + R(5), -R(5)];A3 = [0, R(5), -(R(3) + R(5))];A = [A1; A2; A3];b = [v1; 0; v2];current = A\b;disp(’The currents are:’)disp(current)

>> resist % run resist.m

The currents are:1.0e-003*0.95440.31950.0664 % i_1,i_2,i_3 = 0.9544, 0.3195, 0.0664 mA


Example: Ethanol Production

Engineers in the food and chemical industries use fermentation in many processes.The following equation describes Baker’s yeast fermentation.

a(C6H12O6) + b(O2) + c(NH3)−→ C6H10NO3 + d(H2O) + e(CO2) + f(C2H6O)

The variables a, b, . . . , f represent the masses of the products involved in theequation. In this formula C6H12O6 represents glucose, C6H10NO3 representsyeast, and C2H6O represents ethanol. This reaction produces ethanol, in additionto water and carbon dioxide. We want to determine the amount of ethanol fproduced. The number of C, O, N , and H atoms on the left must balance thoseon the right side of the equation. This gives four equations:

6a = 6 + e + 2f C balance

6a + 2b = 3 + d + 2e + f O balance

c = 1 N balance

12a + 3c = 10 + 2d + 6f H balance


The fermentor is equipped with an oxygen sensor and a carbon dioxide sensor.These enable us to compute the respiratory quotient R:

R =CO2

O2=

e

b

Thus the fifth equation is Rb− e = 0. The yeast yield Y (grams of yeast producedper gram of glucose consumed) is related to a as follows .

Y =144180a

Where 144 is the molecular weight of yeast and 180 is the molecular weight of

glucose. By measuring the yeast yield Y we can compute a as follows:

a = 144/180Y . This is the sixth equation.

Write a user-defined function that computes f , the amount of ethanol produced,

with R and Y as the function’s arguments. Test your function for two cases where

Y is measure to be 0.5: (a) R = 1.1 and (b) R = 1.05.


Solution: let x1 ≡ b, x2 ≡ d, x3 ≡ e, x4 ≡ f

−x3 − 2x4 = 6− 6(144/180Y )

2x1 − x2 − 2x3 − x4 = 3− 6(144/180Y )

−2x2 − 6x4 = 7− 12(144/180Y )

Rx1 − x3 = 0

In matrix form:0 0 −1 −2

2 −1 −2 −1

0 −2 0 −6

R 0 −1 0

x1

x2

x3

x4

=

6− 6(144/180Y )

3− 6(144/180Y )

7− 12(144/180Y )

0

The function file and the the session:


funtion E = ethanol(R,Y)% Computes ethanol produced% from yeast reaction.A = [0, 0,-1,-2; 2,-1,-2,-1; ...

0,-2, 0,-6; R, 0,-1, 0];b = [6 - 6*(144./(180*Y)); ...

3 - 6*(144./(180*Y)); ...7 - 12*(144./(180*Y)); 0];

x = A\b;E = x(4);

E_1 = ethanol(1.1, 0.5)E_2 = ethanol(1.05,0.5)

E_1 =0.0654

E_2 =-0.0717


Matrix Inverse

Ax = b

A−1A = AA−1 = I

⇒ A−1Ax = A−1b

x = A−1b


Example: Calculation Of Cable Tension

A mass m is suspended by three cables attached at the three points B, C, and D,as shown in the following figure. Let T1, T2, and T3 be the tensions in the threecables AB, AC, and AD, respectively. If the mass m is stationary, the sum of thetension components in the x, in the y, and in the z directions must each be zero.This requirement gives the following three equations:

T1√35− 3T2√

34+

T3√42

= 0

3T1√35− 4T3√

42= 0

5T1√35

+5T2√

34+

5T3√42−mg = 0

Use MATLAB to find T1, T2, and T3 in terms of an unspecified value of theweight mg.Solution: set mg = 1


A =

1√35

− 3√34

1√42

3√35

0 − 4√42

5√35

5√34

5√42

x =

T1

T2

T3

b =

001

% File cable.m% Computes the tensions in three cables.A1 = [1/sqrt(35), -3/sqrt(34), 1/sqrt(42)];A2 = [3/sqrt(35), 0, -4/sqrt(42)];A3 = [5/sqrt(35), 5/sqrt(34), 5/sqrt(42)];A = [A1; A2; A3];b = [ 0; 0; 1];x = A\b;disp(’The tension T_1 is:’)disp(x(1))disp(’The tension T_2 is:’)disp(x(2))disp(’The tension T_3 is:’)disp(x(3))

The tension T_1 is:0.5071 % 0.5071 mg




Example: The Matrix Inverse Method

Solve the following equations using the matrix inverse:

2x + 9y = 5

3x− 4y = 7

Solution:

A =

[2 93 −4

]Its determinant is |A| = 2(−4)− 9(3) = −35, and its inverse is

A−1 =1−35

[−4 −9−3 2

]=

135

[4 93 −2

]

The solution is

x = A−1b =135

[4 93 −2

] [57

]=

135

[831

]

or x = 83/85 = 2.3714 and y = 1/35 = 0.0286.


The Matrix Inverse in MATLAB

A = [2, 9; 3, -4];b = [5; 7]x = inv(A)*b

x =2.37140.0286



T6.2-1Use the matrix inverse method to solve the following set by hand

by using MATLAB: (Answer:x = 7, y = 4.)

3x− 4y = 5

6x− 10y = 2

T6.2-2Use the matrix inverse method to solve the following set by hand

and by using MATLAB:

3x− 4y = 5

6x− 8y = 2

(Answer: no solution.)


Cramer’s Method

a11x + a12y = b1

a21x + a22y = b2

⇒ a22(a11x + a12y) = a22b1

−a12(a21x + a22y) = −a12b2

⇒ x =b1a22 − b2a12

a22a11 − a12a21=

∣∣∣∣∣∣∣b1 a12

b2 a22

∣∣∣∣∣∣∣∣∣∣∣∣∣∣a11 a12

a21 a22

∣∣∣∣∣∣∣≡ D1

D

y =b2a11 − b1a21

a22a11 − a12a21=

∣∣∣∣∣∣∣a11 b1

a21 b2

∣∣∣∣∣∣∣∣∣∣∣∣∣∣a11 a12

a21 a22

∣∣∣∣∣∣∣≡ D2

D

Note: If D = 0 but D1 6= 0 then x is undefined

If D = 0 and D1 = 0 then x has infinitely many soln.s



T6.3-1Use Cramer’s method to solve for x and y in terms of the

parameter b. For what value of b is the set singular ?

4x− by = 5

−3x + 6y = 3

(Answer: x = (10 + b)/(8− b), y = 9/(8− b) unless b = 8.)


T6.2-2Use Cramer’s method to solve for y.

Use MATLAB to evaluate the determinants.

2x + y + 2z = 17

3y + z = 6

2x− 3y + 4z = 19

(Answer: y = 1.)


Under-determined Systems

An Under-determinedSystem does not containenough information to solvefor all of the unknownvariables (fewer equationsthan unknowns)

x = 6− 3y

A = [1, 3];b = 6;soln = A\b

soln = % infinite # solutions,0 % giving one of these solutions2 % (x is set to be zero --> y=2)

Note: An infinite number of solutions might exist even if # of equations = #of unknowns (if |A| = 0)

Matrix inverse method and Cramer’s method do not workLeft-division generates an error message warning

Use pseudo-inverse method: x = pinv(A)*bpinv command produces a solution with minimum Euclidean norm


2x− 4y + 5z = −4−4x− 2y + 3z = 4

2x + 6y − 8z = 0

A = [2,-4,5; -4,-2,3; 2,6,-8];b = [-4, 4, 0]’;soln_left_div = A\bsoln_pseudoin = pinv(A)*b

Warning: Matrix is singularto working precision.

soln_left_div =InfInfInf

soln_pseudoin =-1.21480.2074-0.1481


Matrix Rank

An m× n matrix A has a rank r ≥ 1 if and only if |A| contains a

nonzero r × r determinant and every square sub-determinant with

r + 1 or more rows is zero

Example: The rank of A =

3 −4 1

6 10 2

9 −7 3

is 2 because |A| = 0

whereas A contains at least one nonzero 2× 2 subdeterminant.

eg, |A| =

∣∣∣∣∣∣ 10 2

−7 3

∣∣∣∣∣∣ = 44

A = [3,-4,1; 6,10,2; 9,-7,3];rank(A)

ans =2


Existence and Uniqueness of Solutions

Existence and Uniqueness of SolutionsThe set Ax = b with m equations and n unknowns has solutions

if and only if rank(A) = rank([A b])

Let r = rank(A).

1. If previous condition is satisfied and if r = n,

then the solution is unique

2. If previous condition is satisfied and but r < n,

an infinite number of solution exists and r unknown variables can

be expressed as linear combination of the other n − r unknown

variables, whose values are arbitrary.


Homogeneous Case

1. For the homogeneous set Ax = 0, rank(A) = rank([A b])always, and thus the set always has the trivial solution x = 0.

2. A nonzero solution (at least one unknown is nonzero) exists if

and only if rank(A) < n

3. If m < n, the homogeneous set always has a nonzero solution


Example: A Set Having A Unique Solution

Determine whether the following set has a unique solution, and if

so,find it:

3x− 2y + 8z = 48

−6x + 5y + z = −12

9x + 4y + 2z = 24

Solution:

A =

3 −2 8

−6 5 1

9 4 2

b =

48

−12

24

x =

x

y

z


A = [3, -2, 8; -6, 5, 1; 9, 4, 2];b = [48; -12; 24];rank(A)

ans =3

rank([A b])

ans =3

x=A\b

x =2-15



T6.4-1Use MATLAB to show that the following set has a unique solution

and then find the solution:

3x + 12y − 7z = 5

5x− 6y − 5z = −8

−2x + 7y + 9z = 5

(Answer: The unique solution is

x = −1.0204, y = 0.5940, z = −0.1332.)


The Minimum Euclidean Norm Solution

The pinv command can obtain a solution of an

underdetermined set

v = [x y z]

N =√

vTv =

√√√√√√√[x y z]T

x

y

z

=√

x2 + y2 + z2


Example: An Under-determined Set

Show that the following set does not have a unique solution. How many of theunknowns will be undetermined ? Interpret the results given by the left-divisionmethod.

2x− 4y + 5z = −4

−4x− 2y + 3z = 4

2x + 6y − 8z = 0

Solution:A = [2, -4, 5; -4, -2, 3;...

2, 6, -8];b = [-4; 4; 0];r1 = rank(A)r2 = rank([A b])

r1 =2

r2 =2

⇒ infinite no. of solutionssoln = pinv(A)*b

soln =-1.21480.2074-0.1481


Ex: A Statically-Indeterminate Problem

Determine the forces in the three equally spaced

supports that hold up a light fixture. The supports

are 5 feet apart. The fixture weights 400 pounds,

and its mass center is 4 feet from the right end.

(a) Solve the problem by hands. (b) obtain the

solution using the MATLAB left-division method

and the pseudoinverse method.

Solution:vertical force must cancel; total moments about right endpoint are zero


T1 + T2 + T3 − 400 = 0

400(4)− 10T1 − 5T2 = 0

or T1 + T2 + T3 = 400

10T1 + 5T2 + 0T3 = 1600

[1 1 1

10 5 0

]︸︷︷︸

A

T1

T2

T3

︸︷︷︸

x

=

[400

1600

]︸︷︷︸

b

[A b] =

[1 1 1 400

10 5 0 1600

]

T2 =1600− 10T1

5= 320− 2T1

T1 = T3 − 80T2 = 320− 2T1 = 320− 2(T3 − 80) = 480− 2T3


A = [1, 1, 1; 10, 5, 0];b = [400; 1600];rank(A)

ans =2

rank([A b])

ans =2

soln_left = A\bsoln_pseu = pinv(A)*b

soln_left =160.00000

240.0000soln_pseu =

93.3333 % min norm soln133.3333173.3333

norm_left = sqrt(sum(soln_left.^2))norm_pseu = sqrt(sum(soln_pseu.^2))

norm_left =288.4441

norm_pseu =237.7674



T6.4-2Use MATLAB to find two solutions to the following set:

x + 3y + 2z = 2

x + y + z = 4

(Answer:

Minimum-norm solution: x = 4.33,y = −1.67,z = 1.34.

Left-division solution: x = 5,y = −1,z = 0.)


The Reduced Row Echelon Form

T1 = T3 − 80T2 = 480− 2T3

⇒ T1 − T3 = −80T2 + 2T3 = 480

⇒

[1 0 −10 1 2

] T1

T2

T3

=

[−80480

]

⇒

[1 0 −1 −800 1 2 480

]rref([A b]) command provides a procedure to reduce an underdetermined set to

such a reduced row echelon form

Its output is the augmented matrix [C d] that corresponds to the equation set

Cx = d


Example: A Singular Set

The following under-determined equation set was analyzed in

previous Example. There it was shown that an infinite number of

solutions exists. Use the pinv and the rref commands to obtain

solutions.

2x− 4y + 5z = −4

−4x− 2y + 3z = 4

2x + 6y − 8z = 0

Solution:


A = [2, -4, 5; -4, -2, 3;...2, 6, -8];

b = [-4; 4; 0];x = pinv(A)*b

x =-1.21480.2074-0.1481

rref([A b])

ans =1 0 -0.1 -1.20000 1 -1.3 0.40000 0 0 0

The answer corresponds to the augmented matrix [C d],

[C d] =

1 0 −0.1 −1.20 1 −1.3 0.40 0 0 0

The matrix corresponds to the matrix equation Cx = d, or

x + 0y − 0.1z = −1.20x + y − 1.3z = 0.40x + 0y + 0z = 0.0

⇒x = 0.1z − 1.2y = 1.3z + 0.4z = arbitrary value


Example: Production Planning

The following table shows how many hours reactors A and B need to produce 1ton each of the chemical products 1, 2, and 3. The two reactors are available for40 hours and 30 hours per week, respectively. Determine how many tons of eachproduct can be produced each week.

Hours Product 1 Product 2 Product 3

Reactor A 5 3 3

Reactor B 3 3 4

Solution:

5x + 3y + 3z = 40

3x + 3y + 4z = 30

A =

[5 3 33 3 4

]b =

[4030

]x =

x

y

z


Note: rank(A) = rank([A b]) = 2, which is less than the number of unknowns(3). Thus an infinite number of solution exists, and we can determine two of thevariables in terms of the third.Using rref([A b]), where A=[5,3,3; 3,3,4] and b=[40;30], we obtain the followingreduced echelon augmented matrix,

[1 0 −0.5 50 1 1.8333 5

]

x− 0.5z = 5

y + 1.8333z = 5

⇒ x = 5 + 0.5z

y = 5− 1.8333z

Suppose we make a profit of $400, $600, $100 per ton for products 1, 2 and 3,


respectively.

P = 400x + 600y + 100z

= 400(5 + 0.5z) + 600(5− 1.8333z) + 100z

= 5000− 800z

To maximize profit P , choose z = 0 ⇒ x = y = 5 tons.


Example: Traffic Engineering

A traffic engineer wants to know whether measurements of traffic flow entering

and leaving a road network are sufficient to predict the traffic flow on each street

in the network. For example, consider the network of one-way streets shown in the

following Figure. The numbers in the figure give the measured traffic flows in

vehicles per hour. Assume that no vehicles park anywhere within the network. If

possible, calculate the traffic f1, f2, f3, and f4. If this is not possible, suggest

how to obtain the necessary information.


Solution:

100 + 200 = f1 + f4

f1 + f2 = 300 + 200

600 + 400 = f2 + f3

f3 + f4 = 300 + 500

A =

1 0 0 11 1 0 00 1 1 00 0 1 1

b =

300500

1000800

x =

f1

f2

f3

f4

rref([A b]) ⇒

1 0 0 1 3000 1 0 −1 2000 0 1 1 8000 0 0 0 0

⇒f1 = 300− f4

f2 = 200 + f4

f3 = 800− f4



T6.4-3Use the rref and pinv commands and the left-division method to solve thefollowing set:

3x + 5y + 6z = 6

8x− y + 2z = 1

5x− 6y − 4z = −5

(Answer: The set has an infinite number of solutions. The result obtainedwith the rref commands is x = 0.2558 − 0.3721z, y = 1.0465 − 0.9767z, z isarbitrary. The pinv commands gives x = 0.0571, y = 0.5249, z = 0.5340. Theleft-division method generates an error message.)


T6.4-4Use the rref and pinv commands and the left-division method to solve thefollowing set:

3x + 5y + 6z = 4

x− 2y − 3z = 10

(Answer: The set has an infinite number of solutions. The result obtained withthe rref commands is x = 0.2727z +5.2727− 0.3721z, y = −1.3636z− 2.3636,z is arbitrary. The pinv commands gives x = 4.8000, y = 0, z = −1.7333. Thepseudoinverse method gives x = 4.8396, y = −0.1972, z = −1.5887.)


Overdetermined Systems: The LeastSquares Method

Suppose we have the following three data points, and we want to find the straightline y = mx + b that best fits the data in some sense.

x y

0 2

5 6

10 11

(a) Find the coefficients m and b by using the least squares criterion. (b) Find the

coefficients by using MATLAB to solve the three equations (one for each data

point) for the two unknowns m and b. Compare the answers from (a) and (b).

Solution:


J =i=3∑i=1

(mxi + b− yi)2

J = (0m + b− 2)2 + (5m + b− 6)2 + (10m + b− 11)2


∂J

∂m= 2(5m + b− 6)(5) + 2(10m + b− 11)(10)

= 250m + 30b− 280 = 0∂J

∂b= 2(b− 2) + 2(5m + b− 6) + 2(10m + b− 11)

= 30m + 6b− 38 = 0

250m + 30b = 28030m + 6b = 38

⇒ m = 0.9, b =116

(y = 0.9x + 11/6)

Evaluate y = mx + b at each data point:

0m + b = 2

5m + b = 6

10m + b = 11

CL Chen PSE LAB NTU 58 0 15 1

10 1

[m

b

]=

26

11

A =

0 15 1

10 1

x =

[m

b

]b =

26

11

A = [0, 1; 5, 1; 10, 1];b = [2; 6; 11];rank(A)

ans =2

rank([A b])

ans =3

A\b

ans =0.90001.8333

A*ans

ans =1.8336.33310.8333


Example: An Over-determined Set

(a) Solve the following equations by hand and (b) solve them using MATLAB.Discuss the solution for two cases: c = 9 and c = 10.

x + y = 1x + 2y = 3x + 5y = c

Solution:

A =

1 11 21 5

[A b] =

1 1 11 2 31 5 c

c = 9 ⇒ rank(A) = rank([A b]) = 2⇒ A\b gives unique solution x = −1, y = 2


c = 10 ⇒ rank(A) = 2, rank([A b]) = 3⇒ A\b gives least squares solution x = −1.3846, y = 2.2692

J = (x + y − 1)2 + (x + 2y − 3)2 + (x + 5y − 10)2


Fitting Models by Least Squares

inputs = xi1, xi2, · · · , xip output = yi, i = 1, . . . , n

(y1 ; x11, x12, · · · , x1p) (1st observation data)

(y2 ; x21, x22, · · · , x2p) (2nd observation data)

... ...

(yn ; xn1, xn2, · · · , xnp) (nth observation data)

y = β1x1 + β2x2 + · · ·+ βpxp (linear model)

y1 = β1x11 + β2x12 + · · ·+ βpx1p + ε1

y2 = β1x21 + β2x22 + · · ·+ βpx2p + ε2... ...

yn︸︷︷︸obs

= β1xn1 + β2xn2 + · · ·+ βpxnp︸︷︷︸model output

+ εn︸︷︷︸error


Fitting Models by Least Squares

f =n∑

i=1

εi2 =

n∑i=1

yi −p∑

j=1

βjxij

2

(SSE)

Q: find β1, · · · , βp to minimize SSE ?∂f∂β1

∣∣∣β∗=β∗

= 0 ∂f∂β2

∣∣∣β∗=β∗

= 0 · · · ∂f∂βp

∣∣∣β∗=β∗

= 0n∑

i=1

yixi1 = β1

n∑i=1

xi1xi1 + β2

n∑i=1

xi1xi2 + · · ·+ βp

n∑i=1

xi1xip

n∑i=1

yixi2 = β1

n∑i=1

xi2xi1 + β2

n∑i=1

xi2xi2 + · · ·+ βp

n∑i=1

xi2xip

... ... ...n∑

i=1

yixip = β1

n∑i=1

xipxi1 + β2

n∑i=1

xipxi2 + · · ·+ βp

n∑i=1

xipxip


Fitting Models by Least Squares⇒ Q: how to find β1, · · · , βp?

y1

y2

...

yn

︸︷︷︸obs.s

=

x11 x12 · · · x1p

x21 x22 · · · x2p

... ... ... ...

xn1 xn2 · · · xnp

β1

β2

...

βp

︸︷︷︸

model outputs

+

ε1

ε2...

εp

Y = Xβ + ε

f = εTε = (Y −Xβ)T (Y −Xβ) (SSE)

∂f

∂β

∣∣∣∣ ˆβ= xTY − XTXβ = 0

β = (XTX)−1XTY



T6.5-1Use MATLAB to solve the following set:

x− 3y = 2

3x + 5y = 7

70x− 28y = 153

(Answer: The unique solution, x = 2.2143, y = 0.0714, is given by theleft-division method.)

T6.5-2Use MATLAB to solve the following set:

x− 3y = 2

3x + 5y = 7

5x− 2y = −4

(Answer: no exact solution.)


A MATLAB Program to Solve Linear Equations


% Script file lineq.m% Solve the set Ax=b, given A and b% Check the ranks of A and [A b]if rank(A) == rank([A b]) % A, [A b]: equal ranks

size_A = size(A);if rank(A) == size_A(2) % rank of A = no unknowns

disp(’There is a unique solution: ’)x = A\b % solve using left division

else % rank of A # no unknownsdisp(’There is an infinite no of solutions.’)disp(’The augmented matrix of reduced system: ’)rref([A b]) % compute augmented matrix

endelse % A, [A b]: not equal ranks

disp(’There are no solutions ’)end


A = [1, -1; 1, 1]; b = [3; 5];lineq

There is a unique solution:x =

41

A = [1, -1; 2, -2]; b = [3; 6];lineq

There is an infinite no of solutions.The augmented matrix of reduced system:ans =

1 -1 30 0 0

A = [1, -1; 2, -2]; b = [3; 5];lineq

There are no solutions

6 Linear Algebraic Equations.pdf

Documents