Conjugate Gradient and Linear Equality Constraints

Conjugate Gradient and Linear Equality Constraints

Lecture III

A Slightly More Complicated ProblemStarting with a hyperbolic tangent production

function

We define the profit function as

1 2 3 1 2

3 1 2 2 2

3 3 1 2 1 3 2 3

15, , 1 tanh 0.50 0.2624 0.31252

0.3500 0.0025 0.0028

0.0025 0.00035 0.0028 0.000032

f x x x x x

x x x x x

x x x x x x x x

1 2 3 1 2 3 1 2 3max , , , , 0.375 0.450 0.475x x x f x x x x x x

As a starting point, we assume

Starting with some feasible step s1

3kB I

21 1 1 1 1( ) ( ) ( )x x xxx s x x s

If we let 1 1( ) 0x x s

21 1 1

121 1 1

( ) ( ) 0

( ) ( )

x xx

xx x

x x s

s x x

In the hyperbolic tangent formulation

1 2 3 1 2 3

1 2 3 1 2 3

1 2 3 1 2 3

150.375 0.2625 0.005 0.00035 0.00028 , ,2

150.450 0.3125 0.00035 0.00560 0.000032 , ,2150.475 0.350 0.00028 0.000032 0.00500 , ,2

x

x x x h x x x

x x x x h x x x

x x x h x x x

1 2 3 1 2 3 1 2

2 2 3 3 1 2 1 3 2 3

, , sech 0.50 0.2624 0.3125 0.3500 0.0025

0.0028 0.0025 0.00035 0.0028 0.000032

h x x x x x x x x

x x x x x x x x x x

Let us start by considering

1 1 1 1x

11.258721.494801.71050

x x

Given that B1 is the identity matrix

Thus, in this case we have

1 3 1

1.258721.494801.71050

xs I x

2 1 1

1 1.25872 2.258721 1.49480 2.494801 1.71050 2.71050

x x s

Next, to update the Hessian matrix following the BFGS algorithm we have

1 1

1

1

( ) ( )

0.16973( ) 0.20692

0.20127

1.258721.494801.71050

0.16973 1.25872 1.428450.20692 1.49480 1.701720.20127 1.71050 1.91177

k x k k x k

x

x

y x s x

x s

x

y

Thus, following the BFGS update for the Hessian matrix

1

1

1 1' '' '

1.03315 0.04038 0.039540.04038 1.04915 0.048300.03954 0.04830 1.04636

k k k k k k k kk k k k k

B B B s s B y ys B s y s

B

Updating the solution, next we have the gradient at the new solution as

1 1 2

0.16973( ) ( ) 0.20692

0.20127x xx s x

Updating the point of approximation

3 2 2

2.25872 0.150297 2.108422.49480 0.183235 2.311362.71050 0.178212 2.53229

x x s

Updating the gradient

2 2 3

0.10095( ) ( ) 0.12524

0.10956x xx s x

2 2 2 2

0.06878( ) ( ) 0.08168

0.09172x xy x s x

New Hessian

2

0.8567 0.1783 0.15320.1783 0.7783 0.19150.1532 0.1915 0.8407

B

R-Codefr <- function(b){-15/2*(1+tanh(-0.50+0.2625*b[1]+0.3125*b[2]+0.3500*b[3]- 0.0025*b[1]*b[1]-0.0028*b[2]*b[2]-0.0025*b[3]*b[3]+ 0.00035*b[1]*b[2]+0.00028*b[1]*b[3]+0.000032*b[2]*b[3])) +0.375*b[1]+0.450*b[2]+0.475*b[3]}

dfr <- function(b){ dd <- cosh(0.50-0.2625*b[1]-0.3125*b[2]-

0.3500*b[3]+0.0025*b[1]*b[1]+0.0028*b[2]*b[2]+ 0.0025*b[3]*b[3]-0.00035*b[1]*b[2]-0.00028*b[1]*b[3]-0.000032*b[2]*b[3]) as.matrix(cbind(0.375+(15/2)*(-0.2625+0.00500*b[1]-0.00035*b[2]-0.00028*b[3])/

(dd*dd), 0.450+(15/2)*(-0.3125-0.00035*b[1]+0.00560*b[2]-0.000032*b[3])/(dd*dd), 0.475+(15/2)*(-0.3500-0.00028*b[1]-0.000032*b[2]+0.0050*b[3])/(dd*dd)))}

b0 <- cbind(1.0,1.0,1.0)

res.fr <- optim(b0,fr,dfr,method="BFGS")

print(res.fr)

Results$par [,1] [,2] [,3][1,] 1.062955 0.4756912 4.537824

$value[1] -11.46800

$countsfunction gradient 28 23

$convergence[1] 0

$messageNULL

Starting with a simple 2*3 example, assume that we have the matrix equation

Note that by row operations the A matrix can be transformed to

1

2

3

1 1 1 50 1 1 6

xAx b x

x

1 1 1 1 1 1 0 20 1 0 1 1 0 1 1

This expression implies the following homogeneous relationships:

Setting x3=1 yields

1 3

2 3

2 00

x xx x

1

2

21

xx

Or in vector form:

Next we confirm that z is the nullspace.

211

z

We do this by confirming that both vectors of the matrix are orthogonal to the nullspace:

21 1 1 1 2 1 1 0

1

20 1 1 1 0 1 1 0

1

Implications of the nullspace:Start by defining a feasible solution:

5 1 1 1 1 1 5 1 0 2 116 0 1 0 1 1 6 0 1 1 6

111 1 1 11 6 0 5

60 1 1 6 0 6

0

Ax

Next, we would like to generate another feasible point based on this solution and the nullspace matrix:

Letting yields p = 5

1

*2

3

11 26 10 1

xx x Zp x p

x

1

*2

3

11 2 11 10 16 1 5 6 5 10 1 0 5 5

xx x Zp x

x

Checking the original solution:

11 1 1 1 1 5 5

10 1 1 0 1 5 6

5