Lesson 28: Lagrange Multipliers II

Lesson 28 (Sections 18.2–5)Lagrange Multipliers II

Math 20

November 28, 2007

Announcements

I Problem Set 11 assigned today. Due December 5.

I next OH: Today 1–3 (SC 323)

I Midterm II review: Tuesday 12/4, 7:30-9:00pm in Hall E

I Midterm II: Thursday, 12/6, 7-8:30pm in Hall A

Outline

A homework problem

Restating the Method of Lagrange MultipliersStatementJustifications

Second order conditionsCompact feasibility setsAd hoc argumentsAnalytic conditions

Example: More than two variables

More than one constraint

Problem 17.1.10

ProblemMaximize the quantity f (x , y , z) = Axaybzc subject to theconstraint that px + qy + rz = m. (Here A, a, b, c , p, q, r ,m arepositive constants.)

Solution (By elimination)

Solving the constraint for z in terms of x and y, we get

z =m − px − qy

r

So we optimize the unconstrained function

f (x , y) =A

r cxayb(m − px − qy)c

Problem 17.1.10

ProblemMaximize the quantity f (x , y , z) = Axaybzc subject to theconstraint that px + qy + rz = m. (Here A, a, b, c , p, q, r ,m arepositive constants.)

Solution (By elimination)

Solving the constraint for z in terms of x and y, we get

z =m − px − qy

r

So we optimize the unconstrained function

f (x , y) =A

r cxayb(m − px − qy)c

We have

∂f (x , y)

x=

A

r c

[axa−1yb(m − px − qy)c + xaybc(m − px − qy)c−1(−p)

]=

A

r cxa−1yb(m − px − qy)c−1 [a(m − px − qy)− cpx ]

Likewise

∂f (x , y)

y=

A

r cxayb−1(m − px − qy)c−1 [b(m − px − qy)− cqy ]

So throwing out the critical points where x = 0, y = 0, or z = 0(these give minimal values of f , not maximal), we get

(a + c)px + aqy = am

bpx + (b + c)qy = bm

This is a fun exercise in Cramer’s Rule:

x =

∣∣∣∣am aqbm (b + c)q

∣∣∣∣∣∣∣∣(a + c)p aqbp (b + c)q

∣∣∣∣ =

amq

∣∣∣∣1 1b b + c

∣∣∣∣pq

∣∣∣∣a + c ab b + c

∣∣∣∣=

amqc

pq(ac + bc + c2)=

m

p

a

a + b + c

It follows that

y =m

q

b

a + b + cz =

m

r

c

a + b + c

If this is a utility-maximization problem subject to fixed budget,the portion spent on each good (px

m , for instance) is the relativedegree to which that good multiplies utility ( a

a+b+c ).

Outline

A homework problem

Restating the Method of Lagrange MultipliersStatementJustifications

Second order conditionsCompact feasibility setsAd hoc argumentsAnalytic conditions

Example: More than two variables

More than one constraint

Theorem (The Method of Lagrange Multipliers)

Let f (x1, x2, . . . , xn) and g(x1, x2, . . . , xn) be functions of severalvariables. The critical points of the function f restricted to the setg = 0 are solutions to the equations:

∂f

∂xi(x1, x2, . . . , xn) = λ

∂g

∂xi(x1, x2, . . . , xn) for each i = 1, . . . , n

g(x1, x2, . . . , xn) = 0.

Note that this is n + 1 equations in n + 1 variables x1, . . . , xn, λ.

Graphical Justification

In two variables, the critical points of f restricted to the level curveg = 0 are found when the tangent to the the level curve of f isparallel to the tangent to the level curve g = 0.

These tangents have slopes(dy

dx

)f

= − f ′xf ′y

and

(dy

dx

)g

= −g ′xg ′y

So they are equal when

f ′xf ′y

=g ′xg ′y

=⇒ f ′xg ′x

=f ′yg ′y

or

f ′x = λg ′x

f ′y = λg ′y

These tangents have slopes(dy

dx

)f

= − f ′xf ′y

and

(dy

dx

)g

= −g ′xg ′y

So they are equal when

f ′xf ′y

=g ′xg ′y

=⇒ f ′xg ′x

=f ′yg ′y

or

f ′x = λg ′x

f ′y = λg ′y

Symbolic Justification

Suppose that we can use the relation g(x1, . . . , xn) = 0 to solve forxn in terms of the the other variables x1, . . . , xn−1, after makingsome choices. Then the critical points of f (x1, . . . , xn) areunconstrained critical points of f (x1, . . . , xn(x1, . . . , xn−1)).

f

x1 x2 · · · xn

x1 x2 · · · xn−1

Now for any i = 1, . . . , n − 1,(∂f

∂xi

)g

=∂f

∂xi+

∂f

∂xn

(∂xn

∂xi

)g

=∂f

∂xi− ∂f

∂xn

∂g/∂xi

∂g/∂xn

If(∂f∂xi

)g

= 0, then

∂f /∂xi

∂f /∂xn=∂g/∂xi

∂g/∂xn⇐⇒ ∂f /∂xi

∂g/∂xi=∂f /∂xn

∂g/∂xn

So as before,∂f

∂xi= λ

∂g

∂xifor all i .

Another perspective

To find the critical points of f subject to the constraint thatg = 0, create the lagrangian function

L = f (x1, x2, . . . , xn)− λg(x1, x2, . . . , xn)

If L is restricted to the set g = 0, L = f and so the constrainedcritical points are unconstrained critical points of L . So for each i ,

∂L

∂xi= 0 =⇒ ∂f

∂xi= λ

∂g

∂xi.

But also,∂L

∂λ= 0 =⇒ g(x1, x2, . . . , xn) = 0.

Outline

A homework problem

Restating the Method of Lagrange MultipliersStatementJustifications

Second order conditionsCompact feasibility setsAd hoc argumentsAnalytic conditions

Example: More than two variables

More than one constraint

Second order conditions

The Method of Lagrange Multipliers finds the constrained criticalpoints, but doesn’t determine their “type” (max, min, neither).So what then?

A dash of topologyCf. Sections 17.2–3

DefinitionA subset of Rn is called closed if it includes its boundary.

x2 + y2 ≤ 1closed

x2 + y2 ≤ 1not closed

y ≥ 0closed

Basically, if a subset is described by ≤ or ≥ inequalities, it is closed.

A dash of topologyCf. Sections 17.2–3

DefinitionA subset of Rn is called closed if it includes its boundary.

x2 + y2 ≤ 1closed

x2 + y2 ≤ 1not closed

y ≥ 0closed

Basically, if a subset is described by ≤ or ≥ inequalities, it is closed.

DefinitionA subset of Rn is called bounded if it is contained within someball centered at the origin.

x2 + y2 ≤ 1bounded

x2 + y2 ≤ 1bounded

y ≥ 0not bounded

DefinitionA subset of Rn is called compact if it is closed and bounded.

x2 + y2 ≤ 1compact

x2 + y2 ≤ 1not compact

y ≥ 0not compact

Optimizing over compact sets

Theorem (Compact Set Method)

To find the extreme values of function f on a compact set D ofRn, it suffices to find

I the (unconstrained) critical points of f “inside” D

I the (constrained) critical points of f on the “boundary” of D.

Ad hoc arguments

If D is not compact, sometimes it’s still easy to argue that as xgets farther away, f becomes larger, or smaller, so the criticalpoints are “obviously” maxes, or mins.

(Example later)

Ad hoc arguments

If D is not compact, sometimes it’s still easy to argue that as xgets farther away, f becomes larger, or smaller, so the criticalpoints are “obviously” maxes, or mins.(Example later)

Analytic conditionsRecall Equation 16.13, cf. Section 18.4

I For the two-variable constrained optimization problem, wehave (look in the book if you want the gory details):

(d2f

dx2

)g

=

∣∣∣∣∣∣0 g ′x g ′yg ′x f ′′xx − λg ′′xx f ′′xy − λg ′′xyg ′y f ′′yx − λg ′′yx g ′′yy − λg ′′yy

∣∣∣∣∣∣ =

∣∣∣∣∣∣L ′′λλ L ′′

λx L ′′λy

L ′′xλ L ′′

xx L ′′xy

L ′′yλ L ′′

yx L ′′yy

∣∣∣∣∣∣The critical point is a local max if this determinant isnegative, and a local min if this is positive.

I The matrix on the right is the Hessian of the Lagrangian. Butthere is still a distinction between this and the unconstrainedcase. The constrained extrema are critical points of theLagrangian, not extrema.

I Don’t worry too much about this!

Outline

A homework problem

Restating the Method of Lagrange MultipliersStatementJustifications

Second order conditionsCompact feasibility setsAd hoc argumentsAnalytic conditions

Example: More than two variables

More than one constraint

Problem 17.1.10

ProblemMaximize the quantity f (x , y , z) = Axaybzc subject to theconstraint that px + qy + rz = m. (Here A, a, b, c , p, q, r ,m arepositive constants.)

SolutionThe Lagrange equations are

Aaxa−1ybzc = λp

Abxayb−1zc = λq

Acxaybzc−1 = λr

We rule out any solution with x, y , z, or λ equal to 0 (they willminimize f , not maximize it).

Problem 17.1.10

ProblemMaximize the quantity f (x , y , z) = Axaybzc subject to theconstraint that px + qy + rz = m. (Here A, a, b, c , p, q, r ,m arepositive constants.)

SolutionThe Lagrange equations are

Aaxa−1ybzc = λp

Abxayb−1zc = λq

Acxaybzc−1 = λr

We rule out any solution with x, y , z, or λ equal to 0 (they willminimize f , not maximize it).

Dividing the first two equations gives

ay

bx=

p

q=⇒ y =

bp

aqx

Dividing the first and last equations gives

az

cx=

p

r=⇒ z =

cp

arx

Plugging these into the equation of constraint gives

px +bp

ax +

cp

ax = m =⇒ x =

m

p

a

a + b + c

Outline

A homework problem

Restating the Method of Lagrange MultipliersStatementJustifications

Second order conditionsCompact feasibility setsAd hoc argumentsAnalytic conditions

Example: More than two variables

More than one constraint

General method for more than one constraint

If we are optimizing f (x1, . . . , xn) subject to gj(x1, . . . , xn) ≡ 0,j = 1, . . . ,m we need multiple lambdas for them. The newLagrangian is

L (x1, . . . , xn) = f (x1, . . . , xn)−m∑

j=1

λjgj(x1, . . . , xn)

The conditions are that ∂L∂xi

= 0 and ∂L∂λj

= 0 for all i and j . In

other words,

∂f

∂xi= λ1

∂g1

∂xi+ · · ·+ λm

∂gm

∂xi(all i)

gj(x1, . . . , xn) = 0 (all j)

Example

Find the minimum distance between the curves xy = 1 andx + 2y = 1.

Reframing this, we can minimize

f (x , y , u, v) = (x − u)2 + (y − v)2

subject to the constraints

xy − 1 = 0 u + 2v = 1.

Example

Find the minimum distance between the curves xy = 1 andx + 2y = 1.

Reframing this, we can minimize

f (x , y , u, v) = (x − u)2 + (y − v)2

subject to the constraints

xy − 1 = 0 u + 2v = 1.

xy = 1

x + 2y = 1

•

•

•

•

••

• •

The Lagrangian is

L = (x − u)2 + (y − v)2 − λ(xy − 1)− µ(u + 2v − 1)

So the Lagrangian equations are

2(x − u) = λy −2(x − u) = µ

2(y − v) = λx −2(y − v) = 2µ

Dividing the two λ equations and the two µ equations gives

x − u

y − v=

y

x

x − u

y − v=

1

2.

Since the left-hand-sides are the same, we have 2y = x . Sincexy = 1, we can say either x =

√2, y = 1√

2, or x = −

√2, y = − 1√

2

Suppose x =√

2, y = 1√2

. Then

√2− u

1√2− v

=1

2=⇒ 2u − v =

3√

2

2

This along with u + 2v = 1 gives

u =1

5

(1 + 3

√2)

v =1

10

(4− 3

√2)

If we instead choose x = −√

2, y = − 1√2

, we get

u =1

5

(1− 3

√2)

v =1

5

(2 +

3√2

)

xy = 1

x + 2y = 1

•

•

•

•

1

5

(9− 4

√2)

1

5

(9 + 4

√2)

Because f gets larger as x , y , u, and v get larger, the absoluteminimum is the smaller of these two critical values. So theminimum distance is 1

5

(9− 4

√2).