Math Boot Camp Part II UIC Economics Departmentssc.wisc.edu/~hembre/wp-content/uploads/2015/04/MathCamp_Part… · Plug back in to get (-1,-1), (0,0), (1,1). Erik Hembre Math Boot

IntroductionOptimization

Math Boot Camp Part IIUIC Economics Department

Erik Hembre

August 21st, 2015

Erik Hembre Math Boot Camp Part II UIC Economics Department


Introduction

Welcome!

Let’s do some quick introductions.Just a few comments/suggestions about your first year:

Get on top of things!It will be difficult. That’s ok.Push yourself. High returns on investment.Find a good study group. Learn how to learn by yourself, withother, and to others.Work on your weaknesses. Focus on your interests.Grades << Knowledge



Introduction

Welcome!

Let’s do some quick introductions.

Just a few comments/suggestions about your first year:




Introduction

Welcome!





Introduction

Welcome!


Get on top of things!

It will be difficult. That’s ok.Push yourself. High returns on investment.Find a good study group. Learn how to learn by yourself, withother, and to others.Work on your weaknesses. Focus on your interests.Grades << Knowledge



Introduction

Welcome!


Get on top of things!It will be difficult. That’s ok.

Push yourself. High returns on investment.Find a good study group. Learn how to learn by yourself, withother, and to others.Work on your weaknesses. Focus on your interests.Grades << Knowledge



Introduction

Welcome!


Get on top of things!It will be difficult. That’s ok.Push yourself. High returns on investment.

Find a good study group. Learn how to learn by yourself, withother, and to others.Work on your weaknesses. Focus on your interests.Grades << Knowledge



Introduction

Welcome!


Get on top of things!It will be difficult. That’s ok.Push yourself. High returns on investment.Find a good study group. Learn how to learn by yourself, withother, and to others.

Work on your weaknesses. Focus on your interests.Grades << Knowledge



Introduction

Welcome!


Get on top of things!It will be difficult. That’s ok.Push yourself. High returns on investment.Find a good study group. Learn how to learn by yourself, withother, and to others.Work on your weaknesses. Focus on your interests.

Grades << Knowledge



Introduction

Welcome!





Introduction

Today:

The purpose of today is a brief review some relevantmathematics concepts.

Again: REVIEW. Its ok if you don’t remember or haven’tlearned the material.Constrained/Unconstrained Optimization.Highly recommend: Simon and Blume: Mathematics forEconomists

Optimization: Ch: 16-19Also Consider: Ch: 1-4, 12-14, 20-22. Appendix A1.

Also consider: Real Analysis: A First Course (by RussellGordon). For an introduction to real analysis.



Introduction

Today:

The purpose of today is a brief review some relevantmathematics concepts.Again: REVIEW. Its ok if you don’t remember or haven’tlearned the material.

Constrained/Unconstrained Optimization.Highly recommend: Simon and Blume: Mathematics forEconomists





Introduction

Today:

The purpose of today is a brief review some relevantmathematics concepts.Again: REVIEW. Its ok if you don’t remember or haven’tlearned the material.Constrained/Unconstrained Optimization.

Highly recommend: Simon and Blume: Mathematics forEconomists





Introduction

Today:

The purpose of today is a brief review some relevantmathematics concepts.Again: REVIEW. Its ok if you don’t remember or haven’tlearned the material.Constrained/Unconstrained Optimization.Highly recommend: Simon and Blume: Mathematics forEconomists





Introduction

Applications for Optimization problems are everywhere ineconomics:

Which set of goods to buy?How much/what ratio of inputs for production?How to allocate time between leisure and work?

Economics is all about solving decision problems. This isinherently linked to optimization. Typically there is a scareresource which constrains the choice set.



Introduction

Applications for Optimization problems are everywhere ineconomics:

Which set of goods to buy?How much/what ratio of inputs for production?How to allocate time between leisure and work?

Economics is all about solving decision problems. This isinherently linked to optimization. Typically there is a scareresource which constrains the choice set.



Unconstrained OptimizationConstrained Optimization

Optimization

General Problem:Maximize f (x) : x ∈ C where C is a non-empty subset of dom(f ).

Definition

A point x∗ is a maximum of f on C if f (x∗) ≥ f (x)∀x ∈ C .




Optimization

Definition

x∗ is a critical point of f if f ′(x) = 0

The first-order condition (FOC) for x∗ to be a maximum (orminimum) of a function f (x) is that x∗ is a critical point.

x∗ must be on the interior of dom(f ).

The same FOC holds for a function F of n variables.

If ∂F∂xi

= 0∀i ∈ n, then x∗ satisfies the FOC for F .




Optimization

To determine whether a critical value x∗ is a local minimumor local maximum (or neither), we use conditions on thesecond derivative on function f .




Optimization

If Hess(f ) is a negative definite matrix at x∗, then x∗ is alocal maximum value for f .

In the one-dimensional case, this is the same as if f ′′(x∗) < 0.

Definition

A matrix is positive definite if xTAx > 0∀x 6= 0.

Definition

A matrix is negative definite if xTAx < 0∀x 6= 0.




Optimization

If Hess(f ) is indefinite at x∗, then x∗ is a “saddle point”, meaninga maximum going in some directions, and a minimum going inothers.




Optimization

Let’s do a couple quick examples:

f (x) = −x2

f (x) = 3x + ln(x)

f (x) = 3− 4x + 2x2




Optimization

What about in the two dimensional case?

Consider a 2x2 symmetric matrix: A =

[a bb c

]

A is positive definite if a > 0 and detA > 0⇒ ac − b2 > 0

A is negative definite if a < 0 and ac − b2 > 0.




Optimization

What about in the two dimensional case?

Consider a 2x2 symmetric matrix: A =

[a bb c

]A is positive definite if a > 0 and detA > 0⇒ ac − b2 > 0

A is negative definite if a < 0 and ac − b2 > 0.




Unconstrained Optimziation

Again, lets start with an easier example:

f (x , y) = −x2 − y2




Unconstrained Optimization

Lets solve: f (x) = x4 + x2 − 6xy + 3y2

First lets find the critical values:

∂f

∂x= 4x3 + 2x − 6y = 0

∂f

∂y= −6x + 6y = 0

fy = 0⇒ x = y , and fx = 0⇒ 4x3 = 4x ⇒ x3 = x .Critical values: x=-1,0,1. Plug back in to get (-1,-1), (0,0), (1,1).







∂f

∂x= 4x3 + 2x − 6y = 0

∂f

∂y= −6x + 6y = 0








∂f

∂x= 4x3 + 2x − 6y = 0

∂f

∂y= −6x + 6y = 0

fy = 0⇒ x = y , and fx = 0⇒ 4x3 = 4x ⇒ x3 = x .

Critical values: x=-1,0,1. Plug back in to get (-1,-1), (0,0), (1,1).







∂f

∂x= 4x3 + 2x − 6y = 0

∂f

∂y= −6x + 6y = 0






Hessian: Hess =

[12x2 − 2x− 6−6 6

]





Hessian: Hess =

[12x2 − 2x− 6−6 6

]




Weierstrauss Theorem

Will every function have a maximum?

Theorem (Weierstrauss Theorem)

Let the function f : Rn → R be continuous. Let C be anon-empty, closed, and bounded subset of dom(f ). Then, ∃ amaximum for f in C.

Note that this is simply a sufficient condition (but verygeneral).




Weierstrauss Theorem

Will every function have a maximum?

Theorem (Weierstrauss Theorem)

Let the function f : Rn → R be continuous. Let C be anon-empty, closed, and bounded subset of dom(f ). Then, ∃ amaximum for f in C.

Note that this is simply a sufficient condition (but verygeneral).




Optimization

Definition

A set E is non-empty if it has at least one element.

Definition

A set E is open if ∀ x ∈ E∃B(x) such that B(x) ⊂ E

Definition

A set E is closed if its complement is open.

Definition

A set E is bounded if ∃k such that k ≥ s∀s ∈ E .




Optimization

Which of these graphs has a maximum point? Whichcondition is not satisfied if not?

...................................................................................................................................................................................................................................................................................................................................................................................

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

....

• • xa b

• •

x∗

(1)

..................................................................................................................................................................................................................................................................................................................................................

.

...................................................................................................................................................................................................................................................................................................................................................................................

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

....

• • xa bd

................................................................................................................................◦

........................................................................................................................................

.........................

•

(2)

•................

•

.............................. ..........................................................................................................................................................................................................................................................................................................................................

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

....

x◦ •a b

.....................................................................................................................................................................................................................................................................................................................

◦

•

(3)

...................................................................................................................................................................................................................................................................................................................................................................................

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

....

• xa

........................................................................

........................................................................

........................................................................

....................

(4)

•

..........

.




Optimization

Which of these graphs has a maximum point? Whichcondition is not satisfied if not?

...................................................................................................................................................................................................................................................................................................................................................................................

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

....

• • xa b

• •

x∗

(1)

..................................................................................................................................................................................................................................................................................................................................................

.

...................................................................................................................................................................................................................................................................................................................................................................................

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

....

• • xa bd

................................................................................................................................◦

........................................................................................................................................

.........................

•

(2)

•................

•

.............................. ..........................................................................................................................................................................................................................................................................................................................................

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

....

x◦ •a b

.....................................................................................................................................................................................................................................................................................................................

◦

•

(3)

...................................................................................................................................................................................................................................................................................................................................................................................

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

...............

....

• xa

........................................................................

........................................................................

........................................................................

....................

(4)

•

..........

.




Optimization

1) yes

2) no (not continuous)

3) no (not closed)

4) no (not bounded)




Optimization

1) yes


3) no (not closed)

4) no (not bounded)




Optimization

1) yes


3) no (not closed)

4) no (not bounded)




Optimization

1) yes


3) no (not closed)

4) no (not bounded)




Unconstrained Optimization Problems

For each of the following functions, find the critical points andclassify them as local max, local min, saddle point, or can’t tell(from S&B Ex. 17.1, 17.2):

1 x2 − 6xy + 2y2 + 10x + 2y − 5

2 xy2 + x3y − xy

3 3x4 + 3x2y − y3

4 x2 + 6xy + y2 − 3yz + 4z2 − 10x − 5y − 21z

5 (x2 + 2y2 + 3z2)e−(x2+y2+z2)




Constrained Optimization

Now we move to constrained optimization problems, wherethe choice set becomes limited. These are much morecommon (and interesting) in economics.

Since we will generally assume utility functions are increasingfunctions, with no constraint there will be no maximum...

These can come in a variety of forms:x ≥ 0, x + y = 25, x2 − 3y + z ≤ 5, . . ..

One simple way to solve a constrained maximization problemis to solve it as if it were unconstrained and see if the answersatisfies all the constraints.

If it does, then you are done.If not, you must try something else.























How to solve an optimization problem subject to a givenconstraint?

maxx ,y

f (x , y) such that x + y = Z̄





A convenient way to solve such problems is to use theLagrangian function:

L(x , y , λ) ≡ f (x , y)− λ(Z̄ − x − y)

We re-arranged the constraint to equal zero (in the case of anequality), then add it into the problem, multiplied by λ.

λ is known as the Lagrange multiplier.

Now we have transformed the constrained problem into anunconstrained problem.





So again lets start with an easier example:

f (x) = −x2 such that x = −4 (1)




Constrained Optimization Example

Now moving into two variables:

Maximize f (x1, x2) = x1x2 such that x1 + 4x2 = 16

Begin by forming the Lagrangian: L = x1x2− λ(x1 + 4x2− 16)

Then set partial derivatives to 0:

∂L

∂x1= x2 − λ = 0

∂L

∂x2= x1 − 4λ = 0

∂L

∂λ= −(x1 + 4x2 − 16) = 0









∂L

∂x1= x2 − λ = 0

∂L

∂x2= x1 − 4λ = 0

∂L

∂λ= −(x1 + 4x2 − 16) = 0









∂L

∂x1= x2 − λ = 0

∂L

∂x2= x1 − 4λ = 0

∂L

∂λ= −(x1 + 4x2 − 16) = 0





Then solve the system of equations. 3 equations, 3 unknowns:

λ = x2 = 14x1

x1 = 4x2

(4x2) + 4x2 = 16

x2 = 2

Substituting, we then find the answer to be: x1 = 8, x2 = 2, λ = 2.

λ can have some interpretation, though we won’t talk about ittoday.





Then solve the system of equations. 3 equations, 3 unknowns:

λ = x2 = 14x1

x1 = 4x2

(4x2) + 4x2 = 16

x2 = 2

Substituting, we then find the answer to be: x1 = 8, x2 = 2, λ = 2.λ can have some interpretation, though we won’t talk about ittoday.





Note that the previous example could have been the answer tothe following set up:

Suppose a consumer is deciding how much peanut butter (P)and jelly (J) to purchase.She has $16 to spend. Peanut Butter costs $1/jar and Jellycosts $4/jar.Her utility from consuming Peanut Butter and Jelly isu(P, J) = P ∗ J.How much of each should she purchase?





If given several equality constraints, the process is similar:

maxx ,y ,z

f (x , y , z) such that h(x , y) = A, g(y , z) = B

Again form a Lagrangian, but add separate multipliers for eachconstraint:

L(x , y , z , λ1, λ2) ≡ f (x , y , z)− λ1(A− h(x , y))− λ2(B − g(y , z))

Now we solve a 5-variable optimization problem (Luckily we knowhow to solve for N variables)






maxx ,y ,z










maxx ,y ,z









Some more problems taken from S& B:

Maximize f (x1, x2) = x21x2 such that 2x2

1 + x22 = 3.

Maximize f (x , y , z) = yz + xz such that y2 + z2 = 1; xz = 3.

Maximize U(x1, x2) = kxa1x1−a

2 such that p1x1 + p2x2 = I .

Maximize f (x , y , z) = x2y2z2 such that x2 + y2 + z2 = c .





Inequality constraints become more complicated. But alsomore common:

You don’t have to spend ALL your money...You can’t hire negative people...

We will cover an example here. I encourage you to read S& BChapter 18 for more.





An inequality gives us two possibilities.

Either the inequality is binding, which results in the sameproblem that we had with and equality constraint.Or the inequality is non-binding, which means we should beable to find the answer as an unconstrained problem.

What to do? Basically we will add in extra constraints on theFOC to make sure we allow for either possibility.





Consider the problem of maximizing:

f (x , y) = xy such that g(x , y) = x2 + y2 ≤ 1 (2)

Solve for the critical values of f :

This only one occurs at the origin: (x∗, y∗) = (0, 0), which isfar away from the constraint (and is a minimum).





Consider the problem of maximizing:

f (x , y) = xy such that g(x , y) = x2 + y2 ≤ 1 (2)

Solve for the critical values of f :

This only one occurs at the origin: (x∗, y∗) = (0, 0), which isfar away from the constraint (and is a minimum).


Math Boot Camp Part II UIC Economics Departmentssc.wisc.edu/~hembre/wp-content/uploads/2015/04/MathCamp_Part… · Plug back in to get (-1,-1), (0,0), (1,1). Erik Hembre Math Boot

Documents