Dynamic ProgrammingExamples
Based on:1. Michael A. Trick: A Tutorial on Dynamic Programming
http://mat.gsia.cmu.edu/classes/dynamic/dynamic.html
2. M. A. Rosenman: Tutorial - Dynamic Programming Formulation
http://people.arch.usyd.edu.au/~mike/DynamicProg/DPTutorial.95.html
Motivation Problems
• Capital Budgeting Problem• Minimum cost from Sydney to Perth• Economic Feasibility Study• Travelling Salesman Problem
Capital Budgeting Problem
• A corporation has $5 million to allocate to its three plants for possible expansion.
• Each plant has submitted a number of proposals specifying (i) the cost of the expansion c and (ii) the total revenue expected r.
• Each plant is only permitted to realize one of its proposals.• The goal is to maximize the firm’s revenues resulting from the
allocation of the $5 million.Investment proposals
Plant 1 Plant 2 Plant 3
Proposal c1 r1 c1 r1 c1 r1
1 0 0 0 0 0 0
2 1 5 2 8 1 4
3 2 6 3 9 - -
4 - - 4 12 - -
Minimum cost from Sydney to Perth
• Travelling from home in Sydney to a hotel in Perth – choose from 3 hotels.
• Three stopovers on the way – a number of choices of towns for each stop.
• Each trip has a different distance resulting in a different cost (petrol).
• Hotels have different costs.• The goal is to select a route to and a hotel in Perth so that
the overall cost of the trip is minimized.
Economic Feasibility Study
• We are asked for an advice how to best utilize a large urban area in an expanding town.
• Envisaged is a mixed project of housing, retil, office and hotel areas.
• Rental income is a function of the floor areas allocated to each activity.
• The total floor area is limited to 7 units.• The goal is to find the mix of development which will
maximize the return.• Additional constraint determines that the project must
include at least 1 unit of offices.
Traveling Salesman Problem
• Given a number of cities and the costs of travelling from any city to any other city, what is the cheapest round-trip route that visits each city exactly once and then returns to the starting city?
• Example
Random tour Optimized tour
New York
Miami Dallas Chicago
1. New York
- 1334 1559 809
2. Miami 1334 - 1343 13973. Dallas 1559 1343 - 9214. Chicago
809 1397 921 -
How to Solve it?
No problem, just try all possibilities and choose the best.
It is not that easy for problem instances of a realistic size. Try it out!
DP: Common characteristics
• The problem can be divided into stages with a decision required at each stage.
• Each stage has a number of states associated with it.• The decision at one stage transforms one state into a state in
the next stage.• Principal of optimality: Given the current state, the
optimal decision for each of the remaining states does not depend on the previous states or decisions.
• There exists a recursive relationship that identifies the optimal decision for stage j, given that stage j+1 has already been solved.
• The final stage must be solvable by itself.
• It is an art to determine stages and states.
DP: Capital Budgeting Problem
• Stages – money allocated to a single plant• States variables
x1 = {0,1,2,3,4,5} … amount of money spent on plant 1x2 = {0,1,2,3,4,5} … amount of money spent on plant 1 and
2x3 = {5} … amount of money spent on palnts 1, 2
and 3We want x3 to be 5.
• Ordering on the stages – first allocate to plant 1, then plant 2, then plant 3.
DP: Capital Budgeting Problem
• Stage 1 computations
If the available Then the optimal And the revenue
capital x1 is proposal is for stage 1 is0 1 01 2 52 3 63 3 64 3 6
5 3 6
DP: Capital Budgeting Problem • Stage 2 computations
• Example: Determine the best allocation for state x2=4.1. Proposal 1 gives revenue of 0, leaves 4 for stage 1, which returns 6. Total=6.2. Proposal 2 gives revenue of 8, leaves 2 for stage 1, which returns 6. Total=14.3. Proposal 3 gives revenue of 9, leaves 1 for stage 1, which returns 5. Total=14.4. Proposal 4 gives revenue of 12, leaves 0 for stage 1, which returns 0. Total=12.
If the available Then the optimal And the revenuecapital x2 is proposal is for stages 1 and 2
is0 1 01 1 52 2 83 2 134 2 or 3 145 4 17
DP: Capital Budgeting Problem
• Stage 3 computations, we are interested in x3=51. Proposal 1 gives revenue 0, leaves 5. Previous stages give
17. Total=17.2. Proposal 2 gives revenue 4, leaves 4. Previous stages give
14. Total=18.
• The optimal solution, that gives a revenue 18, is to implement
- proposal 2 at plant 3, - proposal 2 or 3 at plant 2, and- proposal 3 or 2 at plant 1.
DP: Minimum cost from Sydney to Perth
• Stages – stopovers• States variables – cost of the trip• Decision variables - towns with its hotels
DP: Minimum cost from Sydney to Perth
DP: Minimum cost from Sydney to Perth
DP: Minimum cost from Sydney to Perth
DP: Economic Feasibility Study
• Stage variable – the development type (1, 2, 3, 4)• State variable – the amount of floor area allocated to all
dev. types• Decision variables – the amount of floor area allocated to
the nth dev. type
DP: Economic Feasibility Study
Expected annual profit
DP: Economic Feasibility Study
DP: Economic Feasibility Study
DP: Economic Feasibility Study
How we get it?
DP: Economic Feasibility Study(Constrained Solution)
DP: Economic Feasibility Study(Constrained Solution)
DP: Economic Feasibility Study(Constrained Solution)
Solution?
DP: Economic Feasibility Study(Constrained Solution)
DP: TSP
• Stages – stage t represents t cities visited .• Decisions – where to go next.• States – we need to know where we have already gone, so states
are represented by a pair (i, S), where S is the set of t cities already visited and i is the last city visited.
• Start with stage 3 calculations:1. f3(2, {2,3,4}) = 13342. f3(3, {2,3,4}) = 15593. f3(4, {2,3,4}) = 809
• For other stages, the recursion isft(i,S)=minj1 and jS{cij + ft+1(j, Sj)}
• Problems? Number of states.
• N=100 generates more than 51030 states at stage 50
New York
Miami Dallas Chicago
1. New Y.
- 1334 1559 809
2. Miami1334 - 1343 13973. Dallas1559 1343 - 9214. Chicago
809 1397 921 -
Need for Something Better!
• Curse of dimmensionality– Not enough memory, not enough computational power
to solve hard problems.
• Softcomputing techniques - Evolutionary algortihms, Artificial neural networks, Fuzzy.– The guiding principle of soft computing is: Exploit the
tolerance for imprecision, uncertainty, partial truth, and approximation to achieve tractability, robustness and low solution cost.