Chapter 10 Dynamic Programming
Chapter 10 Dynamic Programming
Jan 14, 2016
Chapter 10 Dynamic Programming. Agenda for This Week. Dynamic Programming Definition Recursive Nature of Computations in DP Forward and Backward Recursion Basic Elements of DP Applications of DP. Dynamic Programming. - PowerPoint PPT Presentation
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Chapter 10
Dynamic Programming
2
Agenda for This Week
• Dynamic Programming– Definition– Recursive Nature of Computations in DP– Forward and Backward Recursion– Basic Elements of DP– Applications of DP
3
Dynamic Programming
• An approach to problem solving that decomposes a large problem into a number of smaller problems for finding optimal solution
• Used for problems that are in multistage in nature
• No standard mathematical formulation
4
Stagecoach Problem
• Used to illustrate the features of DP• Concerns a mythical fortune seeker to go to west to
join the gold rush in California during mid-19th century
• A prudent man who was quite concerned about his safety
• Could not go through some territories • Starting and final destination points are fixed• Had considerable choice as to which states to travel
5
Cost Estimates
5 6 7
2 10 6 -
3 9 8 6
4 6 4 7
2 3 4
1 4 3 5
8 9
5 5 9
6 4 8
7 7 5
3
2
7
6
5
9
810
4
1
4
3
5
10
6
8
9
6
7
4
6
5
49
5
87
6
4
10
8 6
9 4
6
Solving Problem• Select routes that minimizes total cost
• One possible approach is to use trail and error method
• DP starts with a small portion of the original problem and finds the optimal solution
• Optimizes the nest stage using the solution from preceding one
• Continues until all stages are solved
• First decompose it into stages as delineated by the vertical dashed lines
3
2
7
6
5
9
810
4
1
4
3
5
10
6
8
9
6
7
4
6
5
49
5
87
6
4
7
Recursive Nature of Computations in DP
• Start with the smaller problem where he has completed his journey and has only one to go
• Computations are done recursively
• Solution of one subproblem is used as an input for next subproblem
3
2
7
6
5
9
810
4
1
4
3
5
10
6
8
9
6
7
4
6
5
49
5
87
6
4
8
DP Concept• Stages are formed by decomposing the original problem
into a number of subproblems
• Shortest path context– Input— any node or state
– Decision Problem— which arc?
– Decision criteria—shortest distance
– Output– any node or state
Decision Problem Decision Criteria
Input Output
9
DP Notation• Xn=input to stage n (output from stage n+1)
• dn=decision variable at stage n (state variable)
• Stage transformation function• Return function—return of a stage
dNXN
Stage NXN-1 Xn
Xn-1 X0X1
d1dn
Stage n Stage 1
rN(xN,dN) rn(xn,dn) r1(x1,d1)
10
Stage one• First stage begins with nodes that are exactly one arc away from
final destination (CA)
• Nodes 8 and 9 are the input nodes or states for stage 1
• Nodes 10 is the output nodes for stage 1
3
2
7
6
5
9
810
4
1
4
3
5
10
6
8
9
6
7
4
6
5
49
5
87
6
4
11
Decomposing the Problem-Stage two• Second stage begins with all nodes that are exactly two arcs away
from destination and ends with all nodes that are exactly one arc away
• Nodes 5, 6, and 7 are the input nodes for stage 2
• Nodes 8 and 9 are the output nodes for stage 2
3
2
7
6
5
9
810
4
1
4
3
5
10
6
8
9
6
7
4
6
5
49
5
87
6
4
12
Decomposing the Problem-Stage three• Third stage begins with all nodes that are exactly three arcs away
from destination and ends with all nodes that are exactly two arcs away
• Nodes 2, 3, and 4 are the input nodes for stage 3
• Nodes 5, 6 and 7 are the output nodes for stage 3
3
2
7
6
5
9
810
4
1
4
3
5
10
6
8
9
6
7
4
6
5
49
5
87
6
4
13
Decomposing the Problem-Stage Four• Forth stage begins with all nodes that are exactly four arcs away
from destination and ends with all nodes that are exactly three arcs away
• Nodes 1 the input node for stage 4
• Nodes 2, 3 and 4 are the output nodes for stage 3
3
2
7
6
5
9
810
4
1
4
3
5
10
6
8
9
6
7
4
6
5
49
5
87
6
4
Stage 1Stage 4
14
Solving the Problem-Stage one
10
9
86
4
6
4
Input Arc Output Shortest to 10
89
8-109-10
64
1010
15
Solving by DP Notation-Stage 1• X1=input to stage 1 (output from
stage 2)
• d1=decision variable at stage 1 (state variable)
• Stage transformation function
• Return function—return of a stage
f1(x1)= r1(x1,d1)d1
x1
89
10
64
d1*
1010
dnXn
Stage nXn-1
rn(xn,dn)
10
9
86
4
6
4
16
Solving the Problem- Stage two
10
9
8
4
6
4
Input Arc OutputShortest
6
5
6
7
567
5-86-87-9
11109
5
9
4
8
7 5
889
11
10
9Min [(5+6),(9+4)]=11Min [(4+6),(8+4)]=10Min [(7+6),(5+4)]= 9
17
Solving by DP Notation-Stage 2r2(x2,d2)+ f1(x1)d2
x2
567
8 9
111013
13129
d2*
11109
f2(x2)
889
10
9
8
4
6
4
6
5
6
7
5
9
4
8
7 5
11
10
9
18
Solving the Problem- Stage Three
10
9
8
4
6
4
Input Arc OutputShortest
6
5
6
7
234
2-63-74-6
161514
5
9
4
8
7 5
676 11
10
9
2
3
4
10
6
98
6
6 4
7
16
15
14
Min [(10+11),(6+10)]=16Min [(9+11),(8+10), (6+9)]=15Min [(11+6),(10+4),(7+9)]= 14
19
Solving by DP Notation-Stage 3r3(x3,d3)+ f2(x2)d3
x3
234
5 6212017
161814
d3*
161514
f3(x3)
676
7-
1516
10
9
8
4
6
4
6
5
6
7
5
9
4
8
7 5
11
10
9
2
3
4
10
6
98
6
6 4
7
16
15
14
20
Solving the Problem- Stage Four
10
9
8
4
6
4
Input Arc OutputShortest
6
5
6
7
1 18
5
9
4
8
7 5
3
11
10
9
2
3
4
10
6
98
6
6 4
7
16
15
14
1
4
3
5
1-3
18
Min [(4+16),(3+15), (5+14)]=18
21
Solving by DP Notation-Stage 4r4(x4,d4)+ f3(x3)d4
x4
1
2 3
20 18
d3*
18
f4(x4)
3
4
19
10
9
8
4
6
4
6
5
6
7
5
9
4
8
7 5
11
10
9
2
3
4
10
6
98
6
6 4
7
16
15
14
1
4
3
5
18
22
Best Decision at Each stageStage
4321
Arc1-33-77-9
9-10
3
2
7
6
5
9
810
4
1
4
3
5
10
6
8
9
6
7
4
6
5
49
5
87
6
4
23
Dynamic EOQ Models • Dynamic EOQ models are different,
because– Inventory level is reviewed periodically over a
number finite periods– Demand per period, though deterministic, is
dynamic (MRP situation)
• No shortage is allowed and setup cost is incurred each time a production lot is started
24
A Production and Inventory Control Problem
• Making decision on a production quantity for each of periods so that the demand can be met at a minimum cost– N=no. of periods (stages)
– Dn=demand during stage n
– Xn= amount of inventory on hand at beginning of stage n
– dn= production quantity for stage n
– Pn=production capacity in stage n
– Wn=storage capacity at the end of stage
– Cn=production cost per unit in stage
– Hn=holding cost per unit of ending inventory for stage n
25
Example Capacity Cost/unit
Month Demand Production Storage Production Holding
123
233
323
232
175150200
303040
Beginning inventory for month 1is 1 units
26
DP Formulation
Month 1 Month 2 Month 3
D3=
2
P3=
3W
3=2
d3=
?
P2=
2D
2=3
W2=
3d2
=?
D1=
3
P1=
3
W1=
2d1
=?
X3=1 X1X2 X0
r3(x3,d3) r2(x2,d2) r1(x1,d1)
Note: Ending Inventory= Beginning Inventory
+Production-Demand
27
Knapsack Problem
• N items can be put into a knapsack• Each time has a certain weight and a value• Determine how many units of each item to place
in the knapsack to maximize the total value• Constraint is placed on the maximum weight• Similar to a manger who must make a biweekly
selection of jobs from each of four categories to process during a 2-week period
28
Job Data for the Manufacturing Operation • Select certain jobs
during the next 2 weeks such that all the jobs selected can be processed within 10 working days and the total value of the jobs is maximized
Job Category
No. of Jobs to be processed
Estimated Completion time per job
Value rating per job
1 4 1 2
2 3 3 8
3 2 4 11
4 2 7 20
29
Formulation• At stage 1, we must decide how many jobs from
category 1 to process• At stage 2, we must decide how many jobs from
category 2 to process, etc.• Value rating is a subjective score (scale 1 to 20)
• xn=No. of days of processing time remaining at the beginning of stage n (state variable)
• dn=No. of jobs processed from category n (decision variable at stage n)
Related Documents
