1 Integer Programming Approaches for Automated Planning Menkes van den Briel Department of Industrial Engineering Arizona State University [email protected] http://www.public.asu.edu/~dbva n1/
Dec 19, 2015
1
Integer Programming Approaches for Automated Planning
Menkes van den BrielDepartment of Industrial Engineering
Arizona State [email protected]
http://www.public.asu.edu/~dbvan1/
2
What is automated planning?
• Ordering problem
• Scheduling is the problem of deciding when to execute a set of actions
• NP-complete
• Selection and ordering problem
• Planning is deciding both what actions need to be done and when to execute them
• PSPACE-complete
Scheduling Planning
3
What is automated planning?
• Creating a computer program to produce a plan, a sequence of actions that will transform the world from some given initial state to a desired goal state
Initial states0 S
Goalg S
PlanP = a1, …, an
Action
Actions are state transformation functions
1 2 1 2
4
What is automated planning?
• Creating a computer program to produce a plan, a sequence of actions that will transform the world from some given initial state to a desired goal state
Initial states0 S
Goalg S
PlanP = a1, …, an
Actionsi sj
Actions are state transformation functions
5
Planning applications
• Autonomous vehicles– Mars rovers– Underwater robotics– Remote agent experiment
• Games– Bridge Baron – General game playing
• Others– Manufacturing process planning– Composition of web services– Cyber Security
West
North
East
South
62
8Q
QJ65
97
AK53
A9
West
North
East
South
62
8Q
QJ65
97
AK53
A9
6
Planning by integer programming
• Operations research (OR)
• Scheduling problems typically involve solving hard optimization problems
• Integer programming (IP), branch-and-bound
• Artificial intelligence (AI)
• Planning problems typically involve solving hard feasibility problems
• Constraint satisfaction, satisfiability (SAT), A* search
Scheduling Planning
7
Planning by integer programming
• Very little focus on integer programming approaches for planning– [Bylander, 1997]– [Bockmayr and Dimopoulos, 1998, 1999]– [Kautz and Walser, 1999]– [Vossen et al., 1999]– [Dimopoulos, 2001]– [Dimopoulos and Gerevini, 2002]
8
1. IP-based approaches simply don’t work– “Lplan [a linear programming-based heuristic for optimal
planning] was often slower than the other algorithms primarily due to the time to evaluate the linear programming heuristic”[Bylander, 1997]
2. SAT-based approaches are much faster– SAT-based planners have successfully participated in IPC1,
IPC2, IPC4, and IPC5
3. Traditionally there has been little focus on plan quality– Planning is PSPACE-complete, so finding a feasible plan is
already hard enough
Why this lack of interest?
9
1. IP-based approaches do work– Optiplan, first IP-based planner to take part in the IPC series– Ranked 2nd in four out of seven domains in IPC4 in the optimal
track for propositional domains
2. IP-based approaches can compete with SAT-based approaches– Represent planning as a set of interdependent network flow
problems– Generalize the notion of action parallelism
3. Shift in focus towards optimal planning – Applied formulations to partial satisfaction planning problems– Developed a novel framework for optimal planning– Utilized LP relaxations in deriving quality sensitive heuristics
Counter arguments
10
1. IP-based approaches do work•
2. IP-based approaches can compete with SAT-based approaches•
3. Shift in focus towards optimal planning
Contributions
– [Van den Briel, and Kambhampati. Journal of Artificial Intelligence Research, 2005]
– [Van den Briel, Vossen, and Kambhampati. ICAPS, 2005]– [Van den Briel, Vossen, and Kambhampati. Journal of Artificial Intelligence
Research, 2008]
– [Van den Briel, et al. AAAI, 2004]– [Do, Benton, van den Briel, and Kambhampati. IJCAI, 2007]– [J. Benton, van den Briel, and Kambhampati. ICAPS, 2007]– [Van den Briel, Benton, Kambhampati, and Vossen. CP, 2007]
11
1. IP approaches do work
• Optiplan– IP-based planner that extends the state change formulation by
[Vossen et al., 1999]
[van den Briel, and Kambhampati, 2005]
12
Summary of results
• International planning competition (IPC)– Bi-annual event– Provides data sets (domains) that are used as benchmarks
• IPC4– 7 competition domains– 7 participating planners in the “optimal” track
• Domains– Pipesworld
• Control the flow of oil derivatives through a pipeline network, obeying various constraints such as product compatibility and tankage restrictions
– Satellite• Collect image data with a number of satellites
– Philosophers, Optical telegraph• Involves finding deadlocks in communication protocols
13
Summary of results
0.01
0.1
1
10
100
1000
10000
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29
Phillosophers
So
luti
on
tim
e (s
ec.)
Satplan04OptiplanCPTHSPS-ATP4
0.01
0.1
1
10
100
1000
10000
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Optical telegraph
So
luti
on
tim
e (s
ec.)
Satplan04OptiplanCPTHSPS-ATP4
0.01
0.1
1
10
100
1000
10000
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49
Pipesworld(tankage)
So
luti
on
tim
e (s
ec.)
Satplan04
Optiplan
CPT
0.01
0.1
1
10
100
1000
10000
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35
Satellite
So
luti
on
tim
e (s
ec.)
Satplan04
Optiplan
CPT
14
2. IP versus SAT approaches
1. Represent planning as a set of interdependent network flow problems– One network flow problem for each state variable in the
planning domain– Nodes correspond to the values of the state variables, arcs
correspond to the value transitions
2. Generalize the notion of action parallelism – Reduces the plan length of the solution plan (and thus the size
of the formulation)
15
Logistics example1 2
P T
1
2
Truck
Drive(1,2) Drive(2,1)
Load(P,T,1)Unload(P,T,1)
Load(P,T, 1)Unload(P,T, 1)
1
2
T
Package
Load(P,T, 1)
Load(P,T, 2)
unload(P,T, 1)
unload(P,T, 2)
States are described by state variables
16
Logistics example1 2
1
2
Truck
Drive(1,2) Drive(2,1)
Load(P,T,1)Unload(P,T,1)
Load(P,T, 1)Unload(P,T, 1)
1
2
T
Package
Load(P,T, 1)
Load(P,T, 2)
unload(P,T, 1)
unload(P,T, 2)
Actions are state transformation functions
Effect
Prevail
17
One state change (1SC)
• Network representation
• Logistics example
2
1
2
1
2
1
2
1
t t
Truck
Package
h
g
f
h
g
f
h
g
f
t = 1
Prevail
Effect
Planning involves considering plans of increasing length
Plan step
18
One state change (1SC)
• Network representation
• Logistics example
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
t t t t
Drive(1,2)
Load(P,T, 1) Unload(P,T, 2)-
Load(P,T, 1) Unload(P,T, 2)Truck
Package
h
g
f
h
g
f
h
g
f
t = 1 t = 2 t = 3
Prevail
Effect
19
1SC formulation
• Constraints
– State changes (network flow), for all c C
gC ycf,g,t = 1{f I} for f Dc
hC ycg,h,t+1 = fC yc
g,h,t for f Dc , 1
t < T
fC ycf,g,T = 1 for g G
– Effect implications, for all c C, 1 t T
aA:(f,g)SC(a) xa,t = ycf,g,t for f, g Dc, f g
xa,t ycf,f,t for a A, f
PR(a)
20
Summary of results
• Experimental setup– Domains from IPC2, IPC3– Comparing 1SC formulation versus SATPLAN04 (winner of the
“optimal“ track IPC4)– 2.67GHz CPU with 1.0GB memory
• Domains– Logistics, Driverlog
• Involves driving trucks (and flying airplanes) around to deliver packages between locations
– Blocksworld• Stacking and unstacking towers of blocks
– Zenotravel• Transporting people around in planes, using different modes of
movement: fast and slow
21
Summary of results
0.01
0.1
1
10
100
1000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Zenotravel
Tim
e (s
eco
nd
s)
SAT4
1SC0.01
0.1
1
10
100
1000
10000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 1920 21222324 25262728
Logistics
So
luti
on
tim
e (s
ec.)
SAT4
1SC
0.01
0.1
1
10
100
1000
10000
1 2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829303132333435
Blocksmove
So
luti
on
tim
e (s
ec.)
SAT4
1SC
0.01
0.1
1
10
100
1000
1 2 3 4 5 6 7 8 9 1011121314151617181920
Driverlog
So
luti
on
tim
e (s
ec.)
SAT4
1SC
22
2. IP versus SAT approaches
1. Represent planning as a set of interdependent network flow problems– One network flow problem for each state variable in the
planning domain– Nodes correspond to the values of the state variables, arcs
correspond to the value transitions
2. Generalize the notion of action parallelism – Reduces the plan length of the solution plan (and thus the size
of the formulation)
23
Generalized one state change (G1SC)
• Network representation
• Example
2
1
2
1
2
1
2
1
2
1
2
1
t t t
Load(P,T, 1)Drive(1,2)
Load(P,T, 1) Unload(P,T, 2)
Unload(P,T, 2)
h
g
h
ff
g
Truck
Package
h
g
f
t = 1 t = 2
Prevail
Effect
24
Implied precedences (G1SC)
• Example
A1,A2A3
A4A1
A2
A3 A4
Implied precendence graph
25
Implied precedences (G1SC)
• Example
• Ordering (cycle elimination) constraints ensure a feasible ordering of the actions
A1,A2A3
A4
A4
A1
A1
A2
A3 A4
xA1,t + xA3,t + xA4,t 2
Implied precendence graph
26
G1SC formulation
• Constraints– State changes (network flow), for all c C
gC ycf,g,t = 1{f I} for f Dc
hC ycg,h,t+1 = fC yc
g,h,t for f Dc, 1 t T
fC ycf,g,T = 1 for g G
– Effect implications, for all c C, 1 t TaA:(f,f)SC(a) xa,t= yc
f,g,t for f, g Dc, f g,
xa,t ycf,f,t + gDc:f≠g (yc
g,f,t + ycf,g,t) for a A, f PR(a)
– Ordering (Cycle elimination) constraints
aV() xa,t |V()| – 1 for all cycles G, 1 t T
27
Branch-and-cut
Initialize LP
Node selection
LP solver
Branching
Cut generation
START
Nodes found?
Cuts found?
STOP
Feasible? Fathom
Z_lp < Z*?
no
yes
no
no
yes
yes
yes
no
Integer?
no
yes
28
Prevail
Effect
State change path (PathSC)
• Network representation
• Example
2
1
2
1
2
1
2
1
t t
Load(P,T, 1)Drive(1,2)
Unload(P,T, 2)
load(P,T, 1)unload(P,T,2)
h
g
f
h
g
Truck
Package
h
g
f f
t = 1
29
Summary of results
0
4
8
12
16
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Driverlog
Pla
n le
ng
th
SAT4
1SC
G1SC
PathSC
0
3
6
9
12
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Zenotravel
Pla
n le
ng
th
SAT4
1SC
G1SC
PathSC
0
5
10
15
20
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
Logistics
Pla
n le
ng
th
SAT4
1SC
G1SC
PathSC
0
5
10
15
20
1 2 3 4 5 6 7 8 9 10 11121314 1516 17181920212223242526272829303132333435
Blocksmove
Pla
n l
eng
th
SAT4
1SC
G1SC
PathSC
30
Summary of results
0.01
0.1
1
10
100
1000
10000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 1920 21222324 25262728
Logistics
So
luti
on
tim
e (s
ec.)
SAT4
1SC
G1SC
PathSC
[van den Briel, Vossen, and Kambhampati, 2005, 2008]
0.01
0.1
1
10
100
1000
10000
1 2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829303132333435
Blocksmove
So
luti
on
tim
e (s
ec.)
SAT4
1SC
G1SC
PathSC
0.01
0.1
1
10
100
1000
1 2 3 4 5 6 7 8 9 1011121314151617181920
Problems (Zenotravel)
Tim
e (s
eco
nd
s)
SAT4
1SC
g1SC
kSC
0.01
0.1
1
10
100
1000
10000
1 2 3 4 5 6 7 8 9 1011121314151617181920
Driverlog
So
luti
on
tim
e (s
ec.)
SAT4
1SC
G1SC
PathSC
31
3. Shift towards optimal planning
• Applied formulations to partial satisfaction planning problems
• Developed a novel framework for optimal planning• Utilized LP relaxations in deriving quality sensitive
heuristic search approaches
32
Partial satisfaction planning
• PLAN LENGTH is PSPACE-complete– [Bylander, 1994]
• PSP UTILITY COST is PSPACE-complete– [Van den Briel, et al., 2004]
PLAN EXISTENCE
PLAN LENGTH
PSP GOAL LENGTH
PSP GOAL
PLAN COST PSP UTILITY
PSP NET BENEFIT
Total Satisfaction
Problems
Partial Satisfaction
Problems
PSP UTILITY COST
33
Framework for optimal planning
• For step-based IP formulations optimality is restricted to the length of the plan
Plan step
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
t t t t
Drive(1,2)
Load(P,T, 1) Unload(P,T, 2)-
Load(P,T, 1) Unload(P,T, 2)Truck
Package
t = 1 t = 2 t = 3
34
Framework for optimal planning
1 2
P T
1
2
Truck
Drive(1,2) Drive(2,1)
Load(P,T,1)Unload(P,T,1)
Load(P,T, 1)Unload(P,T, 1)
1
2
T
Package
Load(P,T, 1)
Load(P,T, 2)
unload(P,T, 1)
unload(P,T, 2)
35
Action selection formulation
• Variables– xa Z+, for a A; xa is equal to the number of times action a is
executed
– yv(c,a) Z+, for v V, a A, a –(c); yv(c,a) is equal to the number of times transition v(c,a) is executed
• Objective function– MIN aA caxa
• Constraints av+(e) yv(c,a) – a v–(e) yv(c,a)
av+(e) yv(c,a) = xa
1 if c c0,v, c g–1 if c = c0,v, c g0 otherwise
No time indicesNo upper bounds
36
Concurrent automata
• Given a set of state variables V = {v1, …, vn}
• For each v V we define a deterministic automaton
Gv = (Dv, Av, v, v, c0,v, gv)
– Dv is a finite set of states corresponding to the domain of state variable v
– Av is a finite set of actions associated with the transitions in Gv
v : Dv A Dv is the transition function
v : Dv 2A is the active action function
– c0,v S is the initial state of state variable v
– gv S is a set of goal states of state variable v
37
Parallel composition
• The parallel composition of the two automata G1 and G2 is the automaton
G1||G2 := (D1D2, A1A2, 1||2, 1||2, (c0,1, c0,2), g1g2)
1||2((c1,c2),a) :=
1||2(c1,c2) := [1(c1)2(c2)] [1(c1)\A2][2(c2)\A1]
(1(c1,a), 2(c2,a) if a 1(c1)2(c2)(1(c1,a), c2) if a 1(c1)\A2
(c1,2(c2,a)) if a 2(c2)\A1 undefined otherwise
38
Logistics example
1 2
P T
1
2
Truck
Drive(1,2) Drive(2,1)
Load(P,T,1)Unload(P,T,1)
Load(P,T, 1)Unload(P,T, 1)
1
2
T
Package
Load(P,T, 1)
Load(P,T, 2)
unload(P,T, 1)
unload(P,T, 2)
39
Simple logistics example
1,1
1,T
2,T
2,2
1,2
2,1
Truck || Package
Drive(1,2)
Drive(2, 1)
Load(P, T, 1)
Load(P, T, 2)
Unload(P, T, 1)
Unload(P, T, 2)
Drive(1, 2)
Drive(2, 1)
Drive(1, 2)Drive(2, 1)
1 2
P T
40
Summary of results
Highlighted values equal optimal solution
Problem h+ LP LP+ Optimalzenotravel1 1 1 1 1zenotravel2 4 3.0* 6 6zenotravel3 5 4.0* 6 6zenotravel4 6 5.0* 8 8zenotravel5 11 8.0* 11 11zenotravel6 11 8.0* 11 11zenotravel13 23 18.0* 24 -zenotravel19 62 46.0* 66.2* -zenotravel20 - 50.0* 68.3* -tpp1 4 3.0* 5 5tpp2 7 6.0* 8 8tpp3 10 9.0* 11 11tpp4 13 12.0* 14 14tpp5 17 15.0* 19 19tpp6 21 21.0* 25 25tpp28 - 150.0* - -tpp29 - - - -tpp30 - 174.0* - -bw-sussman 5 4 4 6bw-12step 4 4 4 12bw-large-a 12 12 12 12bw-large-b 16 16 16 18
Problem h+ LP LP+ Optimallog4-0 19 16.0* 20 20log4-1 17 14.0* 19 19log4-2 13 10.0* 15 15log5-1 15 12.0* 17 17log5-2 8 6.0* 8 8log6-1 13 10.0* 14 14log6-9 21 18.0* 24 24log12-0 39 32.0* 42 42log15-1 63 54.0* 67 -freecell2-1 9 9 9 9freecell2-2 8 8 8 8freecell2-3 8 8 8 8freecell2-4 8 8 8 8freecell2-5 9 9 9 9freecell3-5 13 12 12 -freecell13-3 - 55 55 -freecell13-4 - 54 54 -freecell13-5 - 52 52 -driverlog1 6 3.0* 7 7driverlog2 14 12.0* 19 19driverlog3 11 8.0* 11 12driverlog4 12 11.0* 16 16driverlog6 10 8.0* 11 11driverlog7 12 11.0* 13 13driverlog13 21 15.0* 24 -driverlog19 89 60.0* 96.6* -driverlog20 84 60.0* 89.5* -
41
Summary of results
0.01
0.1
1
10
100
1000h+
LP
LP+
Logistics Freecell Driverlog Zenotravel TPP Blocks
42
Utilize LP in heuristic search
0
100000
200000
300000
400000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Zenotravel
Net
ben
efit
H_LP
H_LP + RP
H_LP + Cost RP
UB
0
300000
600000
900000
1200000
1 3 5 7 9 11 13 15 17 19
Satellite
Net
ben
efit
H_LP
H_LP + RP
H_LP + Cost RP
UB
0
100000
200000
300000
400000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Rovers
Net
ben
efit
H_LP
H_LP + RP
H_LP + Cost RP
UB
[Benton, van den Briel, and Kambhampati, 2007]
BBOP-LP planner
43
Summary
• IP-based approaches do work– Optiplan, first IP-based planner to take part in the IPC series– Ranked 2nd in four out of seven domains in IPC4 in the optimal
track for propositional domains
• IP-based approaches can compete with SAT-based approaches– Represent planning as a set of interdependent network flow
problems– Generalize the notion of action parallelism
• Shift in focus towards optimal planning – Applied formulations to partial satisfaction planning problems– Developed a novel framework for optimal planning– Utilized LP relaxations in deriving quality sensitive heuristics
44
Publications status
• Journal– [M.H.L. van den Briel, and S. Kambhampati. Optiplan: Unifying IP-based and
graph-based planning. Journal of Artificial Intelligence Research, 24:623-635, 2005]– [M.H.L van den Briel, T. Vossen, and S. Kambhampati. Loosely coupled
formulation for automated planning: An integer programming perspective. Journal of Artificial Intelligence Research, 31:217-257, 2008]
– [(In progress) M.H.L van den Briel, T. Vossen, S. Kambhampati and J. Fowler. Optimal automated planning]
• Conference– [M.H.L. van den Briel, R. Sanchez, M.B. Do, and S. Kambhampati. Effective
approaches for partial satisfaction (oversubscription) planning. In Proceedings of AAAI, pages 562-569, 2004]
– [M.H.L. van den Briel, T. Vossen, and S. Kambhampati. Reviving integer programming approaches for AI planning: A branch-and-cut framework. In Proceedings of ICAPS, pages 161-170, 2005]
– [M.B. Do, J. Benton, M.H.L. van den Briel, and S. Kambhampati. Planning with goal utility dependencies. In Proceedings of IJCAI, pages 1872-1878, 2007]
– [J. Benton, M.H.L. van den Briel, and S. Kambhampati. A hybrid linear programming and relaxed plan heuristic for partial satisfaction planning problems. In Proceedings of ICAPS, pages 24-41, 2007]
– [M.H.L. van den Briel, J. Benton, S. Kambhampati, and T. Vossen. An LP-based heuristic for optimal planning. In Proceedings of CP, pages 651-665, 2007]
Cited by 31
Cited by 15
Cited by 3
Cited by 4
Cited by 6
Cited by 3
45
Publications status
• Workshop and posters– [M.H.L. van den Briel, R. Sanchez, and S. Kambhampati. Over-Subscription in
Planning: a Partial Satisfaction Problem. In Proceedings of ICAPS Workshop on Integrating Planning into Scheduling, 2005]
– [M.H.L. van den Briel,. Kambhampati, and T. Vossen. Planning with numerical state variables through mixed integer programming. In Proceedings of ICAPS Poster Session, pages 5-8, 2005]
– [M.H.L. van den Briel,. Kambhampati, and T. Vossen. Planning with preferences and trajectory constraints by integer programming. In Proceedings of ICAPS Workshop on Preferences and Soft Constraints in Planning, pages 19-22, 2006]
– [J. Benton, M.H.L. van den Briel,. Kambhampati. Finding admissible bounds for oversubscription planning problems. In Proceedings of ICAPS Workshop on Heuristics for Domain-Independent Planning: Progress, Ideas, Limitations, Challenges, 2007]
– [M.H.L. van den Briel,. Kambhampati, and T. Vossen. Fluent merging: A general technique to improve reachability heuristics and factored planning. In Proceedings of ICAPS Workshop on Heuristics for Domain-Independent Planning: Progress, Ideas, Limitations, Challenges, 2007]
Cited by 5
Cited by 1
Cited by 1
Citation count by Google Scholar