CS 236501Introduction to AI
CS 236501Introduction to AI
Tutorial 5Automatic Generation of Heuristic Functions
Tutorial 5Automatic Generation of Heuristic Functions
Problem DefinitionProblem Definition
• Most search algorithms need a heuristic function to guide them in the search space
• Writing a good heuristic function is hard:– Extensive knowledge about the problem is needed– The distance to the goal may be hard to estimate
(especially when variables have nominal values)– It is hard to measure how informative a heuristic
function is– We mostly want admissible heuristics
• We sometimes want a heuristic function for an unknown problem, or a problem that changes with time
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 2
Proposed SolutionProposed Solution
• Automatically generate a heuristic function– Should be a problem and domain independent method– Can be state specific– Should produce informative, preferably admissible
heuristics– Should be fast in practice and preferably have low
theoretical complexity
• Such heuristics generally use a simplified version of the original problem, the (optimal) solution to which serves as the heuristic value for the original states
• Most ideas come from the field of automated planning
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 3
A useful formal definition of search problems
A useful formal definition of search problems
• A search problem P=<S,A,I,G> can be defined in the following terms:– S – The search space. S can be defined as a set of
variables V={v1,v2,…,vn} with value domains D={d1,d2,…,dn} such that each variable vi has an associated domain di, and a state s∊S is an element of the space defined by d1 X d2 X … X dn (alternatively this can be viewed as a complete assignment for the variables, i.e. s∊S={<vi,di>}i=1..n).
– I – The initial state. I is an element of S.– G – The goal state. G is a partial element of S, i.e.
G⊆{<vi,di>}i=1..n. When a state s∊S contains G, we say that s is in a goal state.
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 4
A useful formal definition of search problems
A useful formal definition of search problems
– A – The set of allowed actions (or operators). This is a function from S to S.An action <pre⊆{<vi,di>}i=1..n,eff⊆{<vi,di>}i=1..n>∊A takes as input a state s∊S, and if it meets the preconditions pre (i.e. the variables that pre specifies have the values that pre specifies), then the effect eff takes place (i.e. the variables that eff specifies are set to the values that eff specifies for them). Variables not specified in pre are “don’t cares” and the values of variables not in eff are copied from the input state to the output state.
– The set A*(I) is the feasible state space.– Note: Each action a∊A is a symbolic manipulation on
the values of the variables of a state s∊S.
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 5
RelaxationRelaxation
• We will consider problem relaxation– A relaxation P’ of problem P is achieved by deleting
some constraints present in P– Other methods include: Over constraining, Analogy,
and Probabilistic models
• Example: P = 8-Puzzle– Goal: Order tiles– Action: Swap(x, y) when
tile x is H/V-adjacent totile y, and either x is emptyor y is empty
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 6
1 2 3
4 5 6
7 8
8-Puzzle – The original problem8-Puzzle – The original problem
• The optimal solution of the original (non-relaxed) problem is:
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 7
h*=9
2 4
1 5 3
7 8 6
8-Puzzle – Relaxation I8-Puzzle – Relaxation I
• Tiles can move to any location (whether occupied by another tile or not)– This is the number of misplaced tiles heuristic
Apr 21, 2023
Intro. to AI – Tutorial 7 – By Haggai Toledano 8
2
1 5 3
7 8 6
h=5
4
8-Puzzle – Relaxation II8-Puzzle – Relaxation II
• Tiles can move to any adjacent location (whether occupied by another tile or not)– Summing over all tiles, this is Manhattan Distance
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 9
h=7
2
1 5 3
7 8 6
4
8-Puzzle – Relaxation III8-Puzzle – Relaxation III
• Tiles can move to the empty location (even if it is not adjacent to them)– Optimal solution: Sort (place a misplaced target tile in
the empty place; prefer the target of that place)
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 10
h=5
2
1 5 3
7 8 6
4
Relaxations – UsageRelaxations – Usage
• Receive a problem P=<S,A,I,G>• Create P’=<S’,A’,I’,G’>, the relaxed version of P• Run your favorite heuristic algorithm on P
– For each s∊S, starting with I, get the successors succs
of s by applying all applicable actions from A.
– For each t∊succs compute the optimal solution of P’ from t’ to G’. The solution value (length, cost etc.) will be denoted by h(t).
– Use h(succs) as the heuristic function for the search algorithm.
• Since P’ is only a symbolic manipulation of P, it is problem and domain independent
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 11
h+ Relaxationh+ Relaxation
• A general method for creating a certain type of relaxations:– State variables accumulate values instead of merely
changing their value – never forget achieved values
• Simplifies the original problem – less constraints: – Allows “jumps” between previously unlinked states– In effect: Adds edges to the states transition graph– Preserves preconditions– Computation: Uninformed, complete search, e.g. BFS
• Monotonic: The original actions are still applicable to the states they were applicable to beforehand
• Admissible
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 12
8-Puzzle – h+8-Puzzle – h+
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 13
X2 4
1 5 3
7 8 6
2X2X 4
1 5 3
7 8 6
24X2X 4X
1 5 3
7 8 6
24X12X 4X
1X 5 3
7 8 6
24X12X 34X
1X 5 3X
7 8 6
24X124X
34X
1X 5 3X
7 8 6
0
3
1 2
54
4
8-Puzzle – h+8-Puzzle – h+
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 14
24X124X
34X
1X 5 36X
7 8 6X
24X124X
34X
14X 5 36X
7 8 6X
24X124X
34X
14X 5 36X
7 8 6X
h+ = 7h* = 9
6 7
4
h+ Relaxation – Other problemsh+ Relaxation – Other problems
• Lightz Out
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 15
..
On
Off
On, Off
..On
OffP
P+
Goal!
h+ Relaxation – Other problemsh+ Relaxation – Other problems
• City Map– I = <Boston>– succ(I) = {<Quebec>, <New York>}– succ+(I)= {<Boston, Quebec>,<Boston, New York>}– succ+(<Boston, New York>) =
<Boston, New York, Quebec>)
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 16
Boston
QuebecNew York
Boston
Quebec
New York
h+ Relaxation – Other problemsh+ Relaxation – Other problems
• Missionaries and Cannibals– State: <ML,CL,MR,CR,B>∊NxNxNxNx{right,left}, where
N={0,1,2,3}– I = <3,3,0,0,L> ; G = <0,0,3,3>– A = {CCR = [move 2 cannibals from L to R] =
<pre={CL≥2,B=left,MR≥CR+2},
eff={CL=CL-2,CR=CR+2,B=right}>, MMR, CML, … }
– CCR(I=<3,3,0,0,L>) = <3,1,0,2,R>
– CCR+ = <pre={CL≥2,B=left,MR≥CR+2},
eff={CL=CL U CL-2,CR=CR U CR+2,B=B U right}>
– CCR+(I+=<{3},{3},{0},{0},{L}>)= <{3},{1,3},{0},{0,2},{L,R}>
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 17
h+ Relaxation: Complexityh+ Relaxation: Complexity
• Although the +-relaxed problem seems simpler to solve, its complexity might still be too high
• Proof (Bylander ’94):– Propositional planning is a type of search problem– PLANMIN(k) is the decision problem of determining
whether an instance of a propositional planning problem is satisfiable in k or less actions
– PLANMIN(k)+ is PLANMIN(k) restricted to positive
postconditions (can only add facts, can’t delete them)– PLANMIN(k) is PSPACE complete
– PLANMIN(k)+ is NP complete (reduction from 3SAT)
• Solution: Approximate h+
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 18
h+ Approximation I: hmaxh+ Approximation I: hmax
• hmax – Find the longest path to any goal value for the relaxed problem– Build a layered planning graph:
• Start from the initial state• At each layer apply all applicable actions
– The first layer to hold all of the goal variables in their goal values is the approximated solution length
• Properties:– Not very informative (too optimistic; no mutex
detection)– Admissible
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 19
8-Puzzle – hmax8-Puzzle – hmax
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 20
X2 4
1 5 3
7 8 6
245X
2X 4X
1 5X 3
7 8 6
245X
1245X
2345X
1X123458X
3X
7 8X 6
123458X
1245X
12345X
1234578X
123458X
1234568X
7X12345678X
6X
123458X
1245X
12345X
1234578X
123458X
1234568X
7X12345678X
6X
0 1 2
3
h+ = 7h* = 9
hmax = 3
hmax
= max(2(1), 1(2), 2(3), 3(4), 0(5),
3(6), 0(7), 0(8)) = 3
h+ Approximation II: haddh+ Approximation II: hadd
• hadd – Sum over the costs to achieve all goal variables values
• Compute just like hmax but sum the costs instead of finding their maximum
• Assumes total independence of goal variables• Properties:
– Informative in problems where the goal variables have little interaction with one another
– Not admissible – often over estimates
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 21
8-Puzzle – hadd8-Puzzle – hadd
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 22
X2 4
1 5 3
7 8 6
245X
2X 4X
1 5X 3
7 8 6
0 1 2
3
h+ = 7h* = 9
hmax = 3hadd = 11
hadd
= 2(1)+1(2)+2(3)+3(4)+0(5)+3(6)+0(7)+0(8) = 11
245X
1245X
2345X
1X123458X
3X
7 8X 6
123458X
1245X
12345X
1234578X
123458X
1234568X
7X12345678X
6X
123458X
1245X
12345X
1234578X
123458X
1234568X
7X12345678X
6X
h+ Approximation III: hFFh+ Approximation III: hFF
• hFF – Fast Forward: The length of a non-optimal solution to the relaxed problem:– Build the planning graph– Backchain some solution (i.e. necessary actions) from
the last (goal) layer to the first (initial state):• For each goal in layer i, set its preconditions as goals for
layer i-1• Select actions that were applied on layer i-1 and contain
the goals from layer i as their combined postconditions• Repeat until reaching the initial graph layer
– The solution’s length (i.e. the sum of the number of uniquely selected actions for each layer) is the heuristic value for the initial state
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 23
h+ Approximation III: hFFh+ Approximation III: hFF
• Properties:– Takes into account positive interactions– Not admissible– Rarely over estimates
– Uses the fact that while PLANMIN(k)+ is NPC, PLANSAT+
(relaxed satisfiablity) is in P (so if the goal cannot be reached – h+ is unsolvable; if the goal is reached – h+ is solvable and a witness can be constructed)
– Note that solving for the minimal number of actions in each layer is NPC (reduction from SC)
• The original FF algorithm uses enforced hill-climbing (use BFS to escape local minima)
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 24
h+ and its approximationsh+ and its approximations
• Problem:– Variables: x∊{x1,x2,x3}, y∊{y1,y2,y3} , z∊{z1,z2}
– Actions (<prepost>): a1=<x1x2>, a2=<x2x3,y2>, a3=<x2x3>, a4=<x2y2>, a5=<y2y3>, a6=<z1z2>
– Initial state: {x=x1, y=y1, z=z1}
– Goal state: {x=x3, y=y3, z=z2}
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 25
x1
y1
z1
x1x2 x1,x2
y1
z1,z2
x1,x2,x3
y1,y2
z1,z2
y2y3
x1,x2,x3
y1,y2,y3
z1,z2z1z2
x2x3
x2y2
hmax=max{2(x),3(y),1(z)}=3
hadd=sum{2(x),3(y),1(z)}=6
hFF=2(a1,a6)+2(a3,a4)+1(a5)=5
h+=1(a1)+1(a2)+1(a5)+1(a6)=4
x2x3,y2
= h*
8-Puzzle – hFF8-Puzzle – hFF
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 26
X2 4
1 5 3
7 8 6
245X
2X 4X
1 5X 3
7 8 6
0 1 2
3
h+ = 7h* = 9
hmax = 3hadd = 11hFF = 9
hFF = by a certain choice
of actions = 9
245X
1245X
2345X
1X123458X
3X
7 8X 6
123458X
1245X
12345X
1234578X
123458X
1234568X
7X12345678X
6X
123458X
1245X
12345X
1234578X
123458X
1234568X
7X12345678X
6X
8-Puzzle – hFF backchaining8-Puzzle – hFF backchaining
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 27
1(1)
2(2)
3(3)
4(4)
5(5)
6(6)
7(7)
8(8)
1(1)
2(2)
3(3)
4(1)
X(4)
5(5)
6(9)
X(6)
7(7)
8(8)
1(4)
X(1)
2(2)
3(6)
X(3)
4(2)
X(1)
X(5)
5(5)
6(9)
7(7)
8(8)
1(4)
2(1)
X(2)
4(3)
5(5)
6(9)
3(6)
7(7)
8(8)
Unique non-noop actions: 3+4+2 hFF = 9badgood
noopaction
1(4)
X(1)
2(2)
3(6)
X(3)
4(2)
X(1)
X(5)
5(5)
6(9)
7(7)
8(8)
Pattern Database RelaxationPattern Database Relaxation
• The Pattern Database (PDB) relaxation method maps a problem P to a relaxed problem P’ by ignoring some aspects of the states– Option I: Variables deletion: Use only a subset of the
variables from P (but take care to always include a goal variable in this subset)
– Option II: Values domain abstraction: Use only a subset of the domain for some variables
• PDB only clusters states that share some properties (rather than arbitrary states)
• We now have a set of patterns of states– The new state(+actions) space is homomorphic to the
original state space
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 28
Pattern Database RelaxationPattern Database Relaxation
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 29
Use this pattern
Pattern Database RelaxationPattern Database Relaxation
• How to solve using the state patterns:– For each state, optimally solve the abstracted problem
and use the solution length as the heuristic value of the original state
– Construct a database of solutions for all abstracted states in advance (rather than on the fly)
• This method guarantees admissibility: All of the abstracted solutions are shorter than the optimal solution of the original problem
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 30
Pattern Database RelaxationPattern Database Relaxation
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 31
Use this pattern
h* = 3
h = 2
8-Puzzle – PDB Relaxation8-Puzzle – PDB Relaxation
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 32
2 4
1 5 3
7 8 6
tile place
1 4
2 1
3 6
4 3
5 5
6 9
7 7
8 8
tile place
1 4
2 1
4 3
6 9
7 7
8 8
2 4
1
7 8 6
21
4 6
7 8
Goalh = 6
tile place
1 4
2 1
3 6
4 3
5 5
6 9
7 7
8 8
1 2 3
4 5 6
7 8 9
1 2 3
4 5 6
7 8 9
1 2 3
4 5 6
7 8 9
Example action:<5(5), X(2) → 5(2)> <X(2) → Ø> =eff Ø<2(5), X(6) → 2(6)> <2(5), X(6) → 2(6)>
Pattern Database RelaxationPattern Database Relaxation
• Representation is important for understanding the patterned state space – In the last example, the representation was: “Where is
tile X located”. Eliminating tiles 3 and 5 adds two extra empty places on the board
– Using the representation “What tile does place X hold” deletes places, creating strange actions like “place y holds x ↔ place y holds empty”, allowing multiple instances of a tile and allowing the empty board – all of these do not impact admissibility
– However “What tile does place X hold” is a more intuitive representation for values domain abstraction: We could place the empty tile on several placeholders simultaneously
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 33
tile place
1 1
2 2
3 3
Pattern Database RelaxationPattern Database Relaxation
• PDB is not necessarily surjective: The new state space often contains states that have no mapping from the original state space (may affect search speed but not admissibility)
• Example: 3-Puzzle
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 34
This state has no source in the original
state space
tile place
1 1
2 2
3 3
1
3
21 2
3 4
1 21 2
3 4
2 11 2
3 4
Pattern Database RelaxationPattern Database Relaxation
• Very important subject (which we will not discuss): How to choose which variables to delete– Always keep at least one goal variable– Possible strategy: Try to decompose the problem into
as many independent subproblems as possible while retaining as many variables as possible
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 35
SummarySummary
• The solution to a relaxed problem can be used as a heuristic for the original problem
• It is preferable to use an admissible heuristic when possible
• Some (crucial) aspects of the automated process of heuristic functions generation still need to be taken care of manually
Apr 21, 2023
Intro. to AI – Tutorial 5 – By Haggai Toledano 36