10/19/2004 TCSS435A Isabelle Bichi ndaritz 1 Game and Tree Searching
Feb 14, 2016
10/19/2004 TCSS435A Isabelle Bichindaritz 1
Game and Tree Searching
10/19/2004 TCSS435A Isabelle Bichindaritz 2
Learning Objectives
• Introduction to games
• Optimal decisions in games
• Alpha-Beta pruning
• Real-time decisions
10/19/2004 TCSS435A Isabelle Bichindaritz 3
Introduction to Games• Games as Search Problems
– Frameworks: two-player, multi-player; zero-sum; perfect information
– Minimax algorithm• Perfect decisions• Imperfect decisions (based upon static evaluation function)
– Issues• Quiescence• Horizon effect
– Need for pruning (alpha-beta pruning)
10/19/2004 TCSS435A Isabelle Bichindaritz 4
Introduction to Games• Perfect Play
– General framework(s)– What could agent do with perfect info?
• Resource Limits– Search ply– Static evaluation: from heuristic search to heuristic game tree search
– Examples• Tic-tac-toe, connect four, checkers, connect-five • Chess, go
• Games with Uncertainty– Explicit: games of chance (e.g., backgammon, Monopoly, blackjack)– Implicit
10/19/2004 TCSS435A Isabelle Bichindaritz 5
Games versus Search Problems• Unpredictable Opponent
– Games are adversarial search problems– Solution is strategy, contingency plan– Time limits
• Unlikely to find goal• Must approximate
• Plan of Attack– Algorithm for perfect play (J. von Neumann, 1944)– Finite horizon, approximate evaluation (C. Zuse, 1945; C.
Shannon, 1950, A. Samuel, 1952-1957)– First chess program (Turing, 1951)– Pruning to allow deeper search (J. McCarthy, 1956)
10/19/2004 TCSS435A Isabelle Bichindaritz 6
Types of Games• Information: Can Know (Observe)
– … outcomes of actions / moves?– … moves committed by opponent?
• Uncertainty– Deterministic vs. nondeterministic outcomes– Thought exercise: sources of nondeterminism?
10/19/2004 TCSS435A Isabelle Bichindaritz 7
Minimax• Games with two players MIN and MAX are a search problem
with:– Initial state– Successor function– Terminal test / state– Utility function (objective / payoff)
• Win / loss / draw (chess)• +1 / -1 / 0 (chess)• -192 … +192 (backgammon)
10/19/2004 TCSS435A Isabelle Bichindaritz 8
Minimax
10/19/2004 TCSS435A Isabelle Bichindaritz 9
Minimax
• Perfect play for deterministic, perfect-information games
• Choose move to position with highest minimax value= best achievable payoff against best play
• Simple example: 2-ply game, a ply being a half-move
• Game ends after one move each by MAX and MIN (one move deep)
10/19/2004 TCSS435A Isabelle Bichindaritz 10
Minimax
10/19/2004 TCSS435A Isabelle Bichindaritz 11
Minimax Algorithm:Decision and Evaluation
what’s this? what’s this?
10/19/2004 TCSS435A Isabelle Bichindaritz 12
Properties of Minimax• Complete?
10/19/2004 TCSS435A Isabelle Bichindaritz 13
Properties of Minimax• Complete?
– … yes, provided following are finite:• Number of possible legal moves (generative breadth of tree)• “Length of game” (depth of tree) – more specifically?• A finite strategy can exist even in an infinite tree!
– Perfect vs. imperfect information?• Q: What search is perfect minimax analogous to?• A: Bottom-up breadth-first
10/19/2004 TCSS435A Isabelle Bichindaritz 14
Properties of Minimax• Complete?
– … yes, provided the tree is finite:• Number of possible legal moves (generative breadth of tree)• “Length of game” (depth of tree) – more specifically?
• Optimal?
10/19/2004 TCSS435A Isabelle Bichindaritz 15
Properties of Minimax• Complete?
– … yes, provided the tree is finite
• Optimal?– … yes, provided perfect info (evaluation function) and
opponent is optimal!– … otherwise, guaranteed if evaluation function is
correct
• Time Complexity?
10/19/2004 TCSS435A Isabelle Bichindaritz 16
Properties of Minimax• Complete?
– … yes, provided the tree is finite:• Optimal?
– … yes, provided perfect info (evaluation function) and opponent is optimal!
• Time Complexity?– Depth of tree: m– Legal moves at each point: b– O(bm) – NB, m 100, b 35 for chess!
• Space Complexity?
10/19/2004 TCSS435A Isabelle Bichindaritz 17
Properties of Minimax• Complete?
– … yes, provided the tree is finite:• Optimal?
– … yes, provided perfect info (evaluation function) and opponent is optimal!
• Time Complexity?– Depth of tree: m– Legal moves at each point: b– O(bm) – NB, m 100, b 35 for chess!
• Space Complexity? O(bm) – why?
10/19/2004 TCSS435A Isabelle Bichindaritz 18
Resource Limits
10/19/2004 TCSS435A Isabelle Bichindaritz 19
Static Evaluation Function Example: Chess
10/19/2004 TCSS435A Isabelle Bichindaritz 20
Do Exact Values Matter?
10/19/2004 TCSS435A Isabelle Bichindaritz 21
Cutting Off Search
10/19/2004 TCSS435A Isabelle Bichindaritz 22
Cutting Off Search• Issues
– Quiescence• Play has “settled down”• Evaluation function unlikely to exhibit wild swings in value in near
future– Horizon effect
• “Stalling for time”• Postpones inevitable win or damaging move by opponent• See: Figure 6.9 R&N
• Solutions?– Quiescence search: expand non-quiescent positions further– No general solution to horizon problem at present
10/19/2004 TCSS435A Isabelle Bichindaritz 23
Why Prune?
10/19/2004 TCSS435A Isabelle Bichindaritz 24
Game Tree Search• Perfect Play (TBD from Perfect Information)
– Common framework: two-player (or multi-player), zero-sum– Deterministic or known probability distribution function, totally
observable– Minimax algorithm given ability to search for arbitrary ply
• Assumptions about– … game?– … opponent?
• Specifications and Ramifications– Combinatorics: finite depth (m), breadth (b) of search– Resource limits: need for static evaluation function
• Practical Issues– Effects of depth cutoff– Need for pruning
10/19/2004 TCSS435A Isabelle Bichindaritz 25
- Pruning• Properties of - pruning
– Pruning does not affect final result
– Good move ordering improves effectiveness of pruning
– With “perfect ordering,” time complexity = O(bm/2)• Depth of tree: m• Legal moves at each point: b• Effective branching factor becomes b1/2
• Can easily reach depth 8 and play good chess– A simple example of the value of reasoning about which computations
are relevant ( a form of metareasoning)
10/19/2004 TCSS435A Isabelle Bichindaritz 26
- Pruning• is the value of the best (i.e. highest value) choice we have found so far at
any choice point along the path for MAX
is the value of the best (i.e. lowest value) choice we have found so far at any choice point along the path for MIN.
10/19/2004 TCSS435A Isabelle Bichindaritz 27
Alpha-Beta (-) Pruning:Definitions
10/19/2004 TCSS435A Isabelle Bichindaritz 28
Alpha-Beta (-) Pruning Example
≥ 3
3
3
MAX
MIN
MAX12 8
≤ 2
2 14 5
≤ 14
2
≤5 2
10/19/2004 TCSS435A Isabelle Bichindaritz 29
≥ 3
3
3
MAX
MIN
MAX12 8
≤ 2
2 14 5
≤ 14
2
≤5 2
Alpha-Beta (-) Pruning Example What are , , values here? here?
10/19/2004 TCSS435A Isabelle Bichindaritz 30
Alpha-Beta (-) Pruning:Modified Minimax Algorithm
10/19/2004 TCSS435A Isabelle Bichindaritz 31
Alpha-Beta (-) Pruning Properties• Perfect Play (TBD from Perfect Information)
– Common framework: two-player (or multi-player), zero-sum
– Deterministic or known probability distribution function, totally observable
– Minimax algorithm given ability to search to arbitrary ply• Assumptions about
– … game?
– … opponent?• Specifications and Ramifications
– Combinatorics: finite depth (m), breadth (b) of search
– Resource limits: need for static evaluation function• Practical Issues
– Effects of depth cutoff
– Need for pruning