Games as Game Theory Systems (Ch. 19). Game Theory It is not a theoretical approach to games Game theory is the mathematical study of decision making.

Games as Game Theory Systems(Ch. 19)

Game Theory• It is not a theoretical approach to games

• Game theory is the mathematical study of decision making– Origins in the field of economics– Examples of “games”:

• Bankruptcy• Investment

• It concerns with how to make decisions: – Idea: reason with the consequences of decisions (i.e.,

choosing a game action) made

• Contrary to the book claim’s game theory techniques have been used to build successful game playing algorithms– Specifically for turn-based games

Game Trees

• Two players (MAX and MIN) alternate moves (turn-based)

• The tree indicates all possible movements by the players

• The root of the tree indicates the starting state

• The children of the root indicate possible moves by MAX

• The children of the children indicate possible moves by MIN

• Tree continues alternating until it reaches end-game situations

…

First example

MAX moves

…

MIN moves(which depend on MAX’s last move)

Game Trees (2)

Concepts:• State: node in tree• Terminal node: game over

• Utility function: value for outcome of the game.

• MAX: 1st player, maximizing its own utility

• MIN: 2nd player, minimizing Max’s utility

Utility: 1 Max won 0 Max and Min draw -1 Max lost (Min won)

Minimax

• Finding perfect play for deterministic, perfect information games

• Idea: choose move to position with: highest utility for MAX = best achievable payoff against a rational opponent

• Example: Utility is a number between 2 and 14

Properties of Minimax• Always find an optimal strategy for MAX (or MIN)?

• Size of game tree? – b: branching factor (maximum number of moves a

player can make in his/her turn)– m: # turns in an average game

Yes (if tree is finite and opponent is rational)

approximately bm

• For Tic-Tac-Toe, b ≈ 9, m ≈10. Thus, the size of the tree is 910. Any computer nowadays will find the optimal strategy: Always draw against a rational opponent

• For chess, b ≈ 35, m ≈60. Thus, the size of the tree is 3560. Too large for computers today. Therefore tree can only be partially constructed. Look ahead of 6 turns for most computers

• Evaluation functions are used to measure utility of non-terminal states looking ahead

Evaluation Function

• Evaluation Function– Is an estimate of the actual utility– Typically represented as a linear function:

EF(state) = w1f1(state) + w2f2(state) + … + wnfn(state)

– Example:

Chess• weight: Piece Number

(w1) Pawn 1 (w2) Knight 3 (w3) Bishop 3 (w4) Rook 5 (w5) Queen 9

• Function; state Number f1 = #(pawns,w) #(pawns,b) f2 = #(knight,w) #(knight,b) f3 = #(bishop,w) #(bishop,b) f4 = #(rook,w) #(rook,b) f5 = #(knight,w) #(knight,b)

Example: Evaluation Function

“all things been equal”White moves,Who is winning?

Is this consistent with Evaluation function?

Black

Yes!

Evaluation Function (2)

• Obviously, the quality of the AI player depends on the evaluation function

• Conditions for evaluation functions:

If n is a terminal node, Computing EF should not take longEF should reflect chances of winning

EF(n) = Utility(n)

If EF(state) > 3 then is almost-certain that blacks win

Horizon: Cutting Off Search

• When to cutoff minimax expansion?

• Potential problem with cutting off search: Horizon problem

• Solution:

Fixed depth limit Iterative deepening until times runs out

Decision made by opponent is damaging but cannot be “seen” because of cutoff

Quiescent: states that are unlikely to exhibit wild swings in the values of the evaluation functions

Example: Horizon Problem

“all things been equal”White moves,Who is winning?

Is this consistent with Evaluation function?

White

No!

In practice

• Checkers: Chinook ended 40-year-reign of human world champion Marion Tinsley in 1994. Used a precomputed endgame database defining perfect play for all positions involving 8 or fewer pieces on the board, a total of 444 billion positions.

• Chess: Deep Blue defeated human world champion Garry Kasparov in a six-game match in 1997. Deep Blue searches 200 million positions per second, 24 processors, heuristics to guide construction of game tree with help of human grand masters

• Othello: human champions refuse to compete against computers. Computers are too good.

• Go: human champions refuse to compete against computers, who are too bad. In go, b > 300, so most programs use pattern knowledge bases to suggest plausible moves.

Saddle Points

– Saddle point: Developing a strategy for playing the game that ensures victory (or non defeat) every time• Example: Tic-Tac-Toe

– Results in loss of meaningful play• It is a theoretical possibility but if in

practice can’t figured it out then it is ok• But sometimes is discovered by players

as emergent behavior– “Exploits”

Victory!

…… …

Summoning Necromancer

• Some Design principles:– Rather weak character (not much of direct damage;

can’t sustain a lot of damage)– Uses curses to weaken foe’s attacks/defenses– Killed monsters can be revived to serve the

necromancer• But they last 2 minutes only (negative feedback!)

– Reviving monsters, casting curses cost mana • Mana is limited and renews itself very slowly

(negative feedback!)

http://www.youtube.com/watch?v=UJdB-ydfUho

Saddle point: people killed shamans mobs which could revive other mobs! (positive feedback)

Recent Research

• Use Minimax in RTS games• In such games there are frequent battles taken place

between small armies

• Who should attack who?– What happens if X attacks A first and then B– Easily represented as a minimax tree