Challenges for implementing Monte Carlo Tree Search in commercial games

1

Click icon to add picture

Challenges for implementing Monte Carlo Tree Search in commercial games

Matthew Bedder

16/Sept/2016

216/Sept/2016

Disclaimer

This is pitched at people with little/no knowledge of the area

(Sorry, AI people)

3

“A game is a series of interesting choices”

16/Sept/2016

416/Sept/2016

5

Monte Carlo Tree Search

16/Sept/2016

616/Sept/2016

Monte Carlo Tree SearchOnly generate the bits of the game tree we need

716/Sept/2016

MCTS – the basics

Iteratively builds a decision tree

816/Sept/2016

MCTS Selection

916/Sept/2016

MCTS Expansion

1016/Sept/2016

MCTS Simulation

1116/Sept/2016

Monte Carlo evaluationsPerforming random actions gives us a (weak) approximation of state values

X O X OX

OX

O

X

1216/Sept/2016

MCTS Simulation

1316/Sept/2016

MCTS Backpropagation

1416/Sept/2016

MCTS – the basics

15

Applications of MCTS

16/Sept/2016

1616/Sept/2016

MCTS applications

• Amazons• Arimaa• Connect 4• Dou Di Zhu• Go variants• Havannah

• Hex• Morpion Solitaire• Othello• Settlers of Catan• Spades• ...

Many board and card games, including:

1716/Sept/2016

1816/Sept/2016

MCTS applicationsAll of the strongest AI players of Go use variants of MCTS

1916/Sept/2016

MCTS applicationsCurrently there are very limited applications to commercial video games or other domains

2016/Sept/2016

http://www.aifactory.co.uk/AIF_Games_Spades.htm

2116/Sept/2016

MCTS Successes

http://aigamedev.com/open/coverage/mcts-rome-ii/

2216/Sept/2016

http://gwaredd.github.io/nuclai_mcts

23

Challenges for MCTS

16/Sept/2016

2416/Sept/2016

Games are hard

There are as many states of Civilization II as there are atoms in the

observable universe 10,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,

000,000,000,000,000,000,000,000,000,000 times over

2516/Sept/2016

Computational costThis is actually important

Game AIs (usually) are limited to time sliced CPU

AlphaGo used• 1,920 CPUs and 280 GPUs• ~1MW

2616/Sept/2016

Monte Carlo Tree Search optimisationsGenerally either:

Domain specificor

Overly generic

2716/Sept/2016

Monte Carlo Tree Search optimisationsReplacing random simulations helps… usually

Adding prior knowledge helps… usually

2816/Sept/2016

Our approachDomain-specific optimisations for arbitrary domains

I’d love to talk about our work during/after coffee

www-users.cs.york.ac.uk/[email protected]@bedder

Matthew Bedder