Finite State Machine for Games Fall 2012 Ref: Chenney, CS679 lectures AI Game Programming Wisdom 2.

Finite State Machine for Games

Fall 2012Ref: Chenney, CS679 lecturesAI Game Programming Wisdom 2

Fall 2012 2

Outline

AI and GameIntroduction/examplesDesign Intuition State-based

ImplementationExtending Stack-based Fuzzy-state machine

Fall 2012 3

What is AI? (and is NOT)

AI is the control of non-human entities in a game

The other cars in a car game The opponents and monsters in a shooter Your units, your enemy’s units and your enemy in a

RTS game

But, typically does not refer to passive things that just react to the player and never initiate action

That’s physics or game logic For example, the blocks in Tetris are not AI, nor is a

flag blowing in the wind

Fall 2012 4

AI in the Game Loop

AI is updated as part of the game loop, after user input, and before rendering

Fall 2012 5

AI Module

AI Update Step

The sensing phase determines the state of the world

May be very simple - state changes all come by messaging

Or complex - figure out what is visible, where your team is, etc

The thinking phase decides what to do given the world

The core of AI

The acting phase tells the animation what to do

Game Engine

Sensing

Thinking

Acting

Fall 2012 6

AI by Polling

Senses: looks to see what has been changed in the

world

then acts on it

Generally inefficient and complicated Different characters might require different

polling rate

AI gets called at a fixed rate

Event Driven AI

• Events triggered by a message • basically, a function gets called when

a message arrives, just like a user interface)

• Example messages: You have heard a sound Someone has entered your field of

view A certain amount of time has passed,

so update yourself

Fall 2012 7

does everything in response to events in the world

Fall 2012 8

AI Techniques in Games

Basic problem: Given the state of the world, what should I do?A wide range of solutions in games: Finite state machines, Decision trees, Rule

based systems, Neural networks, Fuzzy logic

A wider range of solutions in the academic world: Complex planning systems, logic

programming, genetic algorithms, Bayes-nets Typically, too slow for games

Fall 2012 9

Expressiveness

What behaviors can easily be defined, or defined at all?Propositional logic:

Statements about specific objects in the world – no variables

Jim is in room7, Jim has the rocket launcher, the rocket launcher does splash damage

Go to room8 if you are in room7 through door14Predicate Logic:

Allows general statement – using variables All rooms have doors All splash damage weapons can be used around corners All rocket launchers do splash damage Go to a room connected to the current room

Fall 2012 10

Finite State Machines (FSMs)

A set of states that the agent can be inConnected by transitions that are triggered by a change in the worldNormally represented as a directed graph, with the edges labeled with the transition eventUbiquitous in computer game AI

Fall 2012 11

Formal Definitions (N. Philips)

"An abstract machine consisting of a set of states (including the initial state), a set of input events, a set of output events, and a state transition function. The function takes the current state and an input event and returns the new set of output events and the next state. Some states may be designated as "terminal states".The state machine can also be viewed as a function which maps an ordered sequence of input events into a corresponding sequence of (sets of) output events.Finite State Automaton: the machine with no output

Fall 2012 12

FSM with Output: vending machines

OJ & AJ for 30 centsState table

Fall 2012 13

Vending Machine: state diagram

Fall 2012 14

FSM and Game

Game character behavior can be modeled (in most cases) as a sequence of different “mental state”, where change is driven by the actions of player/other characters, …Natural choice for defining AI in games

Fall 2012 15

FSM Examples

Fall 2012 16

Fall 2012 17

Fall 2012 18

PacMan State Machine

Fall 2012 19

PacMan eats a Power Pill

Timer <= 0

Collision with PacMan

Timer <= 0

Collision with GhostBox

Actions of States

Fall 2012 20

Roam;If PacMan gets close, PathTo (PacMan)

Timer--;PathAwayFrom (PacMan)

PathTo (GhostBox)Timer—Move back-and-forth

Fall 2012 21

Ex: predator vs. prey

Prey (laalaa)

Idle(stand,wave,…)

Flee(run)

Sees predator

No more threat

capturedDead

Fall 2012 22

Predator (Raptor)

Idle(stand)

Hungry(wander)

Pursuit(run)

Tidle > 5

Prey in sight

Tpursuit > 10

Dining

Prey captured

Tdining>5

Fall 2012 23

Idling LaaLaa

Stand

Wave

Wander(set random target)

50%

30%20%

Target arrived

Twave>2

RTstand>4

This page illustrates:hierarchical state,Non-deterministic state transition

This page illustrates:hierarchical state,Non-deterministic state transition

FSM Design

Fall 2012 25

Quake2 ExamplesQuake2 uses 9 different states:

standing, walking, running, dodging, attacking, melee, seeing the enemy, idle and searching.

Incomplete design

Intuitive thinking: model the events and state changes

Intuitive thinking: model the events and state changes

Fall 2012 26

Quake: Rocket

Fall 2012 27

Shambler monster

Fall 2012 28

FSM Advantages

Very fast – one array accessExpressive enough for simple behaviors or characters that are intended to be “dumb”

Can be compiled into compact data structure Dynamic memory: current state Static memory: state diagram – array implementation

Can create tools so non-programmer can build behaviorNon-deterministic FSM can make behavior unpredictable

Fall 2012 29

FSM Disadvantages

Number of states can grow very fast Exponentially with number of events: s=2e

Number of arcs can grow even faster: a=s2

Propositional representation Difficult to put in “pick up the better powerup”,

“attack the closest enemy” Expensive to count: Wait until the third time I see

enemy, then attack Need extra events: First time seen, second time seen,

and extra states to take care of counting

FSM Control System Implementation

Fall 2012 31

FSM Implementation

Fall 2012 32

Previous Example

Fall 2012 33

Code 1 Ad hoc implementation

Fall 2012 34

Code 1p

Fall 2012 35

Code 2Structure, Readable, maintainable

Fall 2012 36

Hierarchical …

FSM Extensions

Advanced Issues

Hierarchical FSMNon-deterministic FSMSwapping FSM

Fall 2012 38

Fall 2012 39

Hierarchical FSMs

What if there is no simple action for a state?Expand a state into its own FSM, which explains what to do if in that stateSome events move you around the same level in the hierarchy, some move you up a levelWhen entering a state, have to choose a state for its child in the hierarchy

Set a default, and always go to that Or, random choice Depends on the nature of the behavior

Fall 2012 40

Stack-based FSM

History stack: Remember previous state; create characters with a memory …

Pushdown automaton(PDA)

Fall 2012 41

Goal-based vs. State-based

There is also a slight derivative to the state-based engine, but it used in more complicated games like flight simulators and games like MechWarrior. They use goal -based engines - each entity within the game is assigned a certain goal, be it 'protect base', 'attack bridge', 'fly in circles'. As the game world changes, so do the goals of the various entities.

Fall 2012 42

Processing Models

Polling FSM update

frequency Easy to implement

and debug Inefficiency (Little

Red example)

Event-driven Publish-subscribe

messaging system Game engine sends

event messages to individual FSMs

An FSM subscribes only to the events that have the potential to change the current state

Higher efficiency, non-trivial implementation

Fall 2012 43

Efficiency and Optimization

In AI, FSM is the most efficient technology availableYet, there is always room for improvement Level of Detail: depending on the

condition (e.g., distance with player), use different FSM, or different update frequency

Fall 2012 44

References

Web references: www.gamasutra.com/features/19970601/build_brain

s_into_games.htm csr.uvic.ca/~mmania/machines/intro.htm www.erlang/se/documentation/doc-4.7.3/doc/design_

principles/fsm.html www.microconsultants.com/tips/fsm/fsmartcl.htm http://www.angelfire.com/dragon/letstry/tutorials/dfa/

Game Programming Gems Sections 3.0 & 3.1

It’s very very detailed, but also some cute programming

Well, the sad news …

Fall 2012 45

Behavior trees

Fall 2012 46

Additional Materials

Fall 2012 47

Fall 2012 48

Intuitive Design

Say, a simple teletube baby has three states: idle, run, and waveScenario: When an idle laalaa sees a butterfly,

it waves to it. When the butterfly flies away, it returns to idle

When an idle laalaa sees a mouse, it flees away. When the mouse is no longer in sight, it returns to idle

Fall 2012 49

Laalaa

idle

flee

wave

mouse

~mouse

~butterfly

butterfly

How to make sure the design complete? I.e., all states and transitions are considered

Is there any systematic way to develop an FSM?

Fall 2012 50

Quake Bot Example (first-cut)

Types of behavior to capture: Wander randomly if don’t see or hear an

enemy When see enemy, attack When not see enemy and hear an enemy,

chase enemy When die, re-spawn (new a bot from start)

Events: see enemy, hear enemy, dieStates: wander, attack, chase, spawn

Fall 2012 51

Remark

With 3 events, potentially there should be 23 states: (E,S,D)=(0,0,0),(1,0,0),(0,1,0), …,(1,1,1)

Some doesn’t make sense E.g., ESD = 111

Name and create a state for the ones that we want to consider Wander (ESD=000) Chase (ESD=010) Attack (ESD=1x0), x for dont-care Die (ESD=xx1)

Fall 2012 52

FSM (first-cut)

Events: E: see an enemy S: hear a sound D: die

States: E: enemy in sight S: sound audible D: dead

D

~E

ED

~S

S

D

E S D

Problem: Can’t go directly from attack to chase. Why not?

Spawnxx1

Wander000

Attack1x0

Chase010

start

Fall 2012 53

FSM (first-cut)

Events: E: see an enemy S: hear a sound D: die

States: E: enemy in sight S: sound audible D: dead

D

~E

ED

~S

S

D

E S DSpawn

xx1

Wander000

Attack100

Chase010

start E

Attack+S110

~E

~S

S

D

Extra state needs to be defined

Extra state needs to be defined

Fall 2012 54

Quake Bot Example (refined)

Types of behavior to capture: Wander randomly if don’t see or hear an enemy When see enemy, attack When not see enemy and hear an enemy, chase

enemy When die, respawn

Extensions: When health is low and see an enemy, retreat When see power-ups during wandering, collect

them [hierarchical FSM]

Fall 2012 55

Example FSM with Retreat

SpawnD

(-E,-S,-L)

Wander-E,-D,-S,-L

E

-SAttack-EE,-D,-S,-L

E

Chase-E,-D,S,-L

S

D

S

D

D

Retreat-EE,-D,-S,L

L

-E

Retreat-S-E,-D,S,L

Wander-L-E,-D,-S,L

Retreat-ESE,-D,S,L

Attack-ESE,-D,S,-L

E

E-E

-L

S

-S

L

-E E

L-L

-L

-L

L

D

• States:– E: enemy in

sight– S: sound

audible– D: dead– L: Low health

A lot more states got added

Fall 2012 56

Hierarchical FSM Example

Note: This is not a complete FSM All links between top level states still

exist Need more states for wander

StartTurn Right

Go-throughDoor

Pick-upPowerup

Wander Attack

Chase

Spawn

~E

E~S

SD

~E

Fall 2012 57

Non-Deterministic HierarchicalFSM (Markov Model)

Adds variety to actionsHave multiple transitions for the same eventLabel each with a probability that it will be takenRandomly choose a transition at run-timeMarkov Model: New state only depends on the previous state

Attack

Start

Approach

Aim & Jump &Shoot

Aim & Slide Left& Shoot

Aim & Slide Right

& Shoot .3.3

.4

.3.3

.4