Finite State Machine for Games Spring 2005 Ref: Chenney, CS679 lectures AI Game Programming Wisdom 2.

Finite State Machine for Games

Spring 2005Ref: Chenney, CS679 lecturesAI Game Programming Wisdom 2

Spring 2005 2

Outline

AI and GameIntroduction/examplesDesign Intuition State-based

ImplementationExtending Stack-based Fuzzy-state machine

Spring 2005 3

What is AI?

AI is the control of every non-human entity in a game

The other cars in a car game The opponents and monsters in a shooter Your units, your enemy’s units and your enemy in a

RTS game

But, typically does not refer to passive things that just react to the player and never initiate action

That’s physics or game logic For example, the blocks in Tetris are not AI, nor is a

flag blowing in the wind

Spring 2005 4

AI in the Game Loop

AI is updated as part of the game loop, after user input, and before renderingThere are issues here: Which AI goes first? Does the AI run on every frame? (LOD

problem) Is the AI synchronized?

Spring 2005 5

AI and Animation

AI determines what to do and the animation does it

AI drives animation, deciding what action the animation system should be animating

Scenario 1: The AI issues orders like “move from A to B”, and it’s up to the animation system to do the rest

Scenario 2: The AI controls everything down to the animation clip to play

Which scenario is best depends on the nature of the AI system and the nature of the animation system

Is the animation system based on move trees (motion capture), or physics, or something else

Does the AI look after collision avoidance? Does it do detailed planning?

Spring 2005 6

AI Module

AI Update Step

The sensing phase determines the state of the world

May be very simple - state changes all come by message

Or complex - figure out what is visible, where your team is, etc

The thinking phase decides what to do given the world

The core of AI

The acting phase tells the animation what to do

Generally not interesting

Game Engine

Sensing

Thinking

Acting

Spring 2005 7

AI by Polling

The AI gets called at a fixed rateSenses: It looks to see what has changed in the world. For instance: Queries what it can see Checks to see if its animation has finished

running

And then acts on itWhy is this generally inefficient? Different characters might require different

polling rate

Spring 2005 8

Event Driven AI

Event driven AI does everything in response to events in the world

Events sent by message (basically, a function gets called when a message arrives, just like a user interface)

Example messages: A certain amount of time has passed, so update

yourself You have heard a sound Someone has entered your field of view

Note that messages can completely replace sensing, but typically do not. Why not?

Real system are a mix - something changes, so you do some sensing

Spring 2005 9

AI Techniques in Games

Basic problem: Given the state of the world, what should I do?A wide range of solutions in games: Finite state machines, Decision trees, Rule

based systems, Neural networks, Fuzzy logic

A wider range of solutions in the academic world: Complex planning systems, logic

programming, genetic algorithms, Bayes-nets Typically, too slow for games

Spring 2005 10

Goals of Game AI

Several goals: Goal driven - the AI decides what it should do, and

then figures out how to do it Reactive - the AI responds immediately to changes in

the world Knowledge intensive - the AI knows a lot about the

world and how it behaves, and embodies knowledge in its own behavior

Characteristic - Embodies a believable, consistent character

Fast and easy development Low CPU and memory usage

These conflict in almost every way

Spring 2005 11

Two Measures of Complexity

Complexity of Execution How fast does it run as more knowledge is

added? How much memory is required as more

knowledge is added? Determines the run-time cost of the AI

Complexity of Specification How hard is it to write the code? As more “knowledge” is added, how much

more code needs to be added? Determines the development cost, and risk

Spring 2005 12

Expressiveness

What behaviors can easily be defined, or defined at all?Propositional logic:

Statements about specific objects in the world – no variables

Jim is in room7, Jim has the rocket launcher, the rocket launcher does splash damage

Go to room8 if you are in room7 through door14Predicate Logic:

Allows general statement – using variables All rooms have doors All splash damage weapons can be used around corners All rocket launchers do splash damage Go to a room connected to the current room

Spring 2005 13

Finite State Machines (FSMs)A set of states that the agent can be inConnected by transitions that are triggered by a change in the worldNormally represented as a directed graph, with the edges labeled with the transition eventUbiquitous in computer game AIYou might have seen them, a long time ago, in formal language theory (or compilers)

What type of languages can be represented by finite state machines?

How might this impact a character’s AI? How does it impact the size of the machine?

Spring 2005 14

Formal Definitions (N. Philips)

"An abstract machine consisting of a set of states (including the initial state), a set of input events, a set of output events, and a state transition function. The function takes the current state and an input event and returns the new set of output events and the next state. Some states may be designated as "terminal states".The state machine can also be viewed as a function which maps an ordered sequence of input events into a corresponding sequence of (sets of) output events.Finite State Automaton: the machine with no output

Spring 2005 15

FSM with Output: vending machines

[description]State table

Spring 2005 16

Vending Machine: state diagram

Spring 2005 17

FSM and Game

Game character behavior can be modeled (in most cases) as a sequence of different “mental state”, where change is driven by the actions of player/other characters, …Natural choice for defining AI in games

Spring 2005 18

FSM with No Output

Spring 2005 19

Ex: predator vs. prey

Prey (laalaa)

Idle(stand,wave,…)

Flee(run)

Sees predator

No more threat

capturedDead

Spring 2005 20

Predator (Raptor)

Idle(stand)

Hungry(wander)

Pursuit(run)

Tidle > 5

Prey in sight

Tpursuit > 10

Dining

Prey captured

Tdining>5

Spring 2005 21

Idling LaaLaa

Stand

Wave

Wander(set random target)

50%

30%20%

Target arrived

Twave>2

RTstand>4

This page illustrates:hierarchical state,Non-deterministic state transition

This page illustrates:hierarchical state,Non-deterministic state transition

Spring 2005 22

Spring 2005 23

FSM + Cinematography

The current state (and state transition) may have an important effect on how camera should be manipulated Idling: focus on character Movement: zoom out to show spatial info State-transition: camera animation …

More complicated camera behavior may be coded in another FSM

Spring 2005 24

Camera Selection Strategy

It is important to choose a camera that has the front of the character

Z

X

Spring 2005 25

Camera Selection (cont)

-v

A good camera:-v lies in the frustum

Spring 2005 26

Other Concerns

• Adjust zoom (fovy) so that the character occupies a good (fixed!?) portion of the screen

x

w

r

percentd

r

near

nearnearnear

w

fovyw

xpercent

d

near

r

x

percentdr

percentx

2/

2tan

:

2/

2/

percentd

r

near

nearnearnear

w

fovyw

xpercent

d

near

r

x

percentdr

percentx

2/

2tan

:

2/

2/

near

d

Spring 2005 27

FOVY based on Postures

Once in running mode, zoom outIn stand/wave modes, zoom inSmooth transition between two fovy settings (by indirectly controlling the percent value)

FSM Design

Spring 2005 29

Quake2 ExamplesQuake2 uses 9 different states:

standing, walking, running, dodging, attacking, melee, seeing the enemy, idle and searching.

Incomplete design

Intuitive thinking: model the events and state changes

Intuitive thinking: model the events and state changes

Spring 2005 30

Quake: Rocket

Spring 2005 31

Shambler monster

Spring 2005 32

Intuitive Design

Say, a simple teletube baby has three states: idle, run, and waveScenario: When an idle laalaa sees a butterfly, it wa

ves to it. When the butterfly flies away, it returns to idle

When an idle laalaa sees a mouse, it flees away. When the mouse is no longer in sight, it returns to idle

Spring 2005 33

Laalaa

idle

flee

wave

mouse

~mouse

~butterfly

butterfly

How to make sure the design complete? I.e., all states and transitions are considered

Is there any systematic way of developing an FSM?

Spring 2005 34

Quake Bot Example (first-cut)

Types of behavior to capture: Wander randomly if don’t see or hear an enemy When see enemy, attack When not see enemy and hear an enemy, chase e

nemy When die, re-spawn (new a bot from start)

Events: see enemy, hear enemy, dieStates: wander, attack, chase, spawn

Spring 2005 35

Remark

With 3 events, potentially there should be 23 states: (E,S,D)=(0,0,0),(1,0,0),(0,1,0), …,(1,1,1)

Some doesn’t make sense E.g., ESD = 111

Name and create a state for the ones that we want to consider Wander (ESD=000) Chase (ESD=010) Attack (ESD=1x0), x for dont-care Die (ESD=xx1)

Spring 2005 36

FSM (first-cut)

Events: E: see an enemy S: hear a sound D: die

States: E: enemy in sight S: sound audible D: dead

D

~E

ED

~S

S

D

E S D

Problem: Can’t go directly from attack to chase. Why not?

Spawnxx1

Wander000

Attack1x0

Chase010

start

Spring 2005 37

FSM (first-cut)

Events: E: see an enemy S: hear a sound D: die

States: E: enemy in sight S: sound audible D: dead

D

~E

ED

~S

S

D

E S DSpawn

xx1

Wander000

Attack100

Chase010

start E

Attack+S110

~E

~S

S

D

Extra state needs to be defined

Extra state needs to be defined

Spring 2005 38

Quake Bot Example (refined)

Types of behavior to capture: Wander randomly if don’t see or hear an enemy When see enemy, attack When not see enemy and hear an enemy, chase ene

my When die, respawn

Extensions: When health is low and see an enemy, retreat When see power-ups during wandering, collect them

[hierarchical FSM]

Spring 2005 39

Example FSM with Retreat

SpawnD

(-E,-S,-L)

Wander-E,-D,-S,-L

E

-SAttack-EE,-D,-S,-L

E

Chase-E,-D,S,-L

S

D

S

D

D

Retreat-EE,-D,-S,L

L

-E

Retreat-S-E,-D,S,L

Wander-L-E,-D,-S,L

Retreat-ESE,-D,S,L

Attack-ESE,-D,S,-L

E

E-E

-L

S

-S

L

-E E

L-L

-L

-L

L

D

• States:– E: enemy in

sight– S: sound

audible– D: dead– L: Low health

A lot more states got added

Spring 2005 40

Hierarchical FSMsWhat if there is no simple action for a state?Expand a state into its own FSM, which explains what to do if in that stateSome events move you around the same level in the hierarchy, some move you up a levelWhen entering a state, have to choose a state for its child in the hierarchy

Set a default, and always go to that Or, random choice Depends on the nature of the behavior

Spring 2005 41

Hierarchical FSM Example

Note: This is not a complete FSM All links between top level states still

exist Need more states for wander

StartTurn Right

Go-throughDoor

Pick-upPowerup

Wander Attack

Chase

Spawn

~E

E~S

SD

~E

Spring 2005 42

Non-Deterministic HierarchicalFSM (Markov Model)

Adds variety to actionsHave multiple transitions for the same eventLabel each with a probability that it will be takenRandomly choose a transition at run-timeMarkov Model: New state only depends on the previous state

Attack

Start

Approach

Aim & Jump &Shoot

Aim & Slide Left& Shoot

Aim & Slide Right

& Shoot .3.3

.4

.3.3

.4

FSM Control System Implementation

Spring 2005 44

FSM Implementation

Spring 2005 45

Efficient ImplementationCompile into an array of state-name, eventstate-namei+1 := array[state-namei, event]Switch on state-name to call execution logicMarkov: Have array of possible transitions for every (state-name,event) pair, and choose one at randomHierarchical

Create array for every FSM Have stack of states

Classify events according to stack Update state which is sensitive to current eve

nt

event

state

Spring 2005 46

FSM Advantages

Very fast – one array accessExpressive enough for simple behaviors or characters that are intended to be “dumb”Can be compiled into compact data structure

Dynamic memory: current state Static memory: state diagram – array implementation

Can create tools so non-programmer can build behaviorNon-deterministic FSM can make behavior unpredictable

Spring 2005 47

FSM Disadvantages

Number of states can grow very fast Exponentially with number of events: s=2e

Number of arcs can grow even faster: a=s2

Propositional representation Difficult to put in “pick up the better powerup”,

“attack the closest enemy” Expensive to count: Wait until the third time I see

enemy, then attack Need extra events: First time seen, second time seen,

and extra states to take care of counting

Spring 2005 48

Example

Spring 2005 49

Code 1 Ad hoc implementation

Spring 2005 50

Code 1p

Spring 2005 51

Code 2Structure, Readable, maintainable

Spring 2005 52

Hierarchical …

FSM Extensions

Spring 2005 54

Stack-based FSM

History stack: Remember previous state; create characters with a memory …

Pushdown automaton(PDA)

Spring 2005 55

Goal-based vs. State-based

There is also a slight derivative to the state-based engine, but it used in more complicated games like flight simulators and games like MechWarrior. They use goal -based engines - each entity within the game is assigned a certain goal, be it 'protect base', 'attack bridge', 'fly in circles'. As the game world changes, so do the goals of the various entities.

Spring 2005 56

Processing Models

Polling FSM update

frequency Easy to implement

and debug Inefficiency (Little

Red example)

Event-driven Publish-subscribe mess

aging system Game engine sends eve

nt messages to individual FSMs

An FSM subscribes only to the events that have the potential to change the current state

Higher efficiency, non-trivial implementation

Spring 2005 57

Interfacing with Game Engine

Query, act on the game worldMove the function calls (to the game engine) as DLL; reduce the amount of recompilation required

Spring 2005 58

Efficiency and Optimization

In AI, FSM is the most efficient technology availableYet, there is always room for improvement Level of Detail: depending on the

condition (e.g., distance with player), use different FSM, or different update frequency

Spring 2005 59

References

Web references: www.gamasutra.com/features/19970601/build_brains_in

to_games.htm csr.uvic.ca/~mmania/machines/intro.htm www.erlang/se/documentation/doc-4.7.3/doc/design_pri

nciples/fsm.html www.microconsultants.com/tips/fsm/fsmartcl.htm http://www.angelfire.com/dragon/letstry/tutorials/dfa/

Game Programming Gems Sections 3.0 & 3.1

It’s very very detailed, but also some cute programming