Optimizing the State Eval Heuristic of Abalone using Evolutionary Algorithms Benjamin Rhew November 28, 2005.

Optimizing the State Eval Heuristic of Optimizing the State Eval Heuristic of Abalone using Evolutionary AlgorithmsAbalone using Evolutionary Algorithms

Benjamin RhewNovember 28, 2005

The Roadmap

Why Should We Optimize?Why Use an Evolutionary Algorithm?Previous WorkAbalone ExplainedExperimental SetupWhat is a Two-Pool EA?Evolutionary Algorithm DetailsFuture Work

Why Should We Optimize?

Heuristics are everywhere

Spam filters

Speech recognition software

Have limitations

Slow, but good solution

Fast, but poor solution

So optimize!

Why Use an Evolutionary Algorithm?

Heuristic development takes time

Improbable for a person to design the optimal heuristic

Need to test many different heuristics

Therefore, an EA is called for

Previous Work

There is a lot of previous work, mainly involving chess heuristics

Chess-specific algorithms

Works mainly on chess playing programs

Do not play among themselves

Population dynamics

This approach should be more general

Abalone Explained

Board Game developed in the 1990s

Sumo Wrestling with Marbles...

Program Demo

Experimental Setup

Main Idea – the fitness of an individual depends on how well that individual plays games

Randomly chosen

Play 2 games, one as each color

Takes a long time

Therefore, a steady-state two-pool EA with high selective pressure is used

What is a Two-Pool EA?

An EA where individuals are in two groups

Can use many separations

Male/Female

Predator/Prey

Child/Adult

In this case, child/adult is used

Evolutionary Algorithm Details

Parent Selection – stochastic, based on fitnessSurvival Selection – stochastic/elitist, with worst individual having highest chance of dyingInitial Population – small, 20 individualsRecombination – N-point crossoverMutation – Gaussian Random VariableFitness – number of wins divided by the number of games playedInitialization – small random floats near 1

Evolutionary Algorithm Details

Evolutionary Process – individual will not be killable in the first 10 generations of its life

Three possible Gene Representations:

Take each part of the previous heuristic and multiply it by a constant

Subdivide parts of the previous heuristic and multiply each part by a constant

Completely subdivide previous heuristic and multiply each part by a constant

Evolutionary Algorithm Details

Gene 1:

board1, board2, mymarbles, oppmarbles, age

Gene 2:

board1_1...board1_5, board2_1...board2_5, mymarbles, oppmarbles, age

Gene 3:

board1_1_1...board1_9_9, board2_1_1...board2_9_9, mymarbles, oppmarbles, age

Evolutionary Algorithm Details

Gene 1:

Gene 2:

Gene 3:

Future Work

Results

Compare different fitness function strategies

Compare different gene representations

Use Genetic Programming to evolve new heuristics from scratch

Questions?