2010 Simulated Car Racing Championship @ CIG-2010
The 2010 Simulated Car Racing
Championship @ CIG-2010Daniele Loiacono, Luigi Cardamone,
Martin V. Butz, and Pier Luca Lanzi
2010 Simulated Car Racing Championship @ CIG-2010
2010 Simulated Car Racing Championship9 races during 3 conferences
Develop a driver for TORCS(hand-coded, learned, evolved, …)
Drivers will be awarded based on their score
in each conference competition
At the end, the team with highest overall scorewins the championship
2
2010 Simulated Car Racing Championship @ CIG-2010
The 2010 Simulated Car Racing Championship
GECCO-2010Portland , July 7-11American LegWinner: Autopia
WCCI-2010Barcelona, July 18-23Formula LegWinner: Jorge Muñoz
CIG-2010Copenhagen, August 18–21Dusty LegWinner: ???
2010 Simulated Car Racing Championship @ CIG-2010
What is the structure of a race?
Three stages: warm up, qualifiers, actual race
During warm-up, each driver can explore the track and learn something useful
During qualifiers, each driver races alone against the clock (the best 8 drivers move to the race)
During the race all the drivers race together
4
2010 Simulated Car Racing Championship @ CIG-2010
What’s new?
If everything seems under control, you're not going fast enough— Mario Andretti
Warm-up stage
Before qualifying stage, competitors have 100000 game ticks to race on the track
Allows track learning and optimization of parameters
Noisy sensors
Track sensors and opponent sensors are affected by a Gaussian noise (standard deviation equal to 10% of the readings)
Extended sensor model
Focus sensors
Z position and speed
Direction of track sensors fully customizable
Added clutch control and focus command
The Open Racing Car Simulator
2010 Simulated Car Racing Championship @ CIG-2010
The Open Racing Car Simulator
TORCS is a state of the art open source simulator written in C++
Main features
Sophisticated dynamics
Provided with several cars, tracks, and controllers
Active community of users and developers
Easy to develop your own controller
OS Support
Linux: binaries and building from sources
Windows: binaries and “limited” bulding from sources support
OSX: legacy binaries and no building from sources support
2010 Simulated Car Racing Championship @ CIG-2010
The Open Racing Car Simulator & the Competition Software
TORCS
BOT BOT BOT
TORCS
PATCH
SBOT SBOT SBOT
BOT BOTBOT
UDP UDPUDP The competition server
Separates the bots from TORCS
Build a well-defined sensor model
Works in real-time
2010 Simulated Car Racing Championship @ CIG-2010
Sensors and actuators
Rangefinders for edges on the track and opponents
Speed, RPM, fuel, damage, angle with track, distance race, position on track, etc.
Six effectors: steering wheel [-1,+1], gas pedal [0, +1], brake pedal [0,+1], gearbox {-1,0,1,2,3,4,5,6}, clutch [0,+1], focus direction
Competitors
2010 Simulated Car Racing Championship @ CIG-2010
The competitors
Five entries in the second leg
AUTOPIA, Madrid and Granada
J. Muñoz, Carlos III University of Madrid
S.Pohl, J. Quadflieg and T. Delbrügger, TU Dortmund
Joseph Alton, University of Birmingham
Timothy Alford (Xiaodong Li), RMIT University, Melbourne
Neil Clarke, University of Birmingham
Two more entries from the 2009 championship
COBOSTAR (T. Lönneker & M.V. Butz, University ofWürzburg)
POLIMI (Cardamone, Politecnico di Milano)
2010 Simulated Car Racing Championship @ CIG-2010
Industrial Computer Science Department.
Centro de Automática y Robótica
Consejo Superior de Investigaciones Científicas
Madrid, Spain
Contact:E. Onieva ([email protected])
AUTOPIA
2010 Simulated Car Racing Championship @ CIG-2010
AUTOPIA
Fuzzy Architecture based on three basic modules for gear, steering and speed control
optimized with a genetic algorithm
Learning in the warm-up stage:
Maintain a vector with as many real values as tracklengthin meters.
Vector initialized to 1.0
If the vehicle goes out of the track or suffers damage then multiply vector positions from 250 meters before the current position by 0.95.
During the race the vector is multiplied by F to make the driver more cautious in function of the damage:
F=1-0.02*round(damage/1000)
2010 Simulated Car Racing Championship @ CIG-2010
Susanna Pohl, Jan Quadflieg and Tim DelbrüggerTU Dortmund
Mr Racer
2010 Simulated Car Racing Championship @ CIG-2010
Mr. Racer
Save the model after warm-up, use it during qualifying and the race
The track model is built using a classifier
Use the model to derive a plan consisting of target speeds and a racing line
Unfortunately the noisy completelybreaks the classifier
2010 Simulated Car Racing Championship @ CIG-2010
Department of Computer ScienceCarlos III University of Madrid
Jorge Muñoz
2010 Simulated Car Racing Championship @ CIG-2010
Jorge Muñoz
Build a model of the track during the warm-up stage.
Two neural networks to predict the trajectory using the track model. Two neural networks to predict the target speed given the model of the track and the current car position
The four neural networks are trained with backpropagation using data retrieved from a human player
Learning during the warm-up:
The car remember where it goes out of the car or drives far form the trajectory and in the next laps goes slower in those points
The car remember where it follows the trajectory perfectly and tries to go faster in the next laps.
2010 Simulated Car Racing Championship @ CIG-2010
Joseph Alton
Joseph Alton
2010 Simulated Car Racing Championship @ CIG-2010
Joseph Alton
Scripted controller
During warm-up the controller drives with a fixed speed of 60 km/h
During this time for each segment (meter) of the track, the turns are recorded and mapped to a speed.
The speeds are as follows:
Noise filtering is done through having multiple sensors at the same position
Type Value Speed ( km/h)
Sharp turn > Absolute 0.1 60
Turn > Absolute 0.05 100
Straight 200
2010 Simulated Car Racing Championship @ CIG-2010
Timothy Alford (supervised by Xiaodong Li)
RMIT University, Melbourne, Victoria
2010 Simulated Car Racing Championship @ CIG-2010
Tim Alford
Y
N
Action object
Action object
on track ?
Recovery
Fuzzy system
Gears
Sensors
All components of the car are controlled by Fuzzy Logic (excluding gears and recovery )
Recovery, Gears are controlled with simple rules
GA is exploited in the Warm-up
2010 Simulated Car Racing Championship @ CIG-2010
Timothy Alford (supervised by Xiaodong Li)
RMIT University, Melbourne, VictoriaNeil ClarkeUniversity of Birmingham
2010 Simulated Car Racing Championship @ CIG-2010
Neil Clarke
Controller based on a Feed Forward Multi Layer Neural Network
Imitation Leaning
Trained using the human data provided by the author
Very simple policy for the opponents
2010 Simulated Car Racing Championship @ CIG-2010
COBOSTAR
Thies Lönneker and Martin V. Butz
University of Würzburg
http://www.coboslab.psychologie.uni-wuerzburg.de
2010 Simulated Car Racing Championship @ CIG-2010
COBOSTAR
One of the best controller of the 2009 Championship
Parameterized controller optimized with CMA-ES
Dynamically saves crash points
2010 Simulated Car Racing Championship @ CIG-2010
CIG-2008 Champ
Luigi Cardamone
Politecnico di Milano
2010 Simulated Car Racing Championship @ CIG-2010
Luigi Cardamone
Winner of the CIG-2008 Car Simulated Competition
The controller is based on a neural network evolved with NEAT
Very simple policy for overtaking
Qualifying
2010 Simulated Car Racing Championship @ CIG-2010
Scoring process: Warm-up Qualifying
Scoring process involves three dirt tracks:
Dirt-1
Olethros-Dirt
Rocky
Only the track Dirt-1 is provided with TORCS distribution:
Olethros-Dirt was created adding the sand to the original track Olethros
Rocky is an evolved track
Each controller raced for 100000 game ticks in the warm-up stage and then its performance is computed in the qualifying stage as the distance covered within 10000 game ticks
2010 Simulated Car Racing Championship @ CIG-2010
Dirt-1
2010 Simulated Car Racing Championship @ CIG-2010
Qualifying: Dirt-1
6125,74
4851,54
2665,89
5340,37
5218,73
4168,25
5696,28
2291,56
0 1000 2000 3000 4000 5000 6000 7000
Autopia
Jorge Muñoz
Joseph Alton
Cardamone
COBOSTAR
MR. Racer
Neil Clarke
Timothy Alford
2010 Simulated Car Racing Championship @ CIG-2010
Olethros-Dirt
2010 Simulated Car Racing Championship @ CIG-2010
Qualifying: Olethros-Dirt
6988,33
2292,59
0
5024,56
6285,36
2352,1
2135,18
3176,7
0 1000 2000 3000 4000 5000 6000 7000 8000
Autopia
Jorge Muñoz
Joseph Alton
Cardamone
COBOSTAR
MR. Racer
Neil Clarke
Timothy Alford
2010 Simulated Car Racing Championship @ CIG-2010
Rocky
2010 Simulated Car Racing Championship @ CIG-2010
6177,81
4437,02
0
5881,37
6409,76
2736,39
2261,45
1971,52
0 1000 2000 3000 4000 5000 6000 7000
Autopia
Jorge Muñoz
Joseph Alton
Cardamone
COBOSTAR
MR. Racer
Neil Clarke
Timothy Alford
Qualifying: Rocky
2010 Simulated Car Racing Championship @ CIG-2010
Qualifying summary
Dirt-1 Olethros-dirt Rocky Total
Autopia10 10 8 28
Cardamone5 8 10 23
COBOSTAR6 6 6 18
Neil8 2 3 13
Mr. Racer4 3 5 12
Tim3 4 4 11
Jorge Muñoz1 5 2 8
Joseph Alton*2 1 1 4
Joseph Alton was disqualified from the race as crashes in 2 over 3 tracks
What about qualifying?
AUTOPIA is the fastest driver
Also, Cardamone (a very old driver ) did very well
For some drivers noise seems to affect significantly the performance
However, some controllers are able to reach almost the same performance even with noise
2010 Simulated Car Racing Championship @ CIG-2010
How much does noise affect the performance?
2010 Simulated Car Racing Championship @ CIG-2010
Dirt-1 with and without noise
6125,74
4851,54
2665,89
5340,37
5218,73
4168,25
5696,28
2291,56
6201,71
4966,6
2666,12
5404,24
5380,04
5097,69
6204,14
1707,26
0 1000 2000 3000 4000 5000 6000 7000
Autopia
Jorge Muñoz
Joseph Alton
Cardamone
COBOSTAR
MR. Racer
Neil Clarke
Timothy Alford
NO-NOISY NOISY
The Race
2010 Simulated Car Racing Championship @ CIG-2010
Three Tracks
For each track we run 7 races with different starting grids
Each race is scored using the F1 point system (10 to first, 8 to second, 6 to third, …)
Two points to the controller with lesser damage
Two points for the fastest lap of the race
43
2010 Simulated Car Racing Championship @ CIG-2010
Race: Dirt-1
Competitor Score
Cardamone 10
Autopia 8
COBOSTAR 8
Neil 7
Jorge Muñoz 4
Tim 4
Mr. Racer 3
2010 Simulated Car Racing Championship @ CIG-2010
Race: Olethros-dirt
Competitor Score
Autopia12
COBOSTAR8
Cardamone6
Tim6
Neil5
Mr. Racer4
Jorge Muñoz2
2010 Simulated Car Racing Championship @ CIG-2010
Race: Rocky
Competitor Score
COBOSTAR8
Cardamone8
Mr. Racer8
Jorge Muñoz6
Autopia5
Neil4
Tim3
2010 Simulated Car Racing Championship @ CIG-2010
Final Results
Competitor Dirt-1 Ole-Dirt Rocky Total
Autopia 8 12 5 25
Cardamone 10 6 8 24
COBOSTAR 8 8 8 24
Neil 7 5 4 16
Mr. Racer 3 4 8 15
Tim 4 6 3 13
Jorge Muñoz 4 2 6 12
2010 Simulated Car Racing Championship @ CIG-2010
Final Standings
Competitor GECCO WCCI CIG Total
AUTOPIA 34 24 25 83
COBOSTAR 14 28 24 66
Jorge Muñoz 22.5 28 12 62.5
Cardamone 16 16 24 56
Mr. Racer 16 9 15 40
Joseph Alton 15.5 11 - 26.5
Tim Alford - 10 13 23
Neil Clarke - - 16 16
2010 Simulated Car Racing Championship @ CIG-2010
Final Standings
Competitor GECCO WCCI CIG Total
AUTOPIA 34 24 25 83
COBOSTAR 14 28 24 66
Jorge Muñoz 22.5 28 12 62.5
Cardamone 16 16 24 56
Mr. Racer 16 9 15 40
Joseph Alton 15.5 11 - 26.5
Tim Alford - 10 13 23
Neil Clarke - - 16 16
2010 Simulated Car Racing Championship @ CIG-2010
The 2010 Simulated Car Racing Championship
GECCO-2010Portland , July 7-11American LegWinner: Autopia
WCCI-2010Barcelona, July 18-23Formula LegWinner: Jorge Muñoz
CIG-2010Copenhagen, August 18–21Dusty LegWinner: Autopia
2010 Simulated Car Racing Championship @ CIG-2010
2010 Simulated Car Racing Championship @ CIG-2010
Analysis of results
Competitor GECCO WCCI CIG
AUTOPIA 34 24 25
COBOSTAR 14 28 24
Jorge Muñoz 22.5 28 12
Cardamone 16 16 24
Mr. Racer 16 9 15
Joseph Alton 15.5 11 -
Tim Alford - 10 13
Neil Clarke - - 16
2010 Simulated Car Racing Championship @ CIG-2010
Analysis of results
1
2
3
4
5
6
7
8
GECCO WCCI CIG
AUTOPIA
COBOSTAR
Jorge Muñoz
Cardamone
Mr. Racer
2010 Simulated Car Racing Championship @ CIG-2010
What about new challenges?
Noisy sensors
Addressed by almost all the competitors
For most of the approaches is critical only for opponent management
Still an open problem
Warm-up stage
Addressed by almost all the competitors
Simple on-line adaptation (e.g., list of crash points implemented in COBOSTAR) still very competitive
We expect big improvements in this task in the future
Custom Rangefinders Setup and Clutch Control
2010 Simulated Car Racing Championship @ CIG-2010
Conclusions
Car Racing Community
Overall 6 competitors entered the competition (last year we got 11 entries)
Almost the same authors in the first positions
New competitors enter for a while and leave
Do not be scared
The source of entries will be available on-line
A lot of opportunities to improve the current controllers