1 Integrating Learning in Interactive Gaming Simulators Integrating Learning in Interactive Gaming Simulators David W. Aha 1 & Matthew Molineaux 2 1 Intelligent Decision Aids Group Navy Center for Applied Research in AI Naval Research Laboratory; Washington, DC 2 ITT Industries; AES Division; Alexandria, VA surname@aic.nrl.navy.mil AAAI’04 Workshop on Challenges in Game AI 25 July 2004
20
Embed
Integrating Learning in Interactive Gaming Simulators
Integrating Learning in Interactive Gaming Simulators. David W. Aha 1 & Matthew Molineaux 2 1 Intelligent Decision Aids Group Navy Center for Applied Research in AI Naval Research Laboratory; Washington, DC 2 ITT Industries; AES Division; Alexandria, VA surname @aic.nrl.navy.mil. - PowerPoint PPT Presentation
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1Integrating Learning in Interactive Gaming Simulators
Integrating Learning in Interactive Gaming Simulators
David W. Aha1 & Matthew Molineaux2
1Intelligent Decision Aids GroupNavy Center for Applied Research in AI
Naval Research Laboratory; Washington, DC2ITT Industries; AES Division; Alexandria, VA
AAAI’04 Workshop on Challenges in Game AI25 July 2004
2Integrating Learning in Interactive Gaming Simulators
Outline
1. Motivation: Learning in cognitive systems2. Objectives:
– Support empirical studies w/ simulators– Encourage studies that address industry
& military M&S concerns3. Design: TIELT functionality & components4. Example: Knowledge base content5. Status: Implementation, collaborations6. Summary
1. Motivation: Learning in cognitive systems2. Objectives:
– Support empirical studies w/ simulators– Encourage studies that address industry
& military M&S concerns3. Design: TIELT functionality & components4. Example: Knowledge base content5. Status: Implementation, collaborations6. Summary
Thanks to our sponsor:
3Integrating Learning in Interactive Gaming Simulators
Rough Anatomy of a Cognitive Agent
External Environment External Environment
Communication(language,gesture,image)
Prediction,planning
Deliberative Processes
Reflective Processes
Reactive Processes
Perception Action
STM
Sensors Effectors
Other reasoning
LTM
Concepts
SentencesCog
nit
ive
Ag
en
t
Affect
Attention
Learning
Learning
(Brachman, 2003)
4Integrating Learning in Interactive Gaming Simulators
Problem
Status of Cognitive Learning
Few deployed cognitive systems integrate techniques that exhibit rapid & enduring learning behavior on complex tasks
– It’s costly to integrate & evaluate embedded learning techniques
Few deployed cognitive systems integrate techniques that exhibit rapid & enduring learning behavior on complex tasks
– It’s costly to integrate & evaluate embedded learning techniques
Complication
The ML research community has been focusing on:¬Rapid: Knowledge poor algorithms
¬Enduring: Learning over a short time period
¬Embedded: Stand-alone evaluations
The ML research community has been focusing on:¬Rapid: Knowledge poor algorithms
¬Enduring: Learning over a short time period
¬Embedded: Stand-alone evaluations
5Integrating Learning in Interactive Gaming Simulators
Wanted: A New Interface (thanks to W. Cohen, others)
Supervised Learning ML
SystemDatabase Interface
(standard format)
(e.g., UCI Repository)
ReasoningSystem
Supervised Learning ML
SystemDatabase Interface
(standard format)
(e.g., UCI Repository)
ReasoningSystem
Supervised Learning ML
Systemj
DatabaseiInterface
(standard format)
(e.g., UCI Repository of ML Databases)
ReasoningSystemk
Cognitive Learning Reasoning Modules
Wor
ld(S
imu
late
d/R
ea
l)
Sensors ML Module
Interface(standard API)
ML Module
ML Module(e.g., TIELT)Effectors
Cognitive Learning Reasoning Modules
Wor
ld(S
imu
late
d/R
ea
l)
Sensors ML Module
Interface(standard API)
ML Module
ML Module(e.g., TIELT)Effectors
Cognitive Learning Reasoning Systemk
Wor
ldi
(Sim
ula
ted/
Re
al)
Sensors ML Module
Interface(standard API)
ML Module
ML Modulej
(e.g., TIELT)Effectors
Testbed for Integrating and Evaluating Learning TechniquesTestbed for Integrating and Evaluating Learning Techniques
6Integrating Learning in Interactive Gaming Simulators
Objectives & Domain
Objective
Facilitate the evaluation of learning techniques in CogSys1. Develop & transition TIELT to help reduce integration costs (time, $)2. Support DARPA Challenge Problems on Cognitive Learning3. Demonstrate research utility prior to approaching industry/military
Facilitate the evaluation of learning techniques in CogSys1. Develop & transition TIELT to help reduce integration costs (time, $)2. Support DARPA Challenge Problems on Cognitive Learning3. Demonstrate research utility prior to approaching industry/military
Domain: Why interactive gaming simulators?
1. Available implementations (cheap to acquire & run)2. Challenging problems for CogSys/ML research 3. Significant interest (military, industry, academia, funding, public)
1. Available implementations (cheap to acquire & run)2. Challenging problems for CogSys/ML research 3. Significant interest (military, industry, academia, funding, public)
7Integrating Learning in Interactive Gaming Simulators
TIELT Specification
1. Simplifies integration & evaluation!• Learning-embedded reasoning systems & gaming simulators• Inputs: 5 descriptions (simulator I/O, game model, learning &
performance tasks, reasoning system I/O, & evaluation strategy)• Free
2. Learning foci: Many• Task (e.g., learn how to execute, or advise on, a task)• Player (e.g., learn/predict a human player’s strategies)• Game (e.g., learn/refine its objects, their relations, & behaviors)
3. Learning methods: Many types• Supervised/unsupervised, immediate/delayed feedback, analytic,
active/passive, online/offline, direct/indirect, automated/interactive• Learning results should be available for inspection
4. Gaming simulators: Those with challenging learning tasks5. Reuse: Provide access to libraries of simulators & reasoning systems
• Abstracts interface definitions from game & task models
1. Simplifies integration & evaluation!• Learning-embedded reasoning systems & gaming simulators• Inputs: 5 descriptions (simulator I/O, game model, learning &
performance tasks, reasoning system I/O, & evaluation strategy)• Free
2. Learning foci: Many• Task (e.g., learn how to execute, or advise on, a task)• Player (e.g., learn/predict a human player’s strategies)• Game (e.g., learn/refine its objects, their relations, & behaviors)
3. Learning methods: Many types• Supervised/unsupervised, immediate/delayed feedback, analytic,
active/passive, online/offline, direct/indirect, automated/interactive• Learning results should be available for inspection
4. Gaming simulators: Those with challenging learning tasks5. Reuse: Provide access to libraries of simulators & reasoning systems
• Abstracts interface definitions from game & task models
8Integrating Learning in Interactive Gaming Simulators
Distinguishing TIELT
System Focus $ Game Engine(s)
Prominent Feature
Reasoning Activity
DirectIA (MASA)
AI SDK FPS, RTS, etc.
Behavior authoring Sense-act, …
SimBionic (SHAI)
AI SDK FPS, etc. Behavior authoring Sense-act, …
FEAR AI SDK Quake 2, etc. Behavior authoring Sense-act, …
RoboCup Research Testbed
RoboCup Soccer game play Sense-act, coaching, etc.
GameBots Research Testbed
UT (FPS) UT game play Sense-act
ORTS Research Testbed
RTS games Hack-free MM RTS Sense-act, strategy
TIELT Research Testbed
Several genres
Experimentation for Learning Systems
Sense-act, advice processing, prediction, model updating, etc.
1. Provides an interface for message-passing interfaces2. Supports composable system-level interfaces
9Integrating Learning in Interactive Gaming Simulators
10Integrating Learning in Interactive Gaming Simulators
TIELT’s Knowledge Bases
GameModel
Description
Task Descriptions
GameInterface
Description
ReasoningInterface
Description
EvaluationMethodology Description
Defines communication processes with the game engine
Defines communication processes with the learning system
Defines interpretation of the game• e.g., initial state, classes, operators, behaviors (rules)• Behaviors could be used to provide constraints on learning
Defines the selected learning and performance tasks• Selected from the game model description
Defines the empirical evaluation to conduct
11Integrating Learning in Interactive Gaming Simulators
13Integrating Learning in Interactive Gaming Simulators
SelectedGame
Engine
EvaluationEditor
Game InterfaceEditor
Percepts
User
ReasoningInterface Editor
Game ModelEditor
TaskEditor
GameModel
Description
TaskDescriptions
Pe
rf.
Ta
sk
EvaluationInterface
Evaluator
Action / Control
Translator(Mapper)
Learning OutputsActions
Translated Model (Subset)
Learning Task
GameInterface
Description
ReasoningInterface
Description
LearningTranslator(Mapper)
CurrentState
ModelUpdater
Database
EvaluationMethodologyDescription
StoredState
AdviceInterface Database
EngineState
Controller
SelectedReasoning
System
TIELT’s Internal Communication Modules
14Integrating Learning in Interactive Gaming Simulators
Sensing the Game State(City placement example)
TIELT
Game Interface
Editor
Sensors
User
Game ModelEditor
GameModel
Description
Up
da
tes
GameInterface
Description
ActionTranslator
Actions
GameEngine
GameEngine
CurrentState
1
2
4 3
4
In Game Engine, the game begins; a colony pod is created and placed.
1
The Game Engine sends a “See” sensor message identifying the pod’s location.
This message template provides updates (instructions) to the Current State, telling it that there is a pod at the location See describes.
4
2
The Model Updater receives the sensor message and finds the corresponding message template in the Game Interface Description.
3
ControllerModel
Updater
3
The Model Updater notifies the Controller that the See action event has occurred.
5
5
15Integrating Learning in Interactive Gaming Simulators
Getting Decisions from the Learning System(City placement example)
TIELT
SelectedReasoning
System
SelectedReasoning
System
LearningModule #1
LearningModule #n
. . .
User
Learning InterfaceEditor
AgentEditor
TaskDescriptions
LearningTranslator
Translated Model (Subset)
ReasoningInterface
Description
ActionTranslator
LearningOutputs
The Controller notifies the Learning Translator that it has received a See message.
The Learning Translator finds a city location task, which is triggered by the See message. It queries the controller for the learning mode, then creates a TestInput message to send to the reasoning system with information on the pod’s location and the map from the Current State.
The Reasoning System transmits output to the Action Translator.
The Learning Translator transmits the TestInput message to the Reasoning System.
1
2 23
4
Controller
CurrentState
1
4
2
3
16Integrating Learning in Interactive Gaming Simulators
TIELT
Game InterfaceEditor
User
ActionTranslator
Actions
GameEngine
GameEngine
1
2
4.a
The Action Translator receives a TestOutput message from the Reasoning System.
The Action Translator finds the TestOutput message template, determines it is associated with the city location task, and builds a MovePod operator (defined by the Current State) with the parameters of TestOutput.
The Game Engine receives Move and updates the game to move the pod toward its destination, or
The Action Translator determines that the Move Action from the Game Interface Description is triggered by the MovePod Operator and binds Move using information from MovePod.
ReasoningInterface Editor
3
GameInterface
Description
ReasoningInterface
Description
AdviceInterface
The Advice Interface receives Move and displays advice to a human player on what to do next, or makes a Prediction.
4.b, c1
4.a
2
3
Acting in the Game World(City placement example)
4.b
CurrentState
2
PredictionInterface
4.c
3
17Integrating Learning in Interactive Gaming Simulators
Initial Work
Status (July 2004)
• TIELT v1 + documentation due 9/04– Message protocols