Toward a Large-Scale Model of Language Comprehension in ACT-R 6 July 2007 Jerry Ball 1 , Andrea Heiberg 2 , and Ronnie Silber 3 Air Force Research Laboratory 1 L3 Communications 2 Lockheed Martin 3
Dec 14, 2015
Toward a Large-Scale Model of Language Comprehension in ACT-R 6
July 2007
Jerry Ball1, Andrea Heiberg2,
and Ronnie Silber3
Air Force Research Laboratory1
L3 Communications2
Lockheed Martin3
2
The VERBOSE Project
• Verbalization Between Operators and Synthetic Entities
• Synthetic Teammate
–Air Vehicle Operator (AVO)
• Flies Simulated Unmanned Aerial Vehicle (UAV)
• Communicates with other team members
–Language Capable
• Comprehension
• Generation
• Dialog Management
3
Chat
Output
Language
Comprehension
Language
Generation
Dialog
Manager
ACT-R Architecture
Motor ModelMotor
Actions
Situation ModelVisual
Input
Chat
Input
VERBOSE System Overview
4
• Unmanned Aerial Vehicle (UAV) Simulation
• Three-Person Team Performing Reconnaissance Missions
– Air Vehicle Operator (AVO)
• Flies UAV
– Payload Operator (PLO)
• Takes pictures of targets
– Data Exploitation Mission Planning & Communications (DEMPC) Officer
• Plans mission
Cognitive Engineering Research on Team Tasks (CERTT) Lab
5
AVODEMPCPLO
Experimenter Station
Cognitive Engineering Research on Team Tasks (CERTT) Lab
6
NextWaypoints
AltitudeAir Speed
To GoalGear and
Flaps Request Flight Plan
Artificial horizon and
course
AVO Screen 1
7
FuelTime
Autopilot
Moving Map
Alarms
AVO Screen 2
8
AVO Chat Screen
9
Theoretical FoundationsLanguage Representation and Processing
• Double R Grammar
– Cognitive Linguistic theory of the grammatical encoding of referential and relational meaning
• Double R Process
– Psycholinguistic theory of the processing of English text into Double R Grammar based representations
• Double R Model
– Computational implementation using the ACT-R cognitive architecture and modeling environment
www.DoubleRTheory.com
10
Empirical Validation of Linguistic Representations
• Empirical study to determine the kinds of linguistic representations that humans construct during language comprehension
– What is/are the basic structure(s) of the clause?• Subject Predicate
• Subject Verb Object
• Other
– Methodology• Ask subjects to group words in expressions into
meaningful units
• Assumes subjects have explicit knowledge of the meaningful groups that occur in expressions
the man bit the dog
11
Empirical Study – Preliminary Results
• A pilot study was conducted
• Results were compared to an early study by Levelt which used multidimensional scaling:
– Levelt, W. (1970). A scaling approach to the study of syntactic relations. In G. B. Flores d’Arcais and W. Levelt (eds.) Advances in psycholinguistics 109-121. NY: Elsevier.
12
Levelt’s Early Study
the boy has lost a dollar
60
80
100
Relative frequency of “more related”
in triadic comparison
%
Connectedness
method
13
Pilot Study vs. Levelt Study
boy has lost a dollarthe
boy has lost a dollarthe
Levelt Study
Pilot Study
14
Double R Model
15
Levelt’s Early Study
Jan
(John)
eet
(eats)
Piet
(Peter) eet
(eats)
en
(and) peren
(pears)
3.0
5.0
7.0
Mean scale value
(7-point rating scale)
Connectedness
method
appels
(apples)
16
Pilot Study vs. Levelt Study
apples and Peter eats pearsJohn eats
Levelt Study
Pilot Study
appels en Piet eet perenJan eet
17
Double R Model
18
Start with Scripted Communication
• AVO: DEMPC this is AVO, please let me know the first waypoint!
• DEMPC: The first waypoint is LVN. It’s an entry point. There are no airspeed or altitude restrictions. The effective radius is 2.5 miles.
• AVO: PLO, I’m heading towards LVN.
• DEMPC: We’re within the effective radius so go to the second waypoint.
• AVO: Are there any altitude or airspeed restrictions for the second waypoint?
• DEMPC: The second waypoint is H-AREA. It’s a target. The airspeed restriction is between 50 and 200 knots. There is no altitude restriction. The effective radius is 5 miles.
• PLO: AVO, please keep the altitude over 3000 feet for the photo!
• PLO: I have a good photo of H-AREA.
• AVO: Moving on to the next waypoint.
19
Double R Model
20
Double R Model
21
Double R Model
22
Double R Model
23
Double R Model