NTL – Converging Constraints • Basic concepts and words derive their meaning from embodied experience. • Abstract and theoretical concepts derive their meaning from metaphorical maps to more basic embodied concepts. • Structured Connectionist Models can capture both of these processes nicely. • Grammar extends this by Constructions: pairings of form with embodied meaning.
NTL – Converging Constraints. Basic concepts and words derive their meaning from embodied experience. Abstract and theoretical concepts derive their meaning from metaphorical maps to more basic embodied concepts. Structured Connectionist Models can capture both of these processes nicely. - PowerPoint PPT Presentation
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
NTL – Converging Constraints
• Basic concepts and words derive their meaning from embodied experience.
• Abstract and theoretical concepts derive their meaning from metaphorical maps to more basic embodied concepts.
• Structured Connectionist Models can capture both of these processes nicely.
• Grammar extends this by Constructions: pairings of form with embodied meaning.
Simulation-based language understanding
“Harry walked to the cafe.”
Schema Trajector Goalwalk Harry cafe
Analysis Process
Simulation Specification
Utterance
SimulationCafe
Constructions
General Knowledge
Belief State
19Konvens, 09.10.2000
The ICSI/BerkeleyNeural Theory of Language Project
Background: Primate Motor Control• Relevant requirements (Stromberg, Latash, Kandel, Arbib,
Jeannerod, Rizzolatti)– Should model coordinated, distributed, parameterized control
programs required for motor action and perception.– Should be an active structure.– Should be able to model concurrent actions and interrupts.
• Model– The NTL project has developed a computational model based on that
satisfies these requirements (x- schemas).– Details, papers, etc. can be obtained on the web at
http://www.icsi.berkeley.edu/NTL
Active representations• Many inferences about actions derive from what we know
about executing them
• Representation based on stochastic Petri nets captures dynamic, parameterized nature of actions
Walking:
bound to a specific walker with a direction or goal
consumes resources (e.g., energy)may have termination condition
(e.g., walker at goal) ongoing, iterative action
walker=Harry
goal=home
energy
walker at goal
Somatotopy of Action ObservationSomatotopy of Action Observation
Foot ActionFoot Action
Hand ActionHand Action
Mouth ActionMouth Action
Buccino et al. Eur J Neurosci 2001
Active Motion Model
Evolving Responses of Competing Models over Time.
Nigel Goddard
1989
Language Development in Children
• 0-3 mo: prefers sounds in native language• 3-6 mo: imitation of vowel sounds only• 6-8 mo: babbling in consonant-vowel segments• 8-10 mo: word comprehension, starts to lose sensitivity to
consonants outside native language• 12-13 mo: word production (naming)• 16-20 mo: word combinations, relational words (verbs,
A model of children learning their first verbs.Assumes parent labels child’s actions.Child knows parameters of action, associates with
wordProgram learns well enough to: 1) Label novel actions correctly 2) Obey commands using new words (simulation)System works across languagesMechanisms are neurally plausible.
Motor Control (X-schema) for SLIDE
Parameters for the SLIDE X-schema
Feature Structures for PUSH
System Overview
Learning Two Senses of PUSH
Model merging based on Bayesian MDL
Training ResultsDavid Bailey
English• 165 Training Examples, 18 verbs• Learns optimal number of word senses (21)• 32 Test examples : 78% recognition, 81% action• All mistakes were close lift ~ yank, etc.• Learned some particle CXN,e.g., pull up
Farsi • With identical settings, learned senses not in English
Learning Two Senses of PUSH
Model merging based on Bayesian MDL
physics lowest energy state
chemistry molecular
minima
biology fitness, MEU
neuroeconomics
vision threats,
friends
language errors,
NTL
Constrained Best Fit in Nature
inanimate animate
Model Merging and Recruitment
Word Learning requires “fast mapping”.
Recruitment Learning is a Connectionist
Level model of this.
Model Merging is a practical Computational
Level method for fast mapping.
Bailey’s thesis outlines the reduction and some versions have been built.
The full story requires Bayesian MDL, later.
The Idea of Recruitment Learning
• Suppose we want to link up node X to node Y
• The idea is to pick the two nodes in the middle to link them up
• Can we be sure that we can find a path to get from X to Y?
KBFP )1(link no the point is, with a fan-out of 1000, if we allow 2 intermediate layers, we can almost always find a path
XX
YY
BBNN
KK
F = B/NF = B/N
Recruiting triangle nodes• Let’s say we are trying to remember a green circle
• currently weak connections between concepts (dotted lines)