A Computational Model of Staged Language Acquisition JACK, Kris DRT/FAR/LIST/DTSI/SRCI/LIC2M [email protected]
May 28, 2015
A ComputationalModel of Staged
Language Acquisition
JACK, KrisDRT/FAR/LIST/DTSI/SRCI/LIC2M
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Introduction
● Children appear to acquire language effortlessly
● They do not, however, do so overnight● Typically, they progress through stages of
linguistic development● Computational modelling can help us to better
understand the language acquisition process by estimating the problem and developing possible solutions
● A computational model that tackles such staged linguistic development is absent from current literature
● Overview of presentation
IntroductionChild LanguageModelsLATTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Stages in Language Acquisition
● How does child language typically develop?
● Language acquisition is consistently described in stages (e.g. Brown, Pinker, Tomasello)
● Five stages from birth to 48 months:
● The Pre-linguistic Stage● The Holophrastic Stage● The Early Multi-word Stage● The Late Multi-word Stage● The Abstract Stage
6 12 18 24 30 360 42 48
Time (months)
Pre-linguisticHolophrastic
Early Multi-wordLate Multi-word
Abstract
IntroductionChild Language➢ Pre-linguistic➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractModelsLATTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
The Pre-linguistic Stage (1/2)
Little activity typically characterised as linguistic
Mini-stages including reflexive vocalisations, cooing, vocal play and babbling
IntroductionChild Language➢ Pre-linguistic➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractModelsLATTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
The Pre-linguistic Stage (2/2)
Can differentiate between languages across rhythmic families of stress-timed, syllable-timed or mora-timed (Mehler et al., 1996)
Sensitive to transitional probabilities within syllable sequences (Saffran et al., 1996)
IntroductionChild Language➢ Pre-linguistic➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractModelsLATTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
The Holophrastic Stage (1/2)
Begin when children achieve joint attention (Tomasello, 1995)
First utterances are typically holophrastic
Holistic or atomic units (e.g. “mummy”, “doggy”)
Even seemingly multi-word utterances are holistic (e.g. “I-wanna-do-it” (Pine and Lieven, 1993))
IntroductionChild Language➢ Pre-linguistic➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractModelsLATTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
The Holophrastic Stage (2/2)
Child and adult meanings for a holophrase often differ resulting in:
Underextensions (Reich, 1986)
Overextensions (Barrett, 1978)
Mismatches (Rodgon, 1976)
IntroductionChild Language➢ Pre-linguistic➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractModelsLATTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
The Early Multi-word Stage
Children regularly combine words to produce multi-word utterances
Novel combinations are made (e.g. “allgone sticky” (Braine, 1971))
Many utterances can be described using a pivot grammar (P)ivot (O)pen, O P, O O (Braine, 1963)
E.g. S = P O, where O = “mummy”, “sticky”, “duck”, “red” and P = “allgone”
Children are not sensitive to word-order (Clark, 1975; MacWhinney, 1980; de Villiers and de Villiers, 1978)
Function words and morphological markings tend to be omitted (Hyams, 1986)
IntroductionChild Language➢ Pre-linguistic➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractModelsLATTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
The Late Multi-word Stage
Child language becomes increasingly complex
The emergence of syntactic awareness
In English, word-order can define participant roles e.g. “Make the doggie bite the cat” (de Villiers and de Villiers, 1973)
Children are found to have an irregular, item-based, knowledge of language:
Verb islands (Tomasello, 1992)
Inconsistent use of determiners (Pine and Lieven, 1997)
IntroductionChild Language➢ Pre-linguistic➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractModelsLATTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
The Abstract Stage
Evidence of all of the grammatical machinery found in adults (Pinker, 1994)
The item-specific quality of child language gives way to a more abstract quality (Tomasello, 2003)
Strong generative capacity asserts itself
6 12 18 24 30 360 42 48
Time (months)
Pre-linguisticHolophrastic
Early Multi-wordLate Multi-word
Abstract
IntroductionChild Language➢ Pre-linguistic➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractModelsLATTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Stages Summary
● Pre-linguistic Stage
– little real linguistic activity
● Holophrastic Stage
– first words
● Early Multi-word Stage
– first word combinations
● Late Multi-word Stage
– word combinations with syntax
● Abstract Stage
– strong generative capacity
6 12 18 24 30 360 42 48
Time (months)
Pre-linguisticHolophrastic
Early Multi-wordLate Multi-word
Abstract
IntroductionChild Language➢ Pre-linguistic➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractModelsLATTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Goals
● What triggers the emergence of each stage?
● What accounts for the linguistic shape of each stage?
● We can produce computational models that estimate learning tasks faced by children to help us better understand the problem
IntroductionChild Language➢ Pre-linguistic➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractModelsLATTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Language Models (1/2)
Many computational models have been produced to study language learning
The Miniature Language Acquisition Paradigm (Feldman et al., 1990)
Place a computational model in an environment with access to visual and acoustic stimuli (simulated or grounded)
Train the model by providing descriptions of visually-based scenes from a miniature language (e.g. “the red square is on top of the green circle”)
The model is said to have acquired the language when it can both comprehend and produce all sentences within the miniature language
IntroductionChild LanguageModels➢ Data Filtering➢ Incremental➢ IteratedLATTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Language Models (2/2)
A computational model that demonstrates stage transitions from the Pre-linguistic Stage to the Abstract Stage is missing from current literature
Some models demonstrate stage-like learning by:
Externally modifying the training data
Modifying the functionality of the model
Internally modifying the training data
IntroductionChild LanguageModels➢ Data Filtering➢ Incremental➢ IteratedLATTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Temporal Data Filtering
Training data are modified during training (e.g. Elman, 1993; Roy, 2002)
Elman (1993) trained a neural network to acquire sentences with both short and long distance dependencies
Success only when the training data were biased towards including more examples of short distance dependencies in early learning and long distance dependencies in late learning
Unrealistic assumption
Although infant direct speech contains different characteristics to adult directed speech, children are still exposed to complex sentences from birth
DataData
DataData
DataData
ModelModelx < t <= y
0 < t <= x
y < t <= z
IntroductionChild LanguageModels➢ Data Filtering➢ Incremental➢ IteratedLATTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Incremental Learning
Functionality is modified during training (Elman, 1993; Dominey and Boucher, 2005)
Elman (1993) trained a neural network to acquire sentences with both short and long distance dependencies
The neural network’s ‘short-term memory’ was incrementally increased during learning, allowing the network to acquire both types of dependencies
Incremental learning has problems:
When should increments be made?
What unit of data should be restricted, syllables
Transitions are not clean and clear
DataData ModelModelt > x
t > 0
t > y
ModuleModule
ModuleModule
ModuleModule
IntroductionChild LanguageModels➢ Data Filtering➢ Incremental➢ IteratedLATTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Iterated Learning
Training data are modified by model
In modelling language evolution, Kirby (2002) shows that learning through cultural transmission can modify the structure of a language
One generation of agents learn a language and then produce a progressively more structured language for teaching to the next generation (Iterated Learning)
However, this is language evolution, not language acquisition
Training data are constant for children
DataData
ModelModel
DataData
ModelModel
DataData
ModelModelModelModel
IntroductionChild LanguageModels➢ Data Filtering➢ Incremental➢ IteratedLATTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Problem
Can we model the stages of language acquisition when:
The functionality of the model is kept constant AND
The training data provided to the model are constant?
IntroductionChild LanguageModels➢ Data Filtering➢ Incremental➢ IteratedLATTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Language Acquisition Toolkit
LAT provides a framework for investigating staged language acquisition
Aim:
Develop a computational model that demonstrates realistic stages of linguistic development
Investigating the problem within a Miniature Language Acquisition Framework where:
The language contains enough complexity to allow the model to demonstrate stages of development
The language is not so complex that it cannot be learned in entirety
Concentrating on comprehension
6 12 18 24 30 360 42 48
Time (months)
Pre-linguisticHolophrastic
Early Multi-wordLate Multi-word
Abstract
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
LAT General Architecture
Environment(real or
simulated)
l1 R esources
renv
r1
r2
.
.
.rx
l2
r1
renv
r2
r1
.
.
.
l3
r3
r2
lx
rx
rx1
c2
r2
c1
r1
c3
r3
cx
rx
.
.
.
Learn ingM odules
C o m prehens ionM odules
WorldPerception Module
renv
Sensory stimuli
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
LAT Architecture Instantiated
Environment(real or
simulated)
lcrosssit R esources
renv
rcrosssit
rholophrastic
rearly
rlate
rabstract
lholophrastic
rcrosssit
renv
rholophrastic
rcrosssit
learly
rearly
rholophrastic
cearly
rearly
cholophrastic
rholophrastic
clate
rlate
cabstract
rabstract
Learn ingM odules
C o m prehens ionM odules
WorldPerception Module
renv
Sensory stimuli
llate
rlate
rholophrastic
labstract
rabstract
rlate
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Learning Environment
The model plays the Scene Building Game (Jack, 2005)
Algorithm:
1.The model watches a scene containing a single geometric object
2.Another geometric object is added to the scene and the event is described
3.Return to 1. Notice that the landmark object is described using the
definite article “the” and the new object is described using the indefinite article “a”
a blue circle below the red square
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
The World Perception Module
The World Perception Module encodes events (renv
)
Simulated Visual input – detects colour, shape and relative positions
{below(rel), blue(2), circle(2), horizontal_even(rel), red(1), square(1)}
Simulated Acoustic input – event description is perceived as a sequence of syllables
a blue cir cle be low the red square
Joint attention is assumed from the outset
6 12 18 24 30 360 42 48
Time (months)
Pre-linguisticHolophrastic
Early Multi-wordLate Multi-word
Abstract
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Cross-situational Learning Module
Aims
Find similarities between observed events
Derive possible form-meaning pairs
Create new resource rcross-sit
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Cross-situational Learning
Method
Form of Cross-situational Analysis (Siskind, 1996)
Words co-occur more often with their intended meanings than with other meanings
Example
Equal string parts are found Equal feature value parts are found New extensions are derived
a blue cir cle be low the red square
{below(rel), blue(2), circle(2), horizontal_even(rel), red(1), square(1)}
a green star to the low er right of the blue tri ang gle
{below(rel), blue(1), green(2), right(rel), star(2), triangle(1)}
1)
2)
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Cross-situational Learning
Method
Form of Cross-situational Analysis (Siskind, 1996)
Words co-occur more often with their intended meanings than with other meanings
Example
Equal string parts are found Equal feature value parts are found New extensions are derived
a blue cir cle be low the red square
{below(rel), blue(2), circle(2), horizontal_even(rel), red(1), square(1)}
a green star to the low er right of the blue tri ang gle
{below(rel), blue(1), green(2), right(rel), star(2), triangle(1)}
1)
2)
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Cross-situational Learning
Method
Form of Cross-situational Analysis (Siskind, 1996)
Words co-occur more often with their intended meanings than with other meanings
Example
Equal string parts are found Equal feature value parts are found New extensions are derived
a blue cir cle be low the red square
{below(rel), blue(2), circle(2), horizontal_even(rel), red(1), square(1)}
a green star to the low er right of the blue tri ang gle
{below(rel), blue(1), green(2), right(rel), star(2), triangle(1)}
1)
2)
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Cross-situational Learning
Method
Form of Cross-situational Analysis (Siskind, 1996)
Words co-occur more often with their intended meanings than with other meanings
Example
Equal string parts are found Equal feature value parts are found New extensions are derived
a blue cir cle be low the red square
{below(rel), blue(2), circle(2), horizontal_even(rel), red(1), square(1)}
a green star to the low er right of the blue tri ang gle
{below(rel), blue(1), green(2), right(rel), star(2), triangle(1)}
a {below(rel), blue(1)}
a {below(rel), blue(2)}
the {below(rel), blue(1)}
the {below(rel), blue(2)}
blue {below(rel), blue(1)}
blue {below(rel), blue(2)}
1)
2)
rcross-sit
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Holophrastic Learning Module
Aims
Reduce ambiguity by removing homonyms
Reduce ambiguity by removing synonyms
Create new resource rholophrastic
blue {blue(1,2)}red {red(1,2)}green {green(1,2)}square {square(1,2)}cir cle {circle(1,2)}tri ang gle {triangle(1,2)}be low {below(rel), vertical_even(rel)}a bove {above(rel), vertical_even(rel)}blue square {blue(1,2), square(1,2)}blue cir cle {blue(1,2), circle(1,2)}
.
.
.
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Holophrastic Learning Module
In order to remove homonyms and synonyms
1. create abstract extensions
3. keep only the most similar meaning for each form using
3. erase meanings of all extensions that have similarities lower than other extensions with the same meaning, where similarity is
(1) red {red(1)}
is merged with
(2) red {red(2)}
to produce
(3) red {red(1, 2)}
Similarity M i , F j=Frequency F j ,M i
Frequency F j
Similarity F i , M j=Frequency M j , Fi
Frequency M j
rholophrastic
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Holophrastic Comprehension Module
Comprehension:
Given a string to comprehend, the model searches r
holophrastic for extensions that contain the string
From those found, the meaning of the extension that is most similar is returned
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Early Multi-word Learning Module
Aims
To find compositional relationships between form-meaning pairs in r
holophrastic
show no sensitivity to word order nor object roles
Create new resource rearly
blue{blue(1,2)}
cir cle{circle(1,2)}
blue cir cle{blue(1,2), circle(1,2)}
green{green(1,2)}
cir cle{circle(1,2)}
green cir cle{circle(1,2), green(1,2)}
1) 2)
the{}
red cir cle{circle(1,2), red(1,2)}
the red cir cle{circle(1), red(1)}
3)
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Early Multi-word Learning Module
Finding compositionality
an extension is a function of two other extensions when its
form is equal to the concatenation of the forms of the parts (ignoring word order)
meaning is equal to the feature set union of the parts (ignoring object roles)
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Early Multi-word Learning Module
Finding compositionality
an extension is a function of two other extensions when its
form is equal to the concatenation of the forms of the parts (ignoring word order)
meaning is equal to the feature set union of the parts (ignoring object roles)
do extensions 1) 2) and 3) express a compositional grammar fragment?
blue{blue(1,2)}
cir cle{circle(1,2)}
blue cir cle{blue(1,2), circle(1,2)}
1) 2) 3)
blue cir cle = blue + cir cle ?
blue cir cle = cir cle + blue ?
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Early Multi-word Learning Module
Finding compositionality
an extension is a function of two other extensions when its
form is equal to the concatenation of the forms of the parts (ignoring word order)
meaning is equal to the feature set union of the parts (ignoring object roles)
do extensions 1) 2) and 3) express a compositional grammar fragment?
blue{blue(1,2)}
cir cle{circle(1,2)}
blue cir cle{blue(1,2), circle(1,2)}
1) 2) 3)
{blue(1,2), circle(1,2)} = {blue(1,2)} U {circle(1,2)} ?
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Early Multi-word Learning Module
Finding compositionality
an extension is a function of two other extensions when its
form is equal to the concatenation of the forms of the parts (ignoring word order)
meaning is equal to the feature set union of the parts (ignoring object roles)
do extensions 1) 2) and 3) express a compositional grammar fragment?
blue{blue(1,2)}
cir cle{circle(1,2)}
blue cir cle{blue(1,2), circle(1,2)}
1)
2) 3) cir cle{circle(1,2)}
blue{blue(1,2)}
blue cir cle{blue(1,2), circle(1,2)}
1)
3) 2)
OR
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Early Multi-word Learning Module
Finding compositionality
To reflect child sensitivity during this period, each grammar fragment must have a part that appears in another fragment
This produces a form of pivot grammar where pivot parts can appear with many open parts (Braine, 1963)
blue{blue(1,2)}
cir cle{circle(1,2)}
blue cir cle{blue(1,2), circle(1,2)}
green{green(1,2)}
cir cle{circle(1,2)}
green cir cle{circle(1,2), green(1,2)}
1) 2)
the{}
red cir cle{circle(1,2), red(1,2)}
the red cir cle{circle(1), red(1)}
3)
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Early Multi-word Learning Module
Finding compositionality
To reflect child sensitivity during this period, each grammar fragment must have a part that appears in another fragment
This produces a form of pivot grammar where pivot parts can appear with many open parts (Braine, 1963)
blue{blue(1,2)}
cir cle{circle(1,2)}
blue cir cle{blue(1,2), circle(1,2)}
green{green(1,2)}
cir cle{circle(1,2)}
green cir cle{circle(1,2), green(1,2)}
1)
3)
2)
the{}
red cir cle{circle(1,2), red(1,2)}
the red cir cle{circle(1), red(1)}
3)
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Early Multi-word Learning Module
Finding compositionality
To reflect child sensitivity during this period, each grammar fragment must have a part that appears in another fragment
This produces a form of pivot grammar where pivot parts can appear with many open parts (Braine, 1963)
blue{blue(1,2)}
cir cle{circle(1,2)}
blue cir cle{blue(1,2), circle(1,2)}
green{green(1,2)}
cir cle{circle(1,2)}
green cir cle{circle(1,2), green(1,2)}
1) 2)
rearly
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Early Multi-word Comprehension Module
Comprehension:
Given a string to comprehend, the model searches rearly
for grammar fragments whose parts can be combined to make the string
For each grammar fragment found, its meanings are combined through union and each result is returned
blue{blue(1,2)}
cir cle{circle(1,2)}
blue cir cle{blue(1,2), circle(1,2)}
{blue(1,2)} U {circle(1,2)} = {blue(1,2), circle(1,2)}
comprehend blue cir cle
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Early Multi-word Comprehension Module
Comprehension:
Given a string to comprehend, the model searches rearly
for grammar fragments whose parts can be combined to make the string
For each grammar fragment found, its meanings are combined through union and each result is returned
blue{blue(1,2)}
cir cle{circle(1,2)}
blue cir cle{blue(1,2), circle(1,2)}
{blue(1,2)} U {circle(1,2)} = {blue(1,2), circle(1,2)}
comprehend cir cle blue
however...
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Early Multi-word Comprehension Module
Comprehension:
Given a string to comprehend, the model searches rearly
for grammar fragments whose parts can be combined to make the string
For each grammar fragment found, its meanings are combined through union and each result is returned
the{}
red cir cle{circle(1,2), red(1,2)}
the red cir cle{circle(1), red(1)}
however...
{} U {circle(1,2), red(1,2)} = {circle(1,2), red(1,2)}
comprehend the red cir cle
a blue cir cle be low the red square
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Late Multi-word Learning Module
Aims
To find compositional relationships between form-meaning pairs in r
holophrastic
show sensitivity to word order and object roles
Create new resource rlate
blue{blue(1,2)}
cir cle{circle(1,2)}
blue cir cle{blue(1,2), circle(1,2)}
((1,2)>(1,2))((1,2)>(1,2))
1)
the{}
blue cir cle{blue(1,2), circle(1,2)}
the blue cir cle{blue(1), circle(1)}
((1,2)>(1))()
2)
a{}
blue cir cle{circle(1,2), blue(1,2)}
a blue cir cle{blue(2), circle(2)}
((1,2)>(2))()
3)
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Late Multi-word Learning Module
Finding compositionality
an extension is a function of two other extensions when its
form is equal to the concatenation of the forms of the parts (consider word order)
meaning is equal to the feature set union of the parts, after transfomation (consider object roles)
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Late Multi-word Learning Module
Finding compositionality
an extension is a function of two other extensions when its
form is equal to the concatenation of the forms of the parts (consider word order)
meaning is equal to the feature set union of the parts, after transfomation (consider object roles)
do extensions 1) 2) and 3) express a compositional grammar fragment?
the{}
blue cir cle{blue(1,2), circle(1,2)}
the blue cir cle{blue(1), circle(1)}
1) 2) 3)
blue cir cle = blue + cir cle ?
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Late Multi-word Learning Module
Finding compositionality
an extension is a function of two other extensions when its
form is equal to the concatenation of the forms of the parts (consider word order)
meaning is equal to the feature set union of the parts, after transfomation (consider object roles)
do extensions 1) 2) and 3) express a compositional grammar fragment?
{blue(1), circle(1)} = T({}, ()) U T({blue(1,2), circle(1,2)}, ((1,2)>(1))) ?
the{}
blue cir cle{blue(1,2), circle(1,2)}
the blue cir cle{blue(1), circle(1)}
1) 2) 3)
i.e. {blue(1), circle(1)} = {} U {blue(1), circle(1)} ?
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Late Multi-word Learning Module
Finding compositionality
an extension is a function of two other extensions when its
form is equal to the concatenation of the forms of the parts (consider word order)
meaning is equal to the feature set union of the parts, after transfomation (consider object roles)
do extensions 1) 2) and 3) express a compositional grammar fragment?
the{}
blue cir cle{blue(1,2), circle(1,2)}
the blue cir cle{blue(1), circle(1)}
((1,2)>(1))()
1)
2) 3)
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Late Multi-word Learning Module
Finding compositionality
Each grammar fragment must have a part that appears in another fragment
They appears on the same side (same word order)
AND the transformations are the same (same object roles)
1) 2)
the{}
blue cir cle{blue(1,2), circle(1,2)}
the blue cir cle{blue(1), circle(1)}
((1,2)>(1))()
the{}
blue square{blue(1,2), square(1,2)}
the blue square{blue(1), square(1)}
((1,2)>(1))()
3)
a{}
red cir cle{circle(1,2), red(1,2)}
a red cir cle{circle(2), red(2)}
((1,2)>(2))()
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Late Multi-word Learning Module
Finding compositionality
Each grammar fragment must have a part that appears in another fragment
They appears on the same side (same word order)
AND the transformations are the same (same object roles)
1) 2)
the{}
blue cir cle{blue(1,2), circle(1,2)}
the blue cir cle{blue(1), circle(1)}
((1,2)>(1))()
the{}
blue square{blue(1,2), square(1,2)}
the blue square{blue(1), square(1)}
((1,2)>(1))()
3)
a{}
red cir cle{circle(1,2), red(1,2)}
a red cir cle{circle(2), red(2)}
((1,2)>(2))()
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Late Multi-word Learning Module
Finding compositionality
Each grammar fragment must have a part that appears in another fragment
They appears on the same side (same word order)
AND the transformations are the same (same object roles)
rlate
1) 2)
the{}
blue cir cle{blue(1,2), circle(1,2)}
the blue cir cle{blue(1), circle(1)}
((1,2)>(1))()
the{}
blue square{blue(1,2), square(1,2)}
the blue square{blue(1), square(1)}
((1,2)>(1))()
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Late Multi-word Comprehension Module
Comprehension:
Given a string to comprehend, the model searches rlate
for grammar fragments whose parts can be combined to make the string
For each grammar fragment found, its meanings are mapped and then combined through union and each result is returned
comprehend blue cir cle
blue{blue(1,2)}
cir cle{circle(1,2)}
blue cir cle{blue(1,2), circle(1,2)}
((1,2)>(1,2))((1,2)>(1,2))
T({blue(1,2)},((1,2)>(1,2))) UT({circle(1,2)},((1,2)>(1,2))) = {blue(1,2), circle(1,2)}
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Late Multi-word Comprehension Module
Comprehension:
Given a string to comprehend, the model searches rlate
for grammar fragments whose parts can be combined to make the string
For each grammar fragment found, its meanings are mapped and then combined through union and each result is returned
comprehend cir cle blue
blue{blue(1,2)}
cir cle{circle(1,2)}
blue cir cle{blue(1,2), circle(1,2)}
((1,2)>(1,2))((1,2)>(1,2))
Meaning not found
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Late Multi-word Comprehension Module
Comprehension:
Given a string to comprehend, the model searches rlate
for grammar fragments whose parts can be combined to make the string
For each grammar fragment found, its meanings are mapped and then combined through union and each result is returned
comprehend a blue cir cle
a{}
blue cir cle{blue(1,2), circle(1,2)}
a blue cir cle{blue(1,2), circle(1,2)}
((1,2)>(2))()
T({},()) UT({blue(1,2),circle(1,2)},((1,2)>(2))) = {blue(2), circle(2)}
a blue cir cle be low the red square
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Abstract Learning Module
Aims
Derive phrasal categories from grammar fragments in r
late
Derive grammar rules that make reference to phrasal categories
Create new resource rabstract
((1)>(1))
((1)>(1), (rel)>(rel))((2)>(2))
((1,2)>(1))
()
((1,2)>(2))
((rel)>(rel))
((1,2)>(1,2))((1,2)>(1,2))((1,2)>(1,2)) ((1,2)>(1,2))
a{}
blue{blue(1,2)}
cir cle{circle(1,2)}
the{}
red{red(1,2)}
square{square(1,2)}
blue cir cle{blue(1,2), circle(1,2)}
red square{red(1,2), square(1,2)}
the red square{red(1), square(1)}
a blue cir cle{blue(2), circle(2)}
be low the red square{below(rel), horizontal_even(rel), red(1), square(1)}
a blue cir cle be low the red square{below(rel), blue(2), circle(2), horizontal_even(rel), red(1), square(1)}
be low{below(rel), horizontal_even(rel)}
()
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Abstract Learning Module
Aims
Derive phrasal categories from grammar fragments in r
late
Derive grammar rules that make reference to phrasal categories
Create new resource rabstract
NP
((1)>(1))
((1)>(1), (rel)>(rel))((2)>(2))
((1,2)>(1))
()
((1,2)>(2))
((rel)>(rel))
((1,2)>(1,2))((1,2)>(1,2))((1,2)>(1,2)) ((1,2)>(1,2))
DET1 ADJ N DET
2 ADJ N
NP
NP2
NP1 POS
S
REL
()
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Abstract Learning Module
Creating phrasal categories:
Phrasal categories can be derived from the grammmar fragments in r
late by assuming that their members share
distributional information
1)
blue{blue(1,2)}
cir cle{circle(1,2)}
blue cir cle{blue(1,2), circle(1,2)}
((1,2)>(1,2))((1,2)>(1,2))
2)
blue{blue(1,2)}
square{square(1,2)}
blue square{blue(1,2), square(1,2)}
((1,2)>(1,2))((1,2)>(1,2))
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Abstract Learning Module
1)
blue{blue(1,2)}
cir cle{circle(1,2)}
blue cir cle{blue(1,2), circle(1,2)}
((1,2)>(1,2))((1,2)>(1,2))
2)
blue{blue(1,2)}
square{square(1,2)}
blue square{blue(1,2), square(1,2)}
((1,2)>(1,2))((1,2)>(1,2))
cir cle{circle(1,2)}
square{square(1,2)}
Phrasal category 1:
Creating phrasal categories:
Phrasal categories can be derived from the grammmar fragments in r
late by assuming that their members share
distributional information
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Abstract Learning Module
1)
blue{blue(1,2)}
Phrasal category 1
blue cir cle{blue(1,2), circle(1,2)}
((1,2)>(1,2))((1,2)>(1,2))
2)
blue{blue(1,2)}
square{square(1,2)}
blue square{blue(1,2), square(1,2)}
((1,2)>(1,2))((1,2)>(1,2))
cir cle{circle(1,2)}
square{square(1,2)}
Phrasal category 1:
Creating phrasal categories:
Phrasal categories can be derived from the grammmar fragments in r
late by assuming that their members share
distributional information
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Abstract Learning Module
Creating phrasal categories:
Phrasal categories often share similar members
Subset categories are replaced by their superset categories
cir cle{circle(1,2)}
square{square(1,2)}
star{star(1,2)}
tri ang gle{triangle(1,2)}
Phrasal category 2:cir cle
{circle(1,2)}
square{square(1,2)}
Phrasal category 1:
is replaced by
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Abstract Learning Module
Creating grammar rules:
Grammar rules are created by linking the grammar fragments that make reference to phrasal categories
((1,2)>(2))
DET1
NP
NP1
()
((1,2)>(1,2)) ((1,2)>(1,2))
ADJ N
NP
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Abstract Learning Module
Creating grammar rules:
Grammar rules are created by linking the grammar fragments that make reference to phrasal categories
((1,2)>(2))
DET1
NP
NP1
()
((1,2)>(1,2)) ((1,2)>(1,2))
ADJ N
NP
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Abstract Learning Module
Creating grammar rules:
Grammar rules are created by linking the grammar fragments that make reference to phrasal categories
((1,2)>(2))
DET1
NP
NP1
()
((1,2)>(1,2)) ((1,2)>(1,2))
ADJ N
rabstract
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Abstract Comprehension Module
Comprehension:
Given a string to comprehend, the model searches rabstract
that can be instantiated to make the string
The accompanying meanings is returned
((1,2)>(2))
DET1
NP
NP1
()
((1,2)>(1,2)) ((1,2)>(1,2))
ADJ N
If DET1 = a, ADJ = red, ye low and N = cir cle, heart,
could comprehend a red cir cle, a red heart, a ye low cir cleand a ye low heart
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Summary
Environment(real or
simulated)
lcrosssit R esources
renv
rcrosssit
rholophrastic
rearly
rlate
rabstract
lholophrastic
rcrosssit
renv
rholophrastic
rcrosssit
learly
rearly
rholophrastic
cearly
rearly
cholophrastic
rholophrastic
clate
rlate
cabstract
rabstract
Learn ingM odules
C o m prehens ionM odules
WorldPerception Module
renv
Sensory stimuli
llate
rlate
rholophrastic
labstract
rabstract
rlate
IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Testing
Training
The model is trained to learn a miniature language by observing event-description pairs
100 sets of 125 event-description pairs were randomly generated
After each pair is entered, the model is tested for comprehension of a set of strings
The results are used to determine the model's stage of linguistic development
6 12 18 24 30 360 42 48
Time (months)
Pre-linguisticHolophrastic
Early Multi-wordLate Multi-word
Abstract
IntroductionChild LanguageModelsLATTesting➢Miniature Lang.➢Templates➢RequirementsResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Miniature Language
S = NP1 REL NP
2
REL = REL1 | REL
2
REL1 = a bove | be low | to the REL
4
REL2 = REL
3 REL
4
REL3 = to the low er | to the u pper
REL4 = left of | right of
NP1 = DET
1 NP
NP2 = DET
2 NP
NP = SHAPE COLOURCOLOUR = black | blue | grey | green | pink | black | red
| whiteSHAPE = cir cle | cross | dia mond | heart | rec tang gle
| star | square | tri ang gle
Can create 32,768 unique sentences such as:
a blue cir cle a bove the green squarea red dia mond to the left of the white stara pink rec tang gle to the low er right of the black square...
IntroductionChild LanguageModelsLATTesting➢Miniature Lang.➢Templates➢RequirementsResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
String Templates for Testing
String Templates
To observe the performance of the model, it is tested for comprehension of a set of strings, shown below by template
String Template Example String Total
Shape cir cle 8
Colour red 8
Position a bove 6
Half Relative Position to the u pper 4
Relative Position a bove the 8
Object red cir cle 82 = 64
Indefinite Object a red cir cle 82 = 64
Definite Object the red cir cle 82 = 64
Object Relative Position a red cir cle above the 83 = 512
Relative Position Object a bove the red cir cle 83 = 512
Event a red cir cle a bove the red square 85 = 32,768
Total No: 34,018
IntroductionChild LanguageModelsLATTesting➢Miniature Lang.➢Templates➢RequirementsResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Judging Linguistic Stage
When the model comprehends strings, its behaviour can be described in terms of stages:
Pre-linguistic – no comprehension
Holophrastic – comprehension of any string
Early – string is comprehended as a composite of its parts
Late – string is comprehended as a composite of its parts that require use of syntactic markings
Abstract – a set of NPs are comprehended, where the set includes all known ADJs and Ns
End point – all sentences are successfully comprehended
6 12 18 24 30 360 42 48
Time (months)
Pre-linguisticHolophrastic
Early Multi-wordLate Multi-word
Abstract
IntroductionChild LanguageModelsLATTesting➢Miniature Lang.➢Templates➢RequirementsResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Results
ho lo ph ra st ic
ea rly mu lti w
or d
lat e mult iw
o rd
po st a bs tr ac t
ab st ra ct
0 20 40 60 80 100 120
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
Stages of Language Acquisition
Holo
EarlyLate
Abstract
End
No. Events Observed
% o
f Req
uire
men
ts M
et
Onsets: Holo (1); Early (11.9); Late (23.88); Abstract (49.83); End (88.04)Lengths: Holo (10.9); Early (11.98); Late (25.95); Abstract (38.21)
IntroductionChild LanguageModelsLATTestingResults➢Holophrastic➢Early Multi-word➢Late Multi-word➢AbstractDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
The Holophrastic Stage
● The majority of strings comprehended (94%) are atomic units in the language
● Word segmentation and association with appropriate meanings
● Discovery of atomic units in the miniature language e.g. cir cle and to the u pper
The Holophrastic Stage
Object3%
Colour29%
Half-Relative-Position
21%
Relative-Position13%
Definite-Object1% Complete-Event
2%
Shape31%
IntroductionChild LanguageModelsLATTestingResults➢Holophrastic➢Early Multi-word➢Late Multi-word➢AbstractDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
The Holophrastic Stage
● Under-extensions such as cross means {cross(1, 2), above(rel)}
● Over-extensions such as blue cross means {blue(1, 2)}
● Mismatches such as low means {pink(1, 2), below(rel)}
The Holophrastic Stage
Object3%
Colour29%
Half-Relative-Position
21%
Relative-Position13%
Definite-Object1% Complete-Event
2%
Shape31%
IntroductionChild LanguageModelsLATTestingResults➢Holophrastic➢Early Multi-word➢Late Multi-word➢AbstractDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
The Early Multi-word Stage
● There is a rise in the comprehension of composite strings
● Strings are comprehended as a composite of their parts e.g. red cir cle is comprehended from the meanings of red and cir cle
The Early Multi-word Stage
Indefinite-Object2%
Definite-Object4%
Object-Relative-Position
1%Complete-Event
2%
Shape24%
Object22%
Relative-Position9%
Half-Relative-Position
12%
Colour24%
IntroductionChild LanguageModelsLATTestingResults➢Holophrastic➢Early Multi-word➢Late Multi-word➢AbstractDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
The Early Multi-word Stage
No sensitivity to syntactic markings
a red square means the same as the red square and red square
a red square a bove the green cir cle means the same as a green cir cle a bove the red square
The Early Multi-word Stage
Indefinite-Object2%
Definite-Object4%
Object-Relative-Position
1%Complete-Event
2%
Shape24%
Object22%
Relative-Position9%
Half-Relative-Position
12%
Colour24%
IntroductionChild LanguageModelsLATTestingResults➢Holophrastic➢Early Multi-word➢Late Multi-word➢AbstractDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
The Late Multi-word Stage
● There is a rise in the comprehension of composite strings that require sensitivity to syntactic markings
● Strings are comprehended as a composite of their parts e.g. “red cir cle” is comprehended from the meanings of “red” and “cir cle”
The Late Multi-word Stage
Complete-Event8%
Relative-Position-Object
2%
Shape6%
Colour6%
Half-Relative-Position
3%
Relative-Position3%
Object-Relative-Position
9%
Definite-Object17%
Indefinite-Object16%
Object30%
IntroductionChild LanguageModelsLATTestingResults➢Holophrastic➢Early Multi-word➢Late Multi-word➢AbstractDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
The Late Multi-word Stage
Sensitivity to syntactic markings
a red square means a novel red square
the red square means the existing red square
red square means a novel or the existing red square
a red square a bove the green cir cle is differentiated from a green cir cle a bove the red square
The Late Multi-word Stage
Complete-Event8%
Relative-Position-Object
2%
Shape6%
Colour6%
Half-Relative-Position
3%
Relative-Position3%
Object-Relative-Position
9%
Definite-Object17%
Indefinite-Object16%
Object30%
IntroductionChild LanguageModelsLATTestingResults➢Holophrastic➢Early Multi-word➢Late Multi-word➢AbstractDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
The Abstract Stage
● The majority of strings comprehended are Complete Events
● Complete Events are comprehended as the composition of multiple atomic units
The Abstract Stage
Object1%
Indefinite-Object1%
Definite-Object1%
Object-Relative-Position
3%
Relative-Position-Object
3%
Complete-Event91%
IntroductionChild LanguageModelsLATTestingResults➢Holophrastic➢Early Multi-word➢Late Multi-word➢AbstractDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
The Abstract Stage
● A form of rote learning is displaced by generative comprehension
● Grammars are derived that allow any string in the miniature language to be comprehended from a relatively small exposure to examples
The Abstract Stage
Object1%
Indefinite-Object1%
Definite-Object1%
Object-Relative-Position
3%
Relative-Position-Object
3%
Complete-Event91%
IntroductionChild LanguageModelsLATTestingResults➢Holophrastic➢Early Multi-word➢Late Multi-word➢AbstractDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Discussion
Behavioural stages emerge:
● In the same order as found in child language
● At similar time intervals as found in child language
● With similar developmental characteristics as found in child language
What accounts for this similar developmental trajectory bearing in mind that:
● Training data are kept constant?
● The model’s functionality is kept constant?
IntroductionChild LanguageModelsLATTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Explaining Development
The Modular Architecture
Each module concentrates on performing a different task
Each task requires a different amount of training to produce results
A new behaviour emerges when a learning mechanism solves a task for the first time
Modules depend upon training data which can be internally filtered by other modules
IntroductionChild LanguageModelsLATTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Explaining Development
● The Cross-situational Learning Module receives unaltered training data
● The Holophrastic Modules breaks the language down into atomic units producing holophrastic behaviour
● The Early Multi-word Modules begins to reconstruct the language by discovering compositional relationships
● Both the Holophrastic and Early Multi-word Modules work simultaneously, allowing the model to continue learning words while discovering compositions
● The Late Multi-word Module begins to reconstruct the language by discovering compositionality WITH sensitivity to word order and syntactic markings
● Why is there such a gap between the results produced by the Early and Late Multi-word Comprehension Modules when they perform similar tasks?
IntroductionChild LanguageModelsLATTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Explaining Development
● Why is there such a gap between the results produced by the Early and Late Multi-word Comprehension Modules when they perform similar tasks?
The Late Multi-word Learning Module is performing a more complex task than the Early Multi-word Learning Module
● The Late Multi-word Learning Module has tougher constraints (word-order and transformations must match in constructions). Given the fragments;
● The Early Multi-word Learning Module can keep the fragments but the Late Multi-word Learning Module cannot
((1,2)>(1,2)) ((1,2)>(1,2))
blue{blue(1,2)}
cir cle{circle(1,2)}
blue cir cle{blue(1,2), circle(1,2)}
() ((1,2)>(2))
a{}
blue{blue(1,2)}
a blue{blue(2)}
IntroductionChild LanguageModelsLATTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Explaining Development
● The Abstract Modules produce results much earlier than the abstract stage begins
● Much of the generative capacity in the model comes from the Abstract Modules
● The Abstract Comprehension Module accounts for comprehension of novel strings even during the early multi-word stage
● It is inappropriate to think of each module’s contribution to comprehension as being limited to a particular stage
● It is better to think of each stage as being the result of all modules producing the best results that they can given their experience
IntroductionChild LanguageModelsLATTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
Conclusions
A computational model that demonstrates a similar developmental trajectory as found in child language has been produced
There is linguistic maturation without physical maturation
The model is given a realistic exposure to training data
A Modular Structure accounts for much of the developmental shape
The stages emerge on a reasonable timescale
The stages emerge in the same order
Different modules focus upon different problems
Different linguistic behaviours may be the best indicators of underlying learning mechanisms in children
Children may also have a modular framework for learning and comprehending
IntroductionChild LanguageModelsLATTestingResultsDiscussionConclusions
A ComputationalModel of Staged
Language Acquisition
Kris Jack
IntroductionChild LanguageModelsLATTestingResultsDiscussionConclusions
Coming soon...
● Language Acquisition Toolkit (LAT) online– Freely available for research
– GNU Licence
– Run language acquisition simulations with your own modules
– Compare results within a common framework