Emergent Adaptive Lexicons Luc Steels 1996 Artificial Intelligence Laboratory Vrije Universiteit Brussel Presented by Achim Ruopp University of Washington
Feb 25, 2016
Emergent Adaptive Lexicons
Luc Steels 1996Artificial Intelligence Laboratory
Vrije Universiteit Brussel
Presented by Achim RuoppUniversity of Washington
Agenda
• Introduction to Emergence and Self-Organization
• Steel’s Experiments on the emergence of the lexicon
• Discussion
Excursion: Emergence andSelf-Organization
• A system exhibits emergence when– There are coherent
emergents at the macro-level
– That dynamically arise from interactions between parts a the micro-level
– Emergents are novel w.r.t. individual parts of the system
Excursion: Emergence and Self-Organization
• Definition– A dynamical and adaptive process– Where systems acquire and maintain structure themselves– Without external control
• Combining emergence and self-organization
Origins of Language
• Still unknown• Chomsky’s hypothesis
– Species-specific innate language ability– Refinement by parameter setting process– Some support by experimental simulation via neural
networks• Alternative hypothesis
– Language as an emergent phenomenon• As a mass-phenomenon• Spontaneously forming/becoming more complex
Steel’s Experiments
• Motivated by symbol grounding problem• Experiments on robotic and software
agents– Grounded meaning creation– Lexicon formation ← Focus of paper– Syntax– Emergent phonology
Experimental Model DefinitionsFeatures
• Set of agents A• ∀a A ∈ ∃ set of features Fa = {f0,…,fn}• A feature fi consists of attribute-value pairs
– Examples: (weight light) (size tall)• Distinctive feature set
– Subset of Fa distiguishing agent a from all agents in a group B
• Filtered subset CK,M = {a|K ⊂ Fa}– K is a set of features– M is a set of agents
Experimental Model DefinitionsLexicon
• Word– sequence of letters drawn from a shared alphabet
• Utterance– Set of words– Word order does not play a role
• Lexicon L– A relation between feature sets and words– A single word can have several associated feature
sets– A feature set can have several associated words
Experimental Model DefinitionsLexicon
• Each agent a has lexicon La
– Initially empty
• Feature set of a word: Fw,L
• Cover functions– cover(F,L): set of utterances U: ∀u ∈ U:
{f|f ∈ Fw,Land w ∈ u}– uncover(u,L): set of features F:
F = {f|f ∈ Fw,Land w ∈ u}
Coherence through Self-Organization
• Agents can– Create new words and associate them with a feature
set– Form new associations between a word and a feature
set• Key to self-organized coherence of the lexicon
– Agents participate in communication– Record the success of particular word-meaning pairs– Agents (re-)use words that led to high communication
success in the past
Language Game• Dialog between two agents – Speaker and Hearer• Dialog topic
– Other Agent– Chosen by extra-linguistic means (“pointing”)
• Speaker and hearer identify possible distinctive feature sets of topic
• Speaker– Selects distinctive feature set– Translates to words using cover function
• Hearer– Interprets utterance using uncover function– Compares interpretation to expectation– Uses game to
• Learn part of the language• Check if right meaning is associated with the right words
Language GamePossible Outcomes
1. No differentiation possible2. Speaker does not have a word
– May create new word
3. Hearer does not have a word– Can associate word– Cannot disambiguate when multiple
distinctive feature sets
Language GamePossible Outcomes
4. Speaker and hearer know word• Meanings are compatible with situation
• Sense-ambiguity possible• Meanings not compatible with situation
• No communicative success
Experimental ResultsOne-word Utterances
• Typical experimental setup (5 agents, 10 meanings, 4000 language games)
• Leads to communicative success soonAverage
communicative success
Number of language games(scale 1/20)
Experimental ResultsOne-word Utterances
• Single meanings soon converge on one word form (10 agents, 5 possible words, 1 meaning)
Average communicative
success
Time
Experimental ResultsMultiple Word Utterances
• In case the distinctive feature set of the topic contains multiple features
• Can be used by hearer to “guess” meaning of unknown words
Conclusions
• Self-organization is effective mechanism for achieving coherence
• Side-effects of lexicon formation– Synonymy– Ambiguity– Multiple-word sentences
Discussion• Supports the notion that absolute synonymy does not
exist– “After about 4000 language games the lexicon stabilizes as all
distinctions that need to be made have been lexicalized”• Are the presented “linguistic” results an artifact of the
experimental setup? I.e. in how far does this experiment reflect the real world?– E.g. multi-word sentences
• Can the results just be explained by basic communication theory?
• Language as an emergent phenomenon– Zipf’s law regarding multiple meanings– Self-organized criticality/highly optimized tolerance
fm
ReferencesBrighton, H., Selina, H.; Introducing Artificial Intelligence; 2003; Icon
Books Ltd.; ISBN 1-84046-463-1De Wolf, Tom; Holvoet, Tom; Emergence Versus Self-Organisation:
Different Concepts but Promising When Combined; 2005; In Engineering Self Organising Systems: Methodologies and Applications, Lecture Notes in Computer Science, volume 3464, pp 1-15
Steels, L.; Emergent Adaptive Lexicons; 1996; In: Maes, P. and Mataric, M.J. and Meyer, J.-A. and Pollack, J. and Wilson, S.W. (eds) From Animals To Animats 4: Proceedings of the Forth International Conference on Simulation of Adaptive Behavior, SAB'96, Complex Adaptive Systems, pp. 562-567, Cambridge, MA: The MIT Press
Zipf, G. K.; The Meaning-Frequency Relationship of Words; Journal of General Psychology 33, 251–256 (1945).