7/31/2019 Issues on Language Processing
1/46
UNIVERSIT DEGLI STUDI DI CATANIA
SCUOLA SUPERIORE DI CATANIA
Daniele Virgillito
ISSUES ON LANGUAGE PROCESSING:
FROM THEORY TO BRAIN INVESTIGATIONS
DIPLOMA DI LICENZA
Relatore: Chiar.mo Prof. Marco Mazzone
ANNO ACCADEMICO 2006/2007
7/31/2019 Issues on Language Processing
2/46
Daniele Virgillito 2
Contents
Abstract ......................................................................................................................3
Sommario ..................................................................................................................4
Acknowledgments...................................................................................................5
1. Linguistic theories and findings ...................................................................71.1 The layers of language...............................................................................7
1.2 Chomskys perspective .............................................................................9
1.3 Models of language processing..............................................................10
1.4 Time for meaning.....................................................................................12
1.5 Looking at the brain.................................................................................16
2. ERPs: what brainwaves can tell about language......................................18
2.1 Event-related Potentials ..........................................................................18
2.2 ERP correlates of semantics and syntax................................................19
2.3 More than meaning and structure.........................................................22
2.4 Semantics or syntax?................................................................................27
2.5 Chomskian account..................................................................................29
3. Mood and language........................................................................................32
3.1 Positive mood effects on language ........................................................32
Conclusions.............................................................................................................42
Bibliography ...........................................................................................................44
7/31/2019 Issues on Language Processing
3/46
Issues on language processing: from theory to brain investigations 3
Abstract
The debate on language processing has seen its centre of gravity
progressively moving from the investigation about inner architecture
of language to the empirical observation of how language is
instantiated in the human brain.
The present work firstly intends to offer a review of the
proposed theoretical models defining general dynamics of meaning
and syntactic processing.It will present the main findings on this topic deriving from
studies on language carried out through the recording of event-related
potentials (ERPs), a research methodology based on the statistical
analysis of data coming from electroencephalogram (EEG) in
cooccurrence with linguistic stimuli.
These data seem to globally speak in favour of those
frameworks that, detaching themselves from the theoretical paradigm
imposed by Noam Chomsky in the last fifty years, conceive syntactic
component as interagent with other components.
It will then show the preliminary results of a study on the effect
of emotive states on language processing, carried out at ERP Lab of
NICI (Nijmegen Institute for Cognition and Information, Nijmegen,
Netherlands) in collaboration with Dorothee Chwilla and Constance
Vissers.
N400 ERP component (index of the cognitive effort of
semantic integration) in correspondance with unexpected linguistic
stimuli, appears to be mitigated, as predicted on the basis of
Federmeier et al. (2001), by positive mood.
This seems to encourage too the idea that language processing is
closely tied to other components of our cognitive system.
7/31/2019 Issues on Language Processing
4/46
Daniele Virgillito 4
Sommario
Il dibattito sullelaborazione del linguaggio ha visto il suo
baricentro progressivamente spostarsi dallindagine sullarchitettura
interna dei sistemi linguistici allosservazione empirica di come il
linguaggio instanziato nel cervello umano.
Il presente lavoro intende offrire in primo luogo una rassegna
dei modelli teorici messi in campo per la definizione dei processi di
elaborazione del significato e della struttura sintattica.Presenter le principali conquiste dei relativi studi sul
linguaggio realizzati attraverso la rilevazione dei potenziali evento-
relati (ERPs), una metodologia di ricerca basata sullanalisi statistica
dei dati provenienti dallelettroencefalogramma (EEG) in cooccorrenza
con stimoli linguistici.
Questi dati sembrano favorire complessivamente quei modelli
che, prendendo le distanze dal modello teorico imposto da NoamChomsky negli ultimi cinquantanni, concepiscono la componente
sintattica del linguaggio come strettamente interagente con le altre
componenti.
Illustrer infine i risultati preliminari di uno studio sulleffetto
degli stati emotivi sullelaborazione del significato, condotto presso il
Laboratorio ERP del NICI (Nijmegen Institute for Cognition and
Information, Nijmegen, Paesi Bassi) in collaborazione con Dorothee
Chwilla e Constance Vissers.
La componente ERP N400 (indice dello sforzo cognitivo di
integrazione semantica) in corrispondenza ad elementi linguistici
inaspettati, appare essere mitigata, come previsto sulla base di
Federmeier et al. (2001), da uno stato emotivo positivo.
Anche questo sembra incoraggiare lidea che lelaborazione del
linguaggio sia strettamente legata ad altre componenti del nostro
sistema cognitivo.
7/31/2019 Issues on Language Processing
5/46
Issues on language processing: from theory to brain investigations 5
Acknowledgments
This work is one of the fruits of a scientific path begun in 2004,
which led me from Sicily to Bologna to the Netherlands.
It would not have been possible without the continuous
guidance and support of its inspirator: Prof. Marco Mazzone.
I had the privilege of attending his course on Language Theory
(2005), fortunately still running at Scuola Superiore di Catania, which
broadened my horizons on the nature of language and cognition
through enlightening explanations and fruitful discussions on a vast
range of topics.
I hope that the SSC intellectual space devoted to neurosciences
of language which he created and promoted will be constantly
growing and attracting more students.
The experimental part of my thesis comes from five months
spent in Nijmegen, a Dutch small and ancient town very close to
Germany. I was a trainee at NICI (Nijmegen Institute for Cognition
and Information) under the supervision of Dr. Dorothee Chwilla,
director of the ERP Lab.
I want to thank her and Dr. Constance Vissers for offering me
the chance of getting hands-on experience on event-related potentials
and for their support during the project.
A dank u wel goes to all the people who helped me with and
during the experiments: Hubert, Gerardt, Nan, Sybrine, Dan and
many others; and to all the participants who lent me their heads and
three hours of their time.
7/31/2019 Issues on Language Processing
6/46
Daniele Virgillito 6
Id like to thank all my mates at Scuola Superiore, for the
unforgettable moments spent with some of them, for letting me
improve my mimicking skills, and for their effort in keeping SSC alive.
A special thank goes to all the people who suffer(ed) the
distance from me: it is hard to go far from who you love in order to
follow your dreams.
In conclusion, Id like to give my endless and deepest thanks to
the one who gave me, alone, most of the things I have now, my
mother. Without her love, sacrifices and encouragement some of thethings I have done would have sounded simply impossible to me.
Thanks to all the people I love and who believe in me.
7/31/2019 Issues on Language Processing
7/46
Issues on language processing: from theory to brain investigations 7
1. Linguistic theories and findings
1.1The layers of language
A large part of theories on language has depicted it as a rigidly
structured system, made up of different parts, combining but not
interacting each other. Each level of language structure would then be
separated from the others and ruled by its own inner principles.
Even in a simple sentence, like John is an American boy, we could
distinguish the different plans we are addressing. We can analyze
sounds, and the way they are structured in abstract entities (called
phonemes) that allow us to recognize words because of slight
articulatory differences (boy is not toy); we can observe the
morphological structure (from the ancient greek morph, shape), i.e. the
way words are built and modified by adding certain linguistic
elements (American is an adjective derived from the noun America plus
a suffix: [[America]N+an]A); it is also possible to look at the way words
are ordered and structured in a sentence, and how these rules differ or
converge in world languages (the phrase American boy lets us see how
7/31/2019 Issues on Language Processing
8/46
Daniele Virgillito 8
English orders coupled adjectives and nouns, namely in a different
way than many other languages do).
In addition to this, we can understand the message the sentence
intends to convey because single words are either linked to non-
linguistic referents or help us to understand the existing relations
between them: this is called the semantic aspect of language.
As proposed by Evans1, one could imagine this theoretical
framework like a cake, composed of several different layers, each
corresponding to a plan of language (phonetic, morphologic, syntactic,
semantic). As we will explain soon, many linguistic theories insist in
addressing empirical questions taking just one of the layers into
account, never looking at a whole slice of the cake. This isautomatically translated in viewing linguistics as a collection of
separated sub-disciplines.
For the present purposes, we will attempt to look more in depth
at the relationship between two of the mentioned linguistic levels,
namely the syntactic and the semantic one, i.e. between the way words
are structured and their meaning. As we will see, this has represented
a huge battleground for modern studies on language.
1 See Evans et al. (2007).
7/31/2019 Issues on Language Processing
9/46
Issues on language processing: from theory to brain investigations 9
We will now move to a brief sketch of some of the assumptions
underlying the approach outlined above.
1.2 Chomskys perspective
The separatist perspective is consistent with a particular view
of mind structure, the one interpreting it as a set of different modules,
each devoted to a specific cognitive function and dealing with a certain
type of information (see Fodor 1983).
Modules would be organized in a hierarchic way, leading from
the lowest levels to the higher ones, namely from the simplest specific
cognitive operations to the most complex and general ones. Modelsbased on such view of mental processes have for this reason been
labeled as bottom up, as opposed to top-down, i.e. based on high-
level influence over lower stages of elaboration.
In the realm of linguistics, one theoretical approach has tried to
offer a totally modular perspective on language, inspired by the work
of Noam Chomsky and his Generative Grammar paradigm. Chomsky
(1957) argues that a sentence like Colorless green ideas sleep furiously
shows the necessary features to be recognized as grammatical by a
native speaker of English. This happens independently from other
kinds of knowledge: in this sense it is particularly significant to point
7/31/2019 Issues on Language Processing
10/46
Daniele Virgillito 10
out that, according to the chomskian view, the syntactic elaboration of
the sentence proceeds autonomously from the meaning of the
individual words. It represents an algorithmic and modular process,
separated at all from semantics.
This means that during sentence processing, a syntactic
structure would be built up first, according to syntactic information
alone (order of constituents, word inflections), and at a later stage
semantic and pragmatic information would be activated.
As we will now quickly see, this neat separation of syntax and
semantics has been questioned by other models of language
processing, drawing also on experimental evidence coming from
psycholinguistic research.
1.3 Models of language processing
Different approaches to how language is processed differ in the
stage at which syntactic structure is built and semantic information is
activated. The generative model, as outlined in 1.2, can be defined as
a syntax-first perspective on sentence perception.
The so-called constraint-based models (see for example
MacDonald, Pearlmutter & Seidenberg 1994) propose that meaning is
partially accessed during syntactic structure building, and can
7/31/2019 Issues on Language Processing
11/46
Issues on language processing: from theory to brain investigations 11
somehow influence sentence structuring. Some semantic features, like
animacy, would tend to be active during the processing and so play an
early and important role in the final outcome of the analysis. In
addition to this, a series of factors such as expectancy and frequency of
use of a syntactic structure would influence the recognition of the
actual configuration of a sentence.
Trueswell et al. (1994) experimentally shows with an eye-
tracking study the effect of meaning over syntactic parsing. The test
reveals longer fixation-durations on disambiguating phrases like by the
lawyer in The defendant examined by the lawyer (see fig.1): one could
plausibly imagine, on a semantic basis, that examined is not a past
participle but the finite verb bound to the subject the defendant.
Figure 1
7/31/2019 Issues on Language Processing
12/46
Daniele Virgillito 12
Bever, Sanz & Townsend (1998) assign a much more important
role to semantics. According to them, only semantic and pragmatic
information is used in a first stage to build an initial representation of
the sentence, that consequently constrains its syntactic analysis. This
and similar approaches have been labeled as semantics-first and
their model can be approximately summarized by the phrase
semantics proposes, syntax disposes. It is natural that these theories
are mainly based on non-modular accounts of mental processes, since
they outline a top-down model of language processing.
Psycholinguistic research has been trying to empirically test the
approaches sketched above, attempting to question theoretical models
by experimentally looking at the way human beings process language.This has led to a strong criticism towards the chomskian framework.
In the next paragraph, we will recall one of the first and best
known findings against Chomskys claims.
1.4Time for meaning
Language works as part of the human cognitive system and it
can then be subject to objective scientific measurement. Human
behaviour is linked to inner mental processes and variations in the
7/31/2019 Issues on Language Processing
13/46
Issues on language processing: from theory to brain investigations 13
latter ones can lead to variation in behaviour. It is generally assumed
that statistically significant time differences between behavioural
responses point to differences in the corresponding mental processes.
One of Chomskys first claims was that sentence complexity
depended only on the numbers of syntactic transformations that the
so-called deep structure (specifying the logical role of each element) had
to undergo in order to get to the final outcome, the superficial structure
(i.e., the actual sentence including the final phrase order and
structure). This has been called the Derivational Theory of Complexity
(henceforth, DTC).
For instance, the deep structure
[[John]NP [[read]V [the book]NP ]VP ]S
can give rise to the following sentences, differing just in the
transformational rule(s) applied:
Transformational Rule Sentence
Active John read the book.
Passive The book was read by John.
7/31/2019 Issues on Language Processing
14/46
Daniele Virgillito 14
Negative John didnt read the book.
Wh-Question What did John read?
Passive + Negative The book was not read by John.
Passive + Wh-Question What was read by John?
One of the first studies addressing the psychological reality of
the DTC hypothesis was the one carried out by Slobin (1966). His
participants had to listen to a sentence while looking at a picture,
depicting objects and human beings interacting in different ways. The
task assigned was judging as quickly as possible, by pressing a button,
whether the picture matched with the sentence.
The linguistic materials were organized according to two
independent variables, i.e. active/passive and reversible/irreversible,
mixed in four final conditions.
The latter variable (reversibility) consisted in the possibility of
inverting Agent and Theme in the same sentence with a plausible
result: The sentence The horse was kicked by the cow is reversible,
whereas The fence was kicked by the cow is not, since the Theme the fence
is inanimate and cannot replace, due to the verbs restrictions, the
Agent the cow.
7/31/2019 Issues on Language Processing
15/46
Issues on language processing: from theory to brain investigations 15
Although the data obtained in the active condition was
compatible with the chomskian approach, the surprising finding was
that participants mean reaction times (RTs) in verifying passive
reversible sentences were higher than those shown for passive
irreversible ones (see fig.2 for an example).
Figure 2
This result was interpreted in the following way: since no
syntactic divergence can be found between these two kinds of
sentence, being reversibility a purely semantic feature, the difference
in the RTs represented a reliable index of semantic influence over
language processing.
Slobins is one of the first experimental studies addressing the
congruence of a theoretical linguistic model with the actual way
language is processed.
7/31/2019 Issues on Language Processing
16/46
Daniele Virgillito 16
1.5Looking at the brain
The natural place for studying cognition in vivo is where mental
processes are born and spread in time: the human brain. Over the last
years many brain-imaging techniques have been developed in order to
find neural correlates of cognitive models. They attempt to find
reliable relationships between certain patterns of brain activation and
specific supposed cognitive (sets of) operations.
At the present state of affairs, these techniques mainly differ in
their capacity and goals: resonance-based methods, such as fMRI,
manage to obtain a very detailed spatial resolution of the studied
processes, but no accurate time resolution. That is to say that they
easily localize brain activity in specific areas, but cannot exactly
correlate it with external occurring factors since the latter are usually
presented very fast.
EEG (electroencephalogram)-based techniques, on the other
hand, dont succeed in obtaining an accurate space mapping of
cerebral activation, but find their strength in their extremely accurate
time resolution. They can then easily correlate external stimuli
7/31/2019 Issues on Language Processing
17/46
Issues on language processing: from theory to brain investigations 17
presented at a very fast rate with variations in brain activity unfolding
in a limited time.
One of the best known EEG-based techniques, namely ERPs
(event-related potentials), will be the object of the next chapter. As we
will see, this experimental methodology has proven to be very useful
to investigate language: it provided evidence for unravelling the
complexity of linguistic processes and trying to link some of them to
more general cognitive domains.
7/31/2019 Issues on Language Processing
18/46
Daniele Virgillito 18
2. ERPs: what brainwaves can tell about language
2.1 Event-related Potentials
Event-related Potentials (ERPs) represent one of the most
widespread experimental techniques in brain imaging studies about
language.2
They are based on Electro-Encephalogram (EEG), a recording of
the spontaneous electrical human brain activity and its variations over
time. It is generally measured via electrodes attached on the scalp,
mounted in an elastic cap.
ERPs are extracted from the raw EEG by time-locking it to a
particular sensorimotor or cognitive event, so that it is possible to
examine the brain response to a specific kind of stimulus. To properly
extract ERPs, it is necessary to average a large number of records time-
locked to the same stimulus type.
The outcome of the averaged signal is a series of negative and
positive voltage peaks, each one having a specific latency and
2 See Kutas & Van Petten (1988).
7/31/2019 Issues on Language Processing
19/46
Issues on language processing: from theory to brain investigations 19
distribution across the scalp. These peaks are called ERP components:
they can be either exogenous or endogenous.
The first ones are the earliest components (around 100 ms after
stimulus onset) and their features mirror the physical properties of
stimuli. Cognitive studies (in particular linguistic ones) only address
the so-called endogenous components: these derive from higher levels
of information processing (their latencies are often over 300 ms) and
are mainly influenced by psychological factors.
Endogenous ERP components are usually labeled according to
two parameters: polarity and peak latency. P300, for instance is a
positive voltage fluctuation peaking around 300 milliseconds after the
stimulus onset. We will now review two of the best known language-related ERP components.
2.2 ERP correlates of semantics and syntax
The "separatist" approach sketched in chapter 1 has influenced
the early stage of ERP studies on language. Many scholars encouraged
the view that semantic and syntactic processing proceeded in
autonomous ways and were mirrored by different and independent
ERP components too (see fig. 5 for an example in the same sentential
7/31/2019 Issues on Language Processing
20/46
Daniele Virgillito 20
context). This assumption was particularly clear at the beginning, but
changed somehow as new results appeared.
The N400 component (a negative potential peaking around 400
ms after word onset) has traditionally been labeled as an index of
semantic integration. Kutas and Hillyard (1980) discovered that all
open class words elicited this kind of negativity. In addition to this,
they found out that words that rendered sentences semantically
anomalous elicited a more negative component.
Figure 3
One of their most famous examples was He spread the warm bread
with socks: compared to the brainwave elicited by the final word butter
in the same sentence, a clearly larger negativity was observed. For this
reason, N400 has been linked to the ease with which a word is
semantically integrated in a sentential context (see figure 3).
7/31/2019 Issues on Language Processing
21/46
Issues on language processing: from theory to brain investigations 21
The brain response to syntactic processing was quite different. A
later positivity (peaking at about 600 ms after word onset) was
observed in relation to verb agreement anomalies, or wrong
constituent order, e.g. The spoilt child throw the toys on the floor, or The
expensive very tulip... (Hagoort et al. 1993). A late positivity was also
elicited by garden-path sentences, i.e. initially syntactically ambiguous
sentences in which the first (more plausible) structural analysis turns
out to be the wrong one, like in The woman convinced her children are
noisy (see fig. 4 for an example), where the parser plausibly supposes
at the beginning that her is a possessive adjective while it is a object
pronoun, as clear from the overall structure of the sentence.
Figure 4
The so called P600 was then defined as the brain reaction to
grammaticality anomalies, and was thought to mirror a process of
syntactic reanalysis and reprocessing.
7/31/2019 Issues on Language Processing
22/46
Daniele Virgillito 22
Figure 5
As we will now see, new experimental results coming from
more recent studies suggested partially different and broader
perspectives on the roles played by N400 and P600 components.
2.3More than meaning and structure
As outlined above, the N400 was thought to represent a
semantic processing component, modulated by meaning relations
existing in the mental lexicon. As shown by many studies addressing
the function of N400 in non-anomalous sentences, an N400 effect was
clearly associated with a lower cloze probability. That is to say that
7/31/2019 Issues on Language Processing
23/46
Issues on language processing: from theory to brain investigations 23
this negativity was not only depending on semantic factors (although
some results could speak in favour of this conclusion: see fig. 6).
Figure 6
An N400 was clearly elicited by non-anomalous but unexpected
words, as the final word in He mailed the letter without a thought,
compared to He mailed the letter without a stamp (Kutas & Hillyard
1984). It was also observed in world knowledge violations, such as in
Dutch trains are white: Dutch speakers know, in fact, that they are
yellow (Hagoort et al. 2004). As a matter of fact, N400 can be defined
as a general index of word integration, sensitive to semantic,
pragmatic and discourse factors.
7/31/2019 Issues on Language Processing
24/46
Daniele Virgillito 24
Several ERP studies showed a clear P600 effect as a response to
non syntactic anomalies. Vissers, Chwilla, & Kolk (2006) investigated
the brain response to misspellings at the word level (see fig.7)
Figure 7
Their paradigm involved the comparison between a high-cloze
condition (the critical, misspelled pseudohomophone word was the
most expected one) and a low-cloze one (the critical, misspelled
pseudohomophone word was not expected at all), e.g. In the library the
pupils borrow books/boeks vs. In the library the pupils borrow chairs/chayrs.
7/31/2019 Issues on Language Processing
25/46
Issues on language processing: from theory to brain investigations 25
A clear late positivity (P600) was found in high-cloze sentences
containing misspellings. This encouraged the perspective according to
which P600 would reflect a more general process of monitoring in
language perception, aiming at the detection of a possible processing
error.
In the study cited above, this component appeared only in the
high-cloze pseudohomophone condition, namely the one in which it is
plausible to suppose that a strong conflict occurred between two
elements: the phonological representation of the word (perfectly
congruent with the sentential constraints, and highly expected) and its
orthographic realization, which triggered a monitoring response.
As summarized by Vissers et al. (2006),
The ERP data confirmed the present prediction in that only
pseudohomophones embedded in a high-cloze context gave rise to a
P600 effect. Because the words from which the pseudohomophones
were derived were highly expected, initially the pseudohomophones
were easily integrated into the higher order meaning representation
of the context. After all, the phonological representation of the
pseudohomophone is congruent with the sentential constraints. But
when the subject detected the misspelling, which signals a possible
processing error, a monitoring response was triggered.
7/31/2019 Issues on Language Processing
26/46
Daniele Virgillito 26
Such perspective on P600s functional role has been confirmed
by further studies addressing a different kind of mismatch at the
sentence level. Vissers, Chwilla, Van de Meerendonk & Kolk (2008),
for instance, presented participants several pictures of spatial arrays
followed by a sentence giving a correct or incorrect description of the
picture.
For example:
Picture Sentence
The triangle stands above the square.
The triangle stands below the circle.
As predicted, the mismatches created a conflict between the
conceptual representation on the basis of the picture and the actual
sentence, and therefore led to a P600 effect.
7/31/2019 Issues on Language Processing
27/46
Issues on language processing: from theory to brain investigations 27
2.4Semantics or syntax?
According to the experimental results illustrated above, no clear
definition of the functional significance of language-related
components can be established. Until a few years ago, however, N400
and P600 were thought to belong to separated, albeit broad, areas of
sentence processing.
Further language ERP studies have shown quite controversial
late positivities in unusual situations (see Kolk & Chwilla 2007). The
most surprising finding was what we could define a semantic P600.
As outlined in 2.2, late positivities traditionally occurred in
sentences showing syntactic anomalies or ambiguities. In particular,
many experiments addressed a morphosyntactic P600, namely the one
elicited by anomalies in subject-verb agreement.
The unexpected result came from semantically anomalous
sentences like The cat that from the mice fledsing (literal translation
from Dutch: De kat die voor de muizen vluchtte, see Kolk, Chwilla, Van
Herten & Oor 2003). Meaning anomalies were supposed to elicit an
N400 effect and no positivity, but the opposite happened (see fig. 8).
7/31/2019 Issues on Language Processing
28/46
Daniele Virgillito 28
Figure 8
It was however possible to reconcile this surprising finding with
a syntactic view of P600. The sentence type reported above did not
involve any kind of syntactic anomaly or ambiguity. Although it is a
highly implausible sentence (mice are supposed to flee from cats), its
structure doesnt violate any grammatical rule.
As a matter of fact, readers processing language dont always
apply a rigid parse ruled by grammar. The processing of sentences is
often guided by heuristics-based principles. Heuristic is represented in
this case by a set of strategies based on plausibility, intended to
process sentence meanings by overcoming a rule-based analysis of
sentence structure. In everyday language it is in fact often possible to
derive a correct structural representation of a sentence simply drawing
7/31/2019 Issues on Language Processing
29/46
Issues on language processing: from theory to brain investigations 29
on the semantic features of the lexical items, and combining them in a
plausible way.
Processing the sentence The cat that from the mice fledsing. could
have led to assume that the plausible subject would be the mice, and
then trigger a P600 due to the morphosyntactic anomaly: no agreement
between the mice (plural) and the verbal form (singular). No
correspondence between the predicted and the observed inflection.
This interpretation was questioned by Van Herten, Kolk &
Chwilla (2005). The same kind of sentences was presented to
participants, with one difference: the two noun phrases never differed
in grammatical number, avoiding then agreement errors between the
plausible subject and its related verb form.Notwithstanding this last modification, a late positivity occurred
at the verb level. This challenged the mentioned syntactic account of
P600 and can thus be explained only in terms of monitoring during
language perception (see 2.3).
2.5 Chomskian account
Electrophysiology of language has proven to show a very
heterogeneous panorama of brain responses to linguistic processes,
7/31/2019 Issues on Language Processing
30/46
Daniele Virgillito 30
leading to difficulties in deciphering the functional origins and range
limits of each ERP component.
In spite of the limits shown by current accounts of language-
related brain activation patterns, a remainder of the chomskian
approach is still present in the debate.
In particular, language-specificity of ERP components, in spite
of the counterevidence presented above, has been defended by
Friederici and other scholars trying to outline a brain activation model
consistent with the separatist thesis, i.e. the one supporting the
autonomy of syntactic processes. We will now move to the analysis of
part of the chomskian approach to neurolinguistic findings (see
Friederici et al. 2006).The so-called ELAN (early left anterior negativity) was the
brainwave found in correlation with phrase structure violations. Due
to its very short latency (between 100 and 250 ms from the stimulus
onset) and its independence from attentional factors, ELAN was
defined as an index of early syntactic processing, involving the
computation of local phrase structures according to word category
information. The definition of a low-level phrase would be influenced
by word-level features. A P600 would then be correlated with a final
integration process leading to the structuring of the overall sentence.
7/31/2019 Issues on Language Processing
31/46
Issues on language processing: from theory to brain investigations 31
Although these results point to a syntactic-only brain mapping
of linguistic processes, they seem as pointed out by Mazzone (in press)
somehow to contradict the theoretical assumptions of the chomskian
Minimalist Program (Chomsky 1995). According to the latter, the same
operation of Merge (the making up of a new syntactic unit from two
syntactic objects) should be working both at the local phrase level (e.g.
the phrase The old man) and at the highest phrase level, i.e. the sentence
(e.g. The old man bought a small cake).
The model outlined above seems to correlate Merge at different
levels with qualitative differences in the cerebral activation timecourse
and mapping. fMRI studies, in fact, showed the predominance of two
distinct brain areas: the frontal operculum and Brocas area, and ERPrecording highlighted two different components too.
These and many other findings discourage the view that
linguistic processes proceeds through fixed serial stages and are
centered on syntax. On the one hand, sub-components of language, i.e.
the layers (see 3.3) in which language is normally dissected, seem
to continuously interact during related cognitive dynamics; on the
other hand, it emerges that non-linguistic factors can affect linguistic
processes, for example their timecourse, and the effort required.
It is the case of the role of emotive states in semantic processing:
we will talk about this in the following chapter.
7/31/2019 Issues on Language Processing
32/46
Daniele Virgillito 32
3. Mood and language
3.1 Positive mood effects on language
Event-related potentials have proven to represent a very
profitable experimental paradigm for assessing not only the basic
nature of cognitive processing, but also how external psychological
factors can globally affect the dynamic of specific processes.
One of the factors involved in these assessments has been
emotive state. As clearly stated in Federmeier et al. (2001),
Empirical research concurs with the everyday intuition that
our moods influence our thinking, judgment, and perceptions.
Transient, mild positive mood has been shown to increase
integrative decision-making and subjective risk assessment and
to facilitate flexibility of thinking and problem solving.
We will now review two of the most acknowledged findings on
the effect of positive mood over the language domain.
7/31/2019 Issues on Language Processing
33/46
Issues on language processing: from theory to brain investigations 33
Kirson (1990) used the sentence verification task (asking subjects
to state if a sentence is true or false). The experiment provided
participants with sentences describing category exemplar relationships
(e.g. A robin/coin is a bird). True sentences contained both exemplars
that were a near associate of the category word and those with a more
distant association (e.g. A robin/parrot is a bird).
After inducing either a neutral or a positive mood by showing
subjects respectively either a boring or an amusing (as rated by a
group of students) videotape, reaction times were recorded during
sentence verification. As expected, sentences depicting a close
relationship were verified in less time than those describing a distant
relationship. This difference was smaller, however, with positivemood induction, as it specifically shortened participants time to verify
distant sentences (see fig. 9).
These findings, replicated in a lexical decision task, supported
the notion that the semantic distance between distant exemplars and
their categories was functionally reduced due to the positive mood
condition.
7/31/2019 Issues on Language Processing
34/46
Daniele Virgillito 34
Figure 9
A second study carried out by Federmeier, Kirson, Moreno &
Kutas (2001) investigated positive mood effect on language processing
through ERP recording.
Participants were presented with sentence pair contexts (e.g.
They wanted to make the hotel look more like a tropical resort. So, along the
driveway they planted rows of) ended with (1) the most expected
ending, as determined by cloze probability ratings (expected
exemplars, e.g. palms), (2) an unexpected item from the expected
semantic category (within category violations, e.g. pines), or (3) an
equally unexpected item from a different, though related, semantic
category (between category violations, e.g. tulips).
Mood was manipulated exposing participants to different kind
of photos, intended to engender particular emotive responses. The
effectiveness of the induction was positively tested after the
experiment through a questionnaire.
7/31/2019 Issues on Language Processing
35/46
Issues on language processing: from theory to brain investigations 35
As summarized by Federmeier et al. (2001),
Under neutral mood, N400 amplitudes were smallest for
expected items and smaller for unexpected items when these
came from the expected category. In contrast, under positive
mood, N400 amplitudes to the two types of unexpected items
did not differ (see fig.10). Positive mood seemed to specifically
facilitate the processing of distantly-related, unexpected items.
Figure 10
In both studies, positive mood contributes to the process of
semantic retrieval and integration. It facilitates in both cases the access
7/31/2019 Issues on Language Processing
36/46
Daniele Virgillito 36
to distant concepts and, according to the N400 effect found, makes
their processing easier.
We will now present the preliminary results of our study on the
effect of emotive state on language processing.
Our study replicated Vissers, Chwilla, & Kolk (2006)
investigating brain response to misspellings at the word level, with the
addition of a mood manipulation.
We first constructed 127 simple declarative sentence fragments
and used these in a cloze test with 25 subjects to obtain highly
expected (high-cloze) critical words. Of these 127 sentences, 116
sentences were completed with the same word by 91% of the
participants. These were used as the high-cloze context sentencefragments in this study.
We then created 116 low-cloze context sentences by exchanging
the critical word from a high-cloze context fragment with the critical
word from another high-cloze context fragment. For example, we
exchanged the critical word from In that library the pupils borrow books
to take home with the critical word from The pillows are stuffed with
feathers which makes them feel soft resulting in the following low-cloze
fragment The pillows are stuffed with books which makes them feel hard. The
critical word was always in mid-sentence position.
7/31/2019 Issues on Language Processing
37/46
Issues on language processing: from theory to brain investigations 37
A further experimental manipulation was Lexicality (correct
word vs. pseudohomophone derived from the correct word).
Every critical word occurred in a correct version and a
pseudohomophonic version. The pseudohomophone was created by
changing the vowel of the second syllable, keeping phonology the
same.
The two experimental manipulations Context and Lexicality
were crossed. As a result, there were four conditions and thus, four
experimental sentence types: high-cloze correct word sentences, high-
cloze pseudohomophone sentences, low-cloze correct word sentences,
and low-cloze pseudohomophone sentences; yielding a total set of 464
sentences. The four versions of each sentence were counterbalancedacross lists. Each list contained each sentence context (in a high-cloze
or a low-cloze version) and each critical word (in a correct word or a
pseudohomophone version) only once.
So, each list contained 29 high-cloze correct word sentences, 29
high-cloze pseudohomophone sentences, 29 low-cloze correct word
sentences, and 29 low-cloze pseudohomophone sentences. To each list,
60 filler sentences were added: 30 correct sentences, 10 sentences with
a pseudohomophone at the beginning of the sentence, 10 sentences
with a pseudohomophone in the middle of the sentence, and 10
sentences with a pseudohomophone at the end of the sentence.
7/31/2019 Issues on Language Processing
38/46
Daniele Virgillito 38
With the purpose of comparing the monitoring effect supposed
to be elicited by high-cloze pseudohomophones with the classical
syntactic P600, we added to the material a set of sentences
containing morphosyntactic violations. The related ERP results will
not be included in the present work.
For the EEG study, participants were seated in an experimental
room. Sentences were presented in serial visual presentation mode at
the center of a PC monitor. Word duration was 345 ms and the
stimulus-onset asynchrony (SOA) was 645 ms. Sentence final words
were followed by a full stop. The intertrial interval was 2 s.
Words were presented in black capitals on a white background
in a 9 cm by 2 cm window at a viewing distance of approximately 1m. Each sentence was preceded by a fixation cross (duration 510 ms)
followed by a 500 ms blank screen. The experimental list was split up
into six blocks; there was a brief pause between blocks and each block
was preceded by two filler items.
The mood manipulation was carried out by means of two sets of
video fragments, each shown before and between each experimental
block to two groups of participants. They were intended to engender
positive and negative mood conditions (henceforth, PMC and NMC).
7/31/2019 Issues on Language Processing
39/46
Issues on language processing: from theory to brain investigations 39
The clips were taken from Happy Feet (G. Miller, Warner
Bros. 2006) and Sophies Choice (A.J. Pakula, Universal Pictures
1982) for the positive and negative manipulation respectively.
The effectiveness of the mood manipulation was assessed
through an online self-rating of participants emotive state.
Participants were instructed to attentively read the sentences.
Because eye movements distort the EEG recording, participants were
trained to make eye movements, e.g., blinks, only in the period
between the end of the last sentence and the beginning of the next one.
The electroencephalogram (EEG) was recorded with 27 tin
electrodes mounted in an elastic electrode cap (Electrocap
International; see fig.11 for the montage).
Figure 11
7/31/2019 Issues on Language Processing
40/46
Daniele Virgillito 40
The electrode positions included standard International 1020
system locations over the left and right hemispheres at the frontal (F3,
F4, F7, and F8), midline (Fz, Cz, Pz, and Oz), parietal (P3 and P4), and
temporal (T5 and T6) sites. Eight extra electrodes were placed at the
frontal (F3A, F4A, F7A, and F8A), midline (Fza and Oz), and parietal
(P3P and P4P) sites. In addition, eight electrodes were placed at
nonstandard electrode positions previously found to be sensitive to
language manipulations (e.g., Holcomb & Neville, 1990): left and right
anteriortemporal sites (LAT and RAT: 50% of the distance between
T3/4 and F7/8), left and right temporal sites (LT and RT: 33% of the
interaural distance lateral to Cz), left and right temporoparietal (LTP
and RTP: Wernicke's area and its right hemisphere homologue: 30% ofthe interaural distance lateral to a point 13% of the nasioninion
distance posterior to Cz), and left and right occipital sites (OL and OR:
50% of the distance between T5/6 and O1/2). The left mastoid served
as reference. Electrode impedance was less than 3 k. The electro-
oculogram (EOG) was recorded bipolarly; vertical EOG was recorded
by placing an electrode above and below the right eye and the
horizontal EOG was recorded via a right to left canthal montage. The
signals were amplified (time constant = 8 s, bandpass = 0.0230 Hz),
and digitized online at 200 Hz.
7/31/2019 Issues on Language Processing
41/46
Issues on language processing: from theory to brain investigations 41
Presentation of stimuli and recording of performance data were
accomplished by a Macintosh computer.
For N400 (300 to 500 ms epoch), an interaction of cloze
probability and mood was found for the midline and the lateral sites.
For the midline sites an N400 was present for the positive mood
condition but absent for the sad condition. For the lateral sites the
interaction indicated that the N400 in the PMC was more broadly
distributed across the scalp than in the NMC.
Further analyses are still under way on the pseudohomophone
and syntactic condition.
7/31/2019 Issues on Language Processing
42/46
7/31/2019 Issues on Language Processing
43/46
Issues on language processing: from theory to brain investigations 43
governed by a set of segregated modules, independent from other
cognitive factors and must be separately studied in all its different
compartments.
This paradigm could be extended to other kinds of cognitive
integration tasks, such as the visual one.
Moreover, future ERP works could also address different facets
of semantic integration (e.g. at the verb level), or be integrated with
computational tools in order to compare and refine ontologies with
live data on how the brain seems to deal with language.
7/31/2019 Issues on Language Processing
44/46
Daniele Virgillito 44
Bibliography
Chomsky, N., 1957. Syntactic Structures. Mouton & Co., The Hague/Paris.
Chomsky, N. (1995). The Minimalist Program. Cambridge, Mass: MIT Press.
Chwilla, D. J., Brown, C. M., & Hagoort, P. (1995). The N400 as a function of
the level of processing. Psychophysiology, 32(3), 274285.
Bever, T.G., Sanz, M., Townsend, D.J., 1998. The emperors psycholinguistics.
Journal of Psycholinguistic Research 27 (2), 261284.
Evans, V., Benjamin Bergen & Joerg Zinken, 2007. The Cognitive Linguistics
Reader. London: Equinox.
Federmeier, K. D., Kirson, D. A., Moreno, E. M., and Kutas, M. (2001). Effects
of transient, mild mood states on semantic memory organization and use: An
event-related potential investigation in humans. Neuroscience Letters,
305, 149-152.
Fodor, J., 2003. The modularity of mind. An essay on faculty psychology,
Cambridge, Mass: MIT Press.
Friederici, A., Bahlmann, J., Heim, S., Schubotz, R.I., Anwander, A., 2006. The
brain differentiates human and non-human grammars: Functional
localization and structural connectivity, in PNAS, 103 (7), 2458-2463.
Hagoort, P., Brown, C., Groothusen, J., 1993. The syntactic positive shift (SPS)
as an ERP measure of syntactic processing. In: Garnsey, S.M. (Ed.),
7/31/2019 Issues on Language Processing
45/46
Issues on language processing: from theory to brain investigations 45
Language and Cognitive Processes. Special Issue: Event-Related Brain
Potentials in the Study of Language, vol. 8(4). Lawrence Erlbaum
Associates, Hove, pp. 439483Hagoort, P., Hald, L., Bastiaansen, M., Petersson, K.M., 2004. Integration of
word meaning and world knowledge in language comprehension. Science
304, 438441.
Kirson, D.A., 1990. The Effect of Mood on Access to Remote Category Associates ,
Department of Psychology, University of Denver.Kolk, H.H., Chwilla, D.J., van Herten, M., Oor, P.J., 2003. Structure and limited
capacity in verbal working memory: a study with event-related potentials .
Brain Lang. 85, 136.
Kolk, H.H., Chwilla, D.J., 2007. Late Positivities in unusual situations.
Brain Lang. 100, 257262.
Kutas, M., Hillyard, S.A., 1980. Reading senseless sentences: brain potentials
reflect semantic incongruity. Science 207, 203205.
Kutas, M., Hillyard, S.A., 1984. Brain potentials during reading reflect word
expectancy and semantic association. Nature 307, 161163.
Kutas, M., Van Petten, C.K.F., 1994. Psycholinguistics electrified. Event-related
brain potential investigations. In: Gernsbacher, M.A. (Ed.), Handbook of
Psycholinguistics. Academic Press, San Diego, CA, pp. 83143.
MacDonald, Maryellen C., Pearlmutter, Neal, and Seidenberg, Mark S., 1994.
Lexical Nature of Syntactic Ambiguity Resolution. Psychological Review
101:676-703.
7/31/2019 Issues on Language Processing
46/46
Daniele Virgillito 46
Mazzone, M., in press. Autonomia del linguistico? Alcune tendenze delle
neuroscienze attuali, in Atti del II Convegno CODISCO
(Coordinamento Dottorati Italiani Scienze Cognitive).
Osterhout, L., Holcomb, P.J., 1992. Event-related potentials elicited by syntactic
anomaly. J. Mem. Lang. 31, 785806.
Slobin, D. I., 1966. Grammatical transformations and sentence comprehension in
childhood and adulthood. Journal of Verbal Learning and Verbal
Behavior, 5, 219-227.
Trueswell, J. C. & Tanenhaus, M. K., 1994. Toward a lexicalist framework for
constraint-based syntactic ambiguity resolution. In Clifton, C., Frazier, L.
and Rayner, K. (Eds.) Perspectives in Sentence Processing. Lawrence
Erlbaum Assoc.: Hillsdale, NJ, pp. 155-179.
Van Herten, M., Kolk, H.H.J., Chwilla, D.J., 2005. An ERP study of P600 effects
elicited by semantic anomalies. Cogn. Brain Res. 22, 241255.
Van Herten, M., Kolk, H. H. J., & Chwilla, D. (2005). An ERP study of
P600 effects elicited by semantic anomalies. Cognitive Brain Research, 22,
241255.
Vissers, C.T., Chwilla, D. J., & Kolk, H. H. J., 2006. Monitoring in Language
Perception: the effect of misspellings of words in highly constrained
sentences. Brain Research.
Vissers C.T., Kolk H.H., van de Meerendonk N., Chwilla D.J., 2008.
Monitoring in language perception: evidence from ERPs in a picture-