-
Edinburgh Research Explorer
Universal Semantic ParsingCitation for published version:Reddy,
S, Täckström, O, Petrov, S, Steedman, M & Lapata, M 2017,
Universal Semantic Parsing. inProceedings of the 2017 Conference on
Empirical Methods in Natural Language Processing. Association
forComputational Linguistics, Copenhagen, Denmark , pp. 89–101,
EMNLP 2017: Conference on EmpiricalMethods in Natural Language
Processing, Copenhagen, Denmark,
7/09/17.https://doi.org/10.18653/v1/D17-1009
Digital Object Identifier (DOI):10.18653/v1/D17-1009
Link:Link to publication record in Edinburgh Research
Explorer
Document Version:Peer reviewed version
Published In:Proceedings of the 2017 Conference on Empirical
Methods in Natural Language Processing
General rightsCopyright for the publications made accessible via
the Edinburgh Research Explorer is retained by the author(s)and /
or other copyright owners and it is a condition of accessing these
publications that users recognise andabide by the legal
requirements associated with these rights.
Take down policyThe University of Edinburgh has made every
reasonable effort to ensure that Edinburgh Research Explorercontent
complies with UK legislation. If you believe that the public
display of this file breaches copyright pleasecontact
[email protected] providing details, and we will remove access to
the work immediately andinvestigate your claim.
Download date: 04. Apr. 2021
https://doi.org/10.18653/v1/D17-1009https://doi.org/10.18653/v1/D17-1009https://www.research.ed.ac.uk/portal/en/publications/universal-semantic-parsing(09d7a4a4-486a-4fdd-870c-4075f9bd6acf).html
-
Universal Semantic Parsing
Siva Reddy† Oscar Täckström‡ Slav Petrov‡ Mark Steedman†
Mirella Lapata††ILCC, School of Informatics, University of
Edinburgh
‡ Google, [email protected], {oscart, slav}@google.com,
{steedman, mlap}@inf.ed.ac.uk
Abstract
Universal Dependencies (UD) provides across-linguistically
uniform syntactic rep-resentation, with the aim of advancing
mul-tilingual applications of parsing and natu-ral language
understanding. Reddy et al.(2016) recently developed a semantic
in-terface for (English) Stanford Dependen-cies, based on the
lambda calculus. In thiswork, we introduce UDEPLAMBDA, a simi-lar
semantic interface for UD, which allowsmapping natural language to
logical formsin an almost language-independent frame-work. We
evaluate our approach on seman-tic parsing for the task of question
answer-ing against Freebase. To facilitate multilin-gual
evaluation, we provide German andSpanish translations of the
WebQuestionsand GraphQuestions datasets. Results showthat
UDEPLAMBDA outperforms strongbaselines across languages and
datasets.For English, it achieves the strongest resultto date on
GraphQuestions, with competi-tive results on WebQuestions.
1 Introduction
The Universal Dependencies (UD) initiative seeksto develop
cross-linguistically consistent annota-tion guidelines as well as a
large number of uni-formly annotated treebanks for many
languages.1
Such resources could advance multilingual applica-tions of
parsing, improve comparability of evalua-tion results, enable
cross-lingual learning, and moregenerally support natural language
understanding.
Seeking to exploit the benefits of UD for natu-ral language
understanding, we introduce UDEP-LAMBDA, a semantic interface for
UD that maps
1http://www.universaldependencies.org/.
natural language to logical forms, representing un-derlying
predicate-argument structures, in an al-most language-independent
manner. Our frame-work is based on DEPLAMBDA (Reddy et al., 2016)a
recently developed method that converts EnglishStanford
Dependencies (SD) to logical forms. Theconversion process is
illustrated in Figure 1 and dis-cussed in more detail in Section 2.
Whereas DEP-LAMBDA works only for English, UDEPLAMBDAapplies to any
language for which UD annotationsare available.2 In this paper, we
describe the ra-tionale behind UDEPLAMBDA and highlight im-portant
differences from DEPLAMBDA, some ofwhich stem from the different
treatment of variouslinguistic constructions in UD.
Our experiments focus on semantic parsing as atestbed for
evaluating the framework’s multilingualappeal. We address the task
of learning to mapnatural language to machine interpretable
formalmeaning representations, specifically retrieving an-swers to
questions from Freebase. To facilitatemultilingual evaluation, we
provide translationsof the English WebQuestions (Berant et al.,
2013)and GraphQuestions (Su et al., 2016) datasets toGerman and
Spanish. We demonstrate that U-DEPLAMBDA can be used to derive
logical formsfor these languages using a minimal amount
oflanguage-specific knowledge. Aside from devel-oping the first
multilingual semantic parsing toolfor Freebase, we also
experimentally show that U-DEPLAMBDA outperforms strong baselines
acrosslanguages and datasets. For English, it achievesthe strongest
result to date on GraphQuestions,with competitive results on
WebQuestions. Be-yond semantic parsing, we believe that the
log-ical forms produced by our framework will beof use in various
natural understanding tasks in-cluding entailment (Beltagy et al.,
2016), text-
2As of v1.3, UD annotations are available for 47 languages.
arX
iv:1
702.
0319
6v2
[cs
.CL
] 1
5 Fe
b 20
17
mailto:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]://www.universaldependencies.org/
-
based question answering (Lewis and Steedman,2013), sentence
simplification (Narayan and Gar-dent, 2014), summarization (Liu et
al., 2015), para-phrasing (Pavlick et al., 2015), and relation
extrac-tion (Rocktäschel et al., 2015). Our implementa-tion and
translated datasets are publicly availableat
https://github.com/sivareddyg/udeplambda.
2 DEPLAMBDA
Before describing UDEPLAMBDA, we provide anoverview of DEPLAMBDA
(Reddy et al., 2016) onwhich our approach is based. DEPLAMBDA
con-verts a dependency tree to its logical form in threesteps:
binarization, substitution, and composition,each of which is
briefly outlined below.
Binarization A dependency tree is first mappedto a Lisp-style
s-expression indicating the orderof semantic composition. Figure
1(b) shows thes-expression for the sentence Disney won an Os-car
for the movie Frozen, derived from the depen-dency tree in Figure
1(a). Here, the sub-expression(dobj won (det Oscar an)) indicates
that the logi-cal form of the phrase won an Oscar is derived
bycomposing the logical form of the label dobj withthe logical form
of the word won and the logicalform of the phrase an Oscar, derived
analogously.
An obliqueness hierarchy is employed to imposea strict ordering
on the modifiers to each head inthe dependency tree. As an example,
won hasthree modifiers in Figure 1(a), which accordingto the
obliqueness hierarchy are composed in theorder dobj> nmod>
nsubj. In constructions likecoordination, this ordering is crucial
to arrive at thecorrect semantics (see Section 3.3).
Substitution Each symbol in the s-expressionsis substituted for
a lambda expression encodingits semantics. Words and dependency
labels areassigned different types of expressions. In general,words
have expressions of the following kind:ENTITY ⇒ λx.word(xa); e.g.
Oscar⇒ λx.Oscar(xa)EVENT ⇒ λx.word(xe); e.g. won⇒
λx.won(xe)FUNCTIONAL⇒ λx.TRUE; e.g. an⇒ λx.TRUE
Here, the subscripts ·a and ·e denote the types ofindividuals
(Ind) and events (Event), respectively,whereas x denotes a paired
variable (xa,xe) of typeInd×Event. Roughly speaking, proper nouns
andadjectives invoke ENTITY expressions, verbs andadverbs invoke
EVENT expressions, and commonnouns invoke both ENTITY and EVENT
expressions(see Section 3.3), while remaining words invoke
Disney won an Oscar for the movie Frozenpropn verb det propn adp
det noun propn
nsubj
dobj
nmod
det
case
det
compound
root
(a) The dependency tree for Disney won an Oscar for themovie
Frozen in the Universal Dependencies formalism.
(nsubj (nmod (dobj won (det Oscar an))(case (det (comp. Frozen
movie) the) for)) Disney)
(b) The binarized s-expression for the dependency tree.
λx.∃yzw.won(xe)∧Disney(ya)∧Oscar(za)∧Frozen(wa)∧
movie(wa)∧arg1(xe,ya)∧ arg2(xe,za)∧ nmod.for(xe,wa)
(c) The composed lambda-calculus expression.
Figure 1: The mapping of a dependency tree to itslogical form
with the intermediate s-expression.
FUNCTIONAL expressions. As in DEPLAMBDA,we enforce the
constraint that every s-expressionis of the type η = Ind×Event →
Bool, whichsimplifies the type system considerably.
Expressions for dependency labels glue thesemantics of heads and
modifiers to articulatepredicate-argument structure. These
expressions ingeneral take one of the following forms:COPY ⇒ λ f
gx.∃y. f (x)∧g(y)∧ rel(x,y)e.g. nsubj, dobj, nmod, advmodINVERT ⇒ λ
f gx.∃y. f (x)∧g(y)∧ reli(y,x)e.g. amod, aclMERGE ⇒ λ f gx. f
(x)∧g(x)e.g. compound, appos, amod, aclHEAD ⇒ λ f gx. f (x)e.g.
case, punct, aux, mark .
As an example of COPY, consider the lambdaexpression for dobj in
(dobj won (det Oscar an)):λ f gx.∃y. f (x)∧ g(y)∧ arg2(xe,ya). This
expres-sion takes two functions f and g as input, wheref represents
the logical form of won and g repre-sents the logical form of an
Oscar. The predicate-argument structure arg2(xe,ya) indicates that
thearg2 of the event xe, i.e. won, is the individual ya,i.e. the
entity Oscar. Since arg2(xe,ya) mimics thedependency structure
dobj(won, Oscar), we referto the expression kind evoked by dobj as
COPY.
Expressions that invert the dependency direc-tion are referred
to as INVERT (e.g. amod in run-ning horse); expressions that merge
two subexpres-sions without introducing any relation predicatesare
referred to as MERGE (e.g. compound in movieFrozen); and
expressions that simply return the par-
https://github.com/sivareddyg/udeplambda
-
ent expression semantics are referred to as HEAD(e.g. case in
for Frozen). While this generalizationapplies to most dependency
labels, several labelstake a different logical form not listed
here, someof which are discussed in Section 3.3. 3 Sometimesthe
mapping of dependency label to lambda expres-sion may depend on
surrounding part-of-speechtags or dependency labels. For example,
amod actsas INVERT when the modifier is a verb (e.g. in run-ning
horse), and as MERGE when the modifier isan adjective (e.g. in
beautiful horse).4
Composition The final logical form is computedby beta-reduction,
treating expressions of the form(f x y) as the function f applied
to the argumentsx and y. For example, (dobj won (det Oscar
an))results in λx.∃z.won(xe)∧Oscar(za)∧ arg2(xe,za)when the
expression for dobj is applied to thosefor won and (det Oscar an).
Figure 1(c) shows thelogical form for the s-expression in Figure
1(b).
3 UDEPLAMBDA
We now introduce UDEPLAMBDA, a semantic in-terface for Universal
Dependencies.5 WhereasDEPLAMBDA only applies to English Stanford
De-pendencies, UDEPLAMBDA takes advantage of thecross-lingual
nature of UD to facilitate an (almost)language independent semantic
interface. This isaccomplished by restricting the binarization,
sub-stitution, and composition steps described aboveto rely solely
on information encoded in the UDrepresentation. Importantly,
UDEPLAMBDA is de-signed to not rely on lexical forms in a language
toassign lambda expressions, but only on informationcontained in
dependency labels and postags.
However, some linguistic phenomena are lan-guage specific (e.g.
pronoun-dropping) or mean-ing specific (e.g. every and the in
English havevery different semantics, despite being both
deter-miners) and are not encoded in the UD schema.Furthermore,
some cross-linguistic phenomena,such as long-distance dependencies,
are not part ofthe core UD representation. To circumvent
thislimitation, a simple enhancement step enrichesthe original UD
representation before binariza-
3Mappings are available at
https://github.com/sivareddyg/udeplambda.
4We use Tregex (Levy and Andrew, 2006) for substitu-tion
mappings and Cornell SPF (Artzi, 2013) as the lambda-calculus
implementation. For example, in running horse, thetregex
/label:amod/=target < /postag:verb/ matches amod toits INVERT
expression λ f gx.∃y. f (x)∧g(y)∧ amodi(ye,xa).
5In what follows, all references to UD are to UD v1.3.
Anna wants to marry Kristoff
nsubj
xcomp
mark dobj
nsubj
(a) With long-distance dependency.
Anna wants to marry Kristoff
Ω Ω
nsubj
xcomp
mark dobj
bind nsubj
(b) With variable binding.
Figure 2: The original and enhanced dependencytrees for Anna
wants to marry Kristoff.
tion takes place (Section 3.1). This step adds tothe dependency
tree missing syntactic informationand long-distance dependencies,
thereby creating agraph. Whereas DEPLAMBDA is not able to han-dle
graph-structured input, UDEPLAMBDA is de-signed to work directly
with dependency graphs(Section 3.2). Finally, the representation of
severallinguistic constructions differ between UD and SD,which
requires different handling in the semanticinterface (Section
3.3).
3.1 Enhancement
Both Schuster and Manning (2016) and Nivre et al.(2016) note the
necessity of an enhanced UD rep-resentation to enable semantic
applications. How-ever, such enhancements are currently only
avail-able for a subset of languages in UD. Instead, werely on a
small number of enhancements for ourmain application—semantic
parsing for question-answering—with the hope that this step can be
re-placed by an enhanced UD representation in the fu-ture.
Specifically, we define three kinds of enhance-ments: (1)
long-distance dependencies; (2) typesof coordination; and (3)
refined question word tags.
First, we identify long-distance dependencies inrelative clauses
and control constructions. We fol-low Schuster and Manning (2016)
and find theseusing the labels acl (relative) and xcomp
(control).Figure 2(a) shows the long-distance dependency inthe
sentence Anna wants to marry Kristoff. Here,marry is provided with
its missing nsubj (dashedarc). Second, UD conflates all
coordinating con-structions to a single dependency label, conj.
Toobtain the correct coordination scope, we refineconj to
conj:verb, conj:vp, conj:sentence,
https://github.com/sivareddyg/udeplambdahttps://github.com/sivareddyg/udeplambda
-
conj:np, and conj:adj, similar to Reddy et al.(2016). Finally,
unlike the PTB tags (Marcus et al.,1993) used by SD, the UD
part-of-speech tags donot distinguish question words. Since these
are cru-cial to question-answering, we use a small lexiconto refine
the tags for determiners (DET), adverbs(ADV) and pronouns (PRON) to
DET:WH, ADV:WHand PRON:WH, respectively. Specifically, we usea list
of 12 (English), 14 (Spanish) and 35 (Ger-man) words, respectively.
This is the only partof UDEPLAMBDA that relies on
language-specificinformation. We hope that, as the coverage of
mor-phological features in UD improves, this refine-ment can be
replaced by relying on morphologicalfeatures, such as the
interrogative feature (INT).
3.2 Graph Structures and BIND
To handle graph structures that may result fromthe enhancement
step, such as those in Figure 2(a),we propose a variable-binding
mechanism that dif-fers from that of DEPLAMBDA. First, each
long-distance dependency is split into independent arcsas shown in
Figure 2(b). Here, Ω is a placeholderfor the subject of marry,
which in turn correspondsto Anna as indicated by the binding of Ω
via thepseudo-label BIND. We treat BIND like an ordinarydependency
label with semantics MERGE and pro-cess the resulting tree as
usual, via the s-expression:(nsubj (xcomp wants (nsubj (mark
(dobj marry Kristoff) to) Ω) (BIND Anna Ω)) ,
with the lambda-expression substitutions:wants, marry ∈ EVENT;
to ∈ FUNCTIONAL;Anna, Kristoff ∈ ENTITY;mark ∈ HEAD; BIND ∈
MERGE;xcomp = λ f gx.∃y. f (x)∧g(y)∧xcomp(xe,ye) .
These substitutions are based solely on unlexi-calized context.
For example, the part-of-speechtag PROPN of Anna invokes an ENTITY
expression.
The placeholder Ω has semantics λx.EQ(x,ω),where EQ(u,ω) is true
iff u and ω are equal (havethe same denotation), which unifies the
subject vari-able of wants with the subject variable of marry.
After substitution and composition, we
get:λz.∃xywv.wants(ze)∧Anna(xa)∧ arg1(ze,xa)∧
EQ(x,ω)∧marry(ye)∧xcomp(ze,ye)∧ arg1(ye,va)∧ EQ(v,ω)∧ Kristoff(wa)∧
arg2(ye,wa) ,
This expression may be simplified further byreplacing all
occurrences of v with x and removingthe unification predicates EQ,
which results in:λz.∃xyw.wants(ze)∧Anna(xa)∧
arg1(ze,xa)∧marry(ye)∧xcomp(ze,ye)∧ arg1(ye,xa)∧ Kristoff(wa)∧
arg2(ye,wa) .
This expression encodes the fact that Anna is thearg1 of the
marry event, as desired. DEPLAMBDA,in contrast, cannot handle
graph-structured input,since it lacks a principled way of
generating s-expressions from graphs. Even given the
aboves-expression, BIND in DEPLAMBDA is defined ina way such that
the composition fails to unify vand x, which is crucial for the
correct semantics.Moreover, the definition of BIND in DEPLAMBDAdoes
not have a formal interpretation within thelambda calculus, unlike
ours.
3.3 Linguistic ConstructionsBelow, we highlight the most
pertinent differencesbetween UDEPLAMBDA and DEPLAMBDA, stem-ming
from the different treatment of various lin-guistic constructions
in UD versus SD.
Prepositional Phrases UD uses a content-headanalysis, in
contrast to SD, which treats functionwords as heads of
prepositional phrases, Accord-ingly, the s-expression for the
phrase presidentin 2009 is (nmod president (case 2009 in)) in
U-DEPLAMBDA and (prep president (pobj in 2009))in DEPLAMBDA. To
achieve the desired semantics,λx.∃y.president(xa)∧president
event(xe)∧
arg1(xe,xa)∧2009(ya)∧prep.in(xe,ya) ,
DEPLAMBDA relies on an intermediate logicalform that requires
some post-processing, whereasUDEPLAMBDA obtains the desired logical
formdirectly through the following entries:in ∈ FUNCTIONAL; 2009 ∈
ENTITY; case ∈ HEAD;president = λx.president(xa)∧president
event(xe)
∧arg1(xe,xa) ;nmod = λ f gx.∃y. f (x)∧g(y)∧nmod.in(xe,ya) .
Other nmod constructions, such as possessives(nmod:poss),
temporal modifiers (nmod:tmod)and adverbial modifiers (nmod:npmod),
are han-dled similarly. Note how the common noun presi-dent, evokes
both entity and event predicates above.
Passives DEPLAMBDA gives special treatmentto passive verbs,
identified by the fine-grained part-of-speech tags in the PTB tag
together with de-pendency context. For example, An Oscar waswon is
analyzed as λx.won.pass(xe)∧Oscar(ya)∧arg1(xe,ya), where won.pass
represents a passiveevent. However, UD does not distinguish
be-tween active and passive forms.6 While the labelsnsubjpass or
auxpass indicate passive construc-tions, such clues are sometimes
missing, such as in
6UD encodes voice as a morphological feature, but mostsyntactic
analyzers do not produce this information yet.
-
language target people
x e1 y e2 Ghanaspeak.arg2
speak.arg1
people.arg1
people.nmod.in
type
type
(a) English
sprache target
x e1 Ghanagesprochen
.arg2gesprochen.nmod.in
type
(b) German
lengua target
x e1 Ghanalengua.arg1
lengua.nmod.de
type
(c) Spanish
language.human language
target
x m Ghanalocation.country
.official language.2location.country
.official language.1
type
(d) Freebase
Figure 3: The ungrounded graphs for What language do the people
in Ghana speak?, Welche Sprachewird in Ghana gesprochen? and Cuál
es la lengua de Ghana?, and the corresponding grounded graph.
reduced relatives. We therefore opt to not have sep-arate
entries for passives, but aim to produce identi-cal logical forms
for active and passive forms whenpossible (for example, by treating
nsubjpass asdirect object). With the following entries,
won ∈ EVENT; an, was ∈ FUNCTIONAL; auxpass ∈ HEAD;nsubjpass = λ
f gx.∃y. f (x)∧g(y)∧ arg2(xe,ya) ,
the lambda expression for An Oscar was won be-comes
λx.won(xe)∧Oscar(ya)∧arg2(xe,ya), iden-tical to that of its active
form. However, not havinga special entry for passive verbs may have
unde-sirable side-effects. For example, in the reduced-relative
construction Pixar claimed the Oscar wonfor Frozen, the phrase the
Oscar won ... willreceive the semantics
λx.Oscar(ya)∧won(xe)∧arg1(xe,ya), which differs from that of an
Oscarwas won. We leave it to the target application todisambiguate
the interpretation in such cases.
Long-Distance Dependencies As discussed inSection 3.2, we handle
long-distance dependen-cies evoked by clausal modifiers (acl) and
con-trol verbs (xcomp) with the BIND mechanism,whereas DEPLAMBDA
cannot handle control con-structions. For xcomp, as seen earlier,
we use themapping λ f gx.∃y. f (x)∧g(y)∧xcomp(xe,ye). Foracl we use
λ f gx.∃y. f (x)∧ g(y), to conjoin themain clause and the modifier
clause. However, notall acl clauses evoke long-distance
dependencies,e.g. in the news that Disney won an Oscar, theclause
that Disney won an Oscar is a subordinatingconjunction of news. In
such cases, we insteadassign acl the INVERT semantics.
Questions Question words are marked with theenhanced
part-of-speech tags DET:WH, ADV:WHand PRON:WH, which are all
assigned the seman-tics λx.${word}(xa)∧ TARGET(xa). The
predicateTARGET indicates that xa represents the variable
ofinterest, that is the answer to the question.
3.4 Limitations
In order to achieve language independence, UDEP-LAMBDA has to
sacrifice semantic specificity, sincein many cases the semantics is
carried by lexicalinformation. Consider the sentences John broke
thewindow and The window broke. Although it is thewindow that broke
in both cases, our inferred logi-cal forms do not canonicalize the
relation betweenbroke and window. To achieve this, we would haveto
make the substitution of nsubj depend on lexicalcontext, such that
when window occurs as nsubjwith broke, the predicate arg2 is
invoked rather thanarg1. We do not address this problem, and leave
itto the target application to infer that arg2 and arg1have the
same semantic function in these cases. Weanticipate that the
ability to make such lexicalizedsemantic inferences in a
task-agnostic cross-lingualframework would be highly useful and a
crucialavenue for future work on universal semantics.
Other constructions that require lexical infor-mation are
quantifiers like every, some and most,negation markers like no and
not, and intentionalverbs like believe and said. UD does not have
spe-cial labels to indicate these. Although not
currentlyimplemented, we discuss how to handle quantifiersin this
framework in the supplementary material.Fancellu et al. (2017) is a
first step in this direction.
4 Cross-lingual Semantic Parsing
To study the multilingual nature of UDEPLAMBDA,we conduct an
empirical evaluation on questionanswering against Freebase in three
different lan-guages: English, Spanish, and German.
Beforediscussing the details of this experiment, we brieflyoutline
the semantic parsing framework employed.
4.1 Semantic Parsing as Graph Matching
UDEPLAMBDA generates ungrounded logicalforms that are
independent of any knowledge base,such as Freebase. We use
GRAPHPARSER (Reddyet al., 2016) to map these logical forms to
their
-
WebQuestions
en What language do the people in Ghana speak?de Welche Sprache
wird in Ghana gesprochen?es ¿Cuál es la lengua de Ghana?
en Who was Vincent van Gogh inspired by?de Von wem wurde Vincent
van Gogh inspiriert?es ¿Qué inspiró a Van Gogh?
GraphQuestions
en NASA has how many launch sites?de Wie viele Abschussbasen
besitzt NASA?es ¿Cuántos sitios de despegue tiene NASA?
en Which loudspeakers are heavier than 82.0 kg?de Welche
Lautsprecher sind schwerer als 82.0 kg?es ¿Qué altavoces pesan
más de 82.0 kg?
Table 1: Example questions and their translations.
grounded Freebase graphs, via corresponding un-grounded graphs.
Figures 3(a) to 3(c) show theungrounded graphs corresponding to
logical formsfrom UDEPLAMBDA, each grounded to the sameFreebase
graph in Figure 3(d). Here, rectangles de-note entities, circles
denote events, rounded rectan-gles denote entity types, and edges
between eventsand entities denote predicates or Freebase
relations.Finally, the TARGET node represents the set of val-ues of
x that are consistent with the Freebase graph,that is the answer to
the question.
GRAPHPARSER treats semantic parsing as agraph-matching problem
with the goal of findingthe Freebase graphs that are structurally
isomorphicto an ungrounded graph and rank them accordingto a model.
To account for structural mismatches,GRAPHPARSER uses two graph
transformations:CONTRACT and EXPAND. In Figure 3(a) there aretwo
edges between x and Ghana. CONTRACT col-lapses one of these edges
to create a graph isomor-phic to Freebase. EXPAND, in contrast,
adds edgesto connect the graph in the case of
disconnectedcomponents. The search space is explored by beamsearch
and model parameters are estimated withthe averaged structured
perceptron (Collins, 2002)from training data consisting of
question-answerpairs, using answer F1-score as the objective.
4.2 Datasets
We evaluate our approach on two public bench-marks of question
answering against Freebase:WebQuestions (Berant et al., 2013), a
widely usedbenchmark consisting of English questions andtheir
answers, and GraphQuestions (Su et al., 2016),a recently released
dataset of English questions
with both their answers and grounded logical forms.While
WebQuestions is dominated by simple entity-attribute questions,
GraphQuestions contains alarge number of compositional questions
involvingaggregation (e.g. How many children of EddardStark were
born in Winterfell? ) and comparison(e.g. In which month does the
average rainfall ofNew York City exceed 86 mm? ). The number
oftraining, development and test questions is 2644,1134, and 2032,
respectively, for WebQuestionsand 1794, 764, and 2608 for
GraphQuestions.
To support multilingual evaluation, we createdtranslations of
WebQuestions and GraphQuestionsto German and Spanish.7 For
WebQuestions twoprofessional annotators were hired per
language,while for GraphQuestions we used a trusted pool of20
annotators per language (with a single annotatorper question).
Examples of the original questionsand their translations are
provided in Table 1.
4.3 Implementation Details
Here we provide details on the syntactic analyzersemployed, our
entity resolution algorithm, and thefeatures used by the grounding
model.
Dependency Parsing The English, Spanish, andGerman Universal
Dependencies (UD) treebanks(v1.3; Nivre et al 2016) were used to
train part ofspeech taggers and dependency parsers. We used
abidirectional LSTM tagger (Plank et al., 2016) anda bidirectional
LSTM shift-reduce parser (Kiper-wasser and Goldberg, 2016). Both
the tagger andparser require word embeddings. For English, weused
GloVe embeddings (Pennington et al., 2014)trained on Wikipedia and
the Gigaword corpus.8
For German and Spanish, we used SENNA em-beddings (Collobert et
al., 2011; Al-Rfou et al.,2013) trained on Wikipedia corpora (589M
wordsGerman; 397M words Spanish).9 Measured on theUD test sets, the
tagger accuracies are 94.5 (En-glish), 92.2 (German), and 95.7
(Spanish), withcorresponding labeled attachment parser scores
of81.8, 74.7, and 82.2.
Entity Resolution We follow Reddy et al. (2016)and resolve
entities in three steps: (1) potential en-tity spans are identified
using seven handcraftedpart-of-speech patterns; (2) each span is
associatedwith potential Freebase entities according to the
7Translations will be publicly released upon
publication.8http://nlp.stanford.edu/projects/glove/.9https://sites.google.com/site/rmyeid/projects/polyglot.
http://nlp.stanford.edu/projects/glove/https://sites.google.com/site/rmyeid/projects/polyglot
-
k WebQuestions GraphQuestionsen de es en de es
1 89.6 82.8 86.7 47.2 39.9 39.510 95.7 91.2 94.0 56.9 48.4
51.6
Table 2: Structured perceptron k-best entity linkingaccuracies
on the development sets.
WebQuestions GraphQuestionsMethod en de es en de es
SINGLEEVENT 47.6 43.9 45.0 15.9 8.3 11.2DEPTREE 47.8 43.9 44.5
15.8 7.9 11.0CCGGRAPH 48.4 – – 15.9 – –UDEPLAMBDA 48.3 44.2 45.7
17.6 9.0 12.4
Table 3: F1-scores on the test for models trainedon the training
set (excluding the development set).
Freebase/KG API;10 and (3) the 10-best entity link-ing lattices,
scored by a structured perceptron, areinput to GRAPHPARSER, leaving
the final disam-biguation to the semantic parsing problem. Table
2shows the 1-best and 10-best entity disambiguationF1-scores for
each language and dataset.11
Features We use features similar to Reddy et al.(2016): basic
features of words and Freebase re-lations, and graph features
crossing ungroundedevents with grounded relations, ungrounded
typeswith grounded relations, and ungrounded answertype crossed
with a binary feature indicating if theanswer is a number. In
addition, we add featuresencoding the semantic similarity of
ungroundedevents and Freebase relations. Specifically, we usedthe
cosine similarity of the translation-invariant em-beddings of Huang
et al. (2015).12
4.4 Comparison Systems
We compared UDEPLAMBDA to prior work andthree versions of
GRAPHPARSER that operate ondifferent representations: entity
cliques, depen-dency trees, and CCG-based semantic derivations.
SINGLEEVENT This model resembles thelearning-to-rank model of
Bast and Haussmann(2015). An ungrounded graph is generated by
con-necting all entities in the question with the TARGETnode,
representing a single event. Note that this
10http://developers.google.com/freebase/.11Due to the recent
Freebase API shutdown, we used the
KG API for GraphQuestions. We observed that this leads
toinferior entity linking results compared to those of
Freebase.
12http://128.2.220.95/multilingual/data/.
Method GraphQ. WebQ.
SEMPRE (Berant et al., 2013) 10.8 35.7JACANA (Yao and Van Durme,
2014) 5.1 33.0PARASEMPRE (Berant and Liang, 2014) 12.8 39.9QA (Yao,
2015) – 44.3AQQU (Bast and Haussmann, 2015) – 49.4AGENDAIL (Berant
and Liang, 2015) – 49.7DEPLAMBDA (Reddy et al., 2016) – 50.3
STAGG (Yih et al., 2015) – 48.4 (52.5)BILSTM (Türe and Jojic,
2016) – 24.9 (52.2)MCNN (Xu et al., 2016) – 47.0
(53.3)AGENDAIL-RANK (Yavuz et al., 2016) – 51.6 (52.6)
UDEPLAMBDA 17.6 49.5
Table 4: F1-scores on the English GraphQuestionsand WebQuestions
test sets (results with additionaltask-specific resources in
parentheses). Followingprior work, for WebQuestions the union of
the train-ing and development sets were used for training.
baseline cannot handle compositional questions, orthose with
aggregation or comparison.
DEPTREE An ungrounded graph is obtained di-rectly from the
original dependency tree. An eventis created for each parent and
its dependents in thetree. Each dependent is linked to this event
with anedge labeled with its dependency relation, while theparent
is linked to the event with an edge labeledarg0. If a word is a
question word, an additionalTARGET predicate is attached to its
entity node.
CCGGRAPH This is the CCG-based semanticrepresentation of Reddy
et al. (2014). Note thatthis baseline exists only for English.
4.5 ResultsTable 3 shows the performance of GRAPHPARSERwith
these different representations. Here and inwhat follows, we use
average F1-score of predictedanswers (Berant et al., 2013) as the
evaluation met-ric. We first observe that UDEPLAMBDA consis-tently
outperforms the SINGLEEVENT and DEP-TREE representations in all
languages.13
For English, performance is almost on par withCCGGRAPH, which
suggests that UDEPLAMBDAdoes not sacrifice too much specificity for
univer-sality. With both datasets, results are lower forGerman
compared to Spanish. This agrees withthe lower performance of the
syntactic parser on
13For the DEPTREE model, we CONTRACT each multi-hop path between
the question word and an entity to a singleedge. Without this
constraint, DEPTREE F1 results are 45.5(en), 42.9 (de), and 44.2
(es) on WebQuestions, and 11.0 (en),6.6 (de), and 2.6 (es) on
GraphQuestions.
http://developers.google.com/freebase/http://128.2.220.95/multilingual/data/
-
the German portion of the UD treebank. Finally,while these
results confirm that GraphQuestions ismuch harder compared to
WebQuestions, we notethat both datasets predominantly contain
single-hopquestions, as indicated by the competitive perfor-mance
of SINGLEEVENT on both datasets.
Table 4 compares UDEPLAMBDA with previ-ously published models
which exist only for En-glish and have been mainly evaluated on
Web-Questions. These are either symbolic like ours (firstblock) or
employ neural networks (second block).Results for models using
additional task-specifictraining resources, such as ClueWeb09,
Wikipedia,or SimpleQuestions (Bordes et al., 2015) are shownin
parentheses. On GraphQuestions, we achievea new state-of-the-art
result with a gain of 4.8 F1-points over the previously reported
best result. OnWebQuestions we are 2.1 points below the bestmodel
using comparable resources, and 3.8 pointsbelow the state of the
art. Most related to ourwork is the English-specific system of
Reddy et al.(2016). We attribute the 0.8 point difference in
F1-score to their use of the more fine-grained PTB tagset and
Stanford Dependencies.
5 Related Work
Our work continues the long tradition of buildinglogical forms
from syntactic representations initi-ated by Montague (1973). The
literature is rife withattempts to develop semantic interfaces for
HPSG(Copestake et al., 2005), LFG (Kaplan and Bresnan,1982;
Dalrymple et al., 1995; Crouch and King,2006), TAG (Kallmeyer and
Joshi, 2003; Gardentand Kallmeyer, 2003; Nesson and Shieber,
2006),and CCG (Steedman, 2000; Baldridge and Kruijff,2002; Bos et
al., 2004; Artzi et al., 2015). Unlikeexisting semantic interfaces,
UDEPLAMBDA (likeDEPLAMBDA) uses dependency syntax, taking
ad-vantage of recent advances in multilingual parsing(McDonald et
al., 2013; Nivre et al, 2016).
A common trend in previous work on semanticinterfaces is the
reliance on rich typed feature struc-tures or semantic types
coupled with strong typeconstraints, which can be very informative
but un-avoidably language specific. Creating rich seman-tic types
from dependency trees which lack a typingsystem would be labor
intensive and brittle in theface of parsing errors. Instead,
UDEPLAMBDArelies on generic unlexicalized information presentin
dependency treebanks and uses a simple typesystem (one type for
dependency labels, and one
for words) along with a combinatory mechanism,which avoids type
collisions. Earlier attempts atextracting semantic representations
from depen-dencies have mainly focused on
language-specificdependency representations (Spreyer and
Frank,2005; Simov and Osenova, 2011; Hahn and Meur-ers, 2011; Reddy
et al., 2016; Falke et al., 2016;Beltagy, 2016), and multi-layered
dependency an-notations (Jakob et al., 2010; Bédaride and
Gar-dent, 2011). In contrast, UDEPLAMBDA derivessemantic
representations for multiple languages ina common schema directly
from Universal Depen-dencies. This work parallels a growing
interest increating other forms of multilingual semantic
repre-sentations (Akbik et al., 2015; Vanderwende et al.,2015;
White et al., 2016; Evang and Bos, 2016).
We evaluate UDEPLAMBDA on semantic pars-ing for question
answering against a knowledgebase. Here, the literature offers two
main model-ing paradigms: (1) learning of task-specific gram-mars
that directly parse language to a groundedrepresentation (Zelle and
Mooney, 1996; Zettle-moyer and Collins, 2005; Wong and Mooney,
2007;Kwiatkowksi et al., 2010; Liang et al., 2011; Be-rant et al.,
2013; Flanigan et al., 2014; Pasupat andLiang, 2015; Groschwitz et
al., 2015); and (2) con-verting language to a linguistically
motivated task-independent representation that is then mapped to
agrounded representation (Kwiatkowski et al., 2013;Reddy et al.,
2014; Krishnamurthy and Mitchell,2015; Gardner and Krishnamurthy,
2017). Ourwork belongs to the latter paradigm, as we mapnatural
language to Freebase indirectly via logi-cal forms. Capitalizing on
natural-language syn-tax affords interpretability, scalability, and
reducedduplication of effort across applications (Benderet al.,
2015). Our work also relates to literatureon parsing multiple
languages to a common exe-cutable representation (Cimiano et al.,
2013; Haasand Riezler, 2016). However, existing
approaches(Kwiatkowksi et al., 2010; Jones et al., 2012; Jieand Lu,
2014) still map to the target meaning rep-resentations (more or
less) directly.
6 Conclusions
We introduced UDEPLAMBDA, a semantic inter-face for Universal
Dependencies, and showed thatthe resulting semantic representation
can be usedfor question-answering against a knowledge basein
multiple languages. We provided translations ofbenchmark datasets
in German and Spanish, in the
-
hope to stimulate further multilingual research onsemantic
parsing and question answering in general.We have only scratched
the surface when it comesto applying UDEPLAMBDA to natural
languageunderstanding tasks. In the future, we would like toexplore
how this framework can benefit other taskssuch as summarization and
machine translation.
Acknowledgements
This work greatly benefitted from discussionswith Michael
Collins, Dipanjan Das, FedericoFancellu, Julia Hockenmaier, Tom
Kwiatkowski,Adam Lopez, Valeria de Paiva, Martha Palmer,Fernando
Pereira, Emily Pitler, Vijay Saraswat,Nathan Schneider, Bonnie
Webber, and the mem-bers of ILCC Edinburgh University, the
MicrosoftResearch Redmond NLP group, the Stanford NLPgroup, and the
UW NLP and Linguistics group.The authors would also like to thank
the Univer-sal Dependencies community for the treebanks
anddocumentation. This research is supported by aGoogle PhD
Fellowship to the first author.
ReferencesAlan Akbik, laura chiticariu, Marina Danilevsky,
Yun-
yao Li, Shivakumar Vaithyanathan, and Huaiyu Zhu.2015.
Generating High Quality Proposition Banksfor Multilingual Semantic
Role Labeling. In Pro-ceedings of the 53rd Annual Meeting of the
Associ-ation for Computational Linguistics and the 7th
In-ternational Joint Conference on Natural LanguageProcessing.
Association for Computational Linguis-tics, Beijing, China, pages
397–407.
Rami Al-Rfou, Bryan Perozzi, and Steven Skiena.2013. Polyglot:
Distributed Word Representationsfor Multilingual NLP. In
Proceedings of the Seven-teenth Conference on Computational Natural
Lan-guage Learning. Sofia, Bulgaria, pages 183–192.
Yoav Artzi. 2013. Cornell SPF: Cornell Semantic Pars-ing
Framework. arXiv:1311.3011 [cs.CL] .
Yoav Artzi, Kenton Lee, and Luke Zettlemoyer.
2015.Broad-coverage CCG Semantic Parsing with AMR.In Proceedings of
Empirical Methods on NaturalLanguage Processing. pages
1699–1710.
Jason Baldridge and Geert-Jan Kruijff. 2002. CouplingCCG and
Hybrid Logic Dependency Semantics. InProceedings of Association for
Computational Lin-guistics. pages 319–326.
Hannah Bast and Elmar Haussmann. 2015. More Ac-curate Question
Answering on Freebase. In Pro-ceedings of ACM International
Conference on Infor-
mation and Knowledge Management. pages 1431–1440.
Paul Bédaride and Claire Gardent. 2011. Deep Seman-tics for
Dependency Structures. In Proceedings ofConference on Intelligent
Text Processing and Com-putational Linguistics. pages 277–288.
Islam Beltagy. 2016. Natural Language Semantics Us-ing
Probabilistic Logic. Ph.D. thesis, Departmentof Computer Science,
The University of Texas atAustin.
Islam Beltagy, Stephen Roller, Pengxiang Cheng, Ka-trin Erk, and
Raymond J Mooney. 2016. Represent-ing meaning with a combination of
logical and dis-tributional models. Computational Linguistics .
Emily M. Bender, Dan Flickinger, Stephan Oepen,Woodley Packard,
and Ann Copestake. 2015. Lay-ers of Interpretation: On Grammar and
Composition-ality. In Proceedings of the 11th International
Con-ference on Computational Semantics. Associationfor
Computational Linguistics, London, UK, pages239–249.
Jonathan Berant, Andrew Chou, Roy Frostig, and PercyLiang. 2013.
Semantic Parsing on Freebase fromQuestion-Answer Pairs. In
Proceedings of Empiri-cal Methods on Natural Language Processing.
pages1533–1544.
Jonathan Berant and Percy Liang. 2014. Semantic Pars-ing via
Paraphrasing. In Proceedings of Associationfor Computational
Linguistics. pages 1415–1425.
Jonathan Berant and Percy Liang. 2015. ImitationLearning of
Agenda-Based Semantic Parsers. Trans-actions of the Association for
Computational Lin-guistics 3:545–558.
Antoine Bordes, Nicolas Usunier, Sumit Chopra, andJason Weston.
2015. Large-scale simple ques-tion answering with memory networks.
CoRRabs/1506.02075.
Johan Bos, Stephen Clark, Mark Steedman, James R.Curran, and
Julia Hockenmaier. 2004. Wide-Coverage Semantic Representations
from a CCGParser. In Proceedings of International Conferenceon
Computational Linguistics. pages 1240–1246.
Philipp Cimiano, Vanessa Lopez, Christina Unger,Elena Cabrio,
Axel-Cyrille Ngonga Ngomo, andSebastian Walter. 2013. Multilingual
question an-swering over linked data (QALD-3): Lab overview.In
Information Access Evaluation. Multilinguality,Multimodality, and
Visualization. Springer, Valencia,Spain, volume 8138.
Michael Collins. 2002. Discriminative Training Meth-ods for
Hidden Markov Models: Theory and Exper-iments with Perceptron
Algorithms. In Proceedingsof Empirical Methods on Natural Language
Process-ing. pages 1–8.
-
Ronan Collobert, Jason Weston, Leon Bottou, MichaelKarlen, Koray
Kavukcuoglu, and Pavel Kuks. 2011.Natural language processing
(almost) from scratch.Journal of Machine Learning Research
12:2493–2537.
Ann Copestake, Dan Flickinger, Carl Pollard, andIvan A. Sag.
2005. Minimal Recursion Semantics:An Introduction. Research on
Language and Com-putation 3(2-3):281–332.
Dick Crouch and Tracy Holloway King. 2006. Seman-tics via
f-structure rewriting. In Proceedings of theLFG’06 Conference. CSLI
Publications, page 145.
Mary Dalrymple, John Lamping, Fernando C. N.Pereira, and Vijay
A. Saraswat. 1995. Linear Logicfor Meaning Assembly. In Proceedings
of Computa-tional Logic for Natural Language Processing.
Kilian Evang and Johan Bos. 2016. Cross-lingualLearning of an
Open-domain Semantic Parser. InProceedings of COLING 2016, the 26th
Interna-tional Conference on Computational Linguistics.The COLING
2016 Organizing Committee, Osaka,Japan, pages 579–588.
Tobias Falke, Gabriel Stanovsky, Iryna Gurevych, andIdo Dagan.
2016. Porting an Open Information Ex-traction System from English
to German. In Pro-ceedings of the 2016 Conference on Empirical
Meth-ods in Natural Language Processing. Associationfor
Computational Linguistics, Austin, Texas, pages892–898.
Federico Fancellu, Siva Reddy, Adam Lopez, and Bon-nie Webber.
2017. Universal Dependencies to Logi-cal Forms with Negation Scope.
arXiv Preprint .
Jeffrey Flanigan, Sam Thomson, Jaime Carbonell,Chris Dyer, and
Noah A. Smith. 2014. A Discrimi-native Graph-Based Parser for the
Abstract MeaningRepresentation. In Proceedings of Association
forComputational Linguistics. pages 1426–1436.
Claire Gardent and Laura Kallmeyer. 2003. SemanticConstruction
in Feature-based TAG. In Proceedingsof European Chapter of the
Association for Compu-tational Linguistics. pages 123–130.
Matt Gardner and Jayant Krishnamurthy. 2017. Open-Vocabulary
Semantic Parsing with both Distribu-tional Statistics and Formal
Knowledge. In Proceed-ings of Association for the Advancement of
ArtificialIntelligence.
Jonas Groschwitz, Alexander Koller, and Christoph Te-ichmann.
2015. Graph parsing with s-graph gram-mars. In Proceedings of
Association for Computa-tional Linguistics. pages 1481–1490.
Carolin Haas and Stefan Riezler. 2016. A Corpus andSemantic
Parser for Multilingual Natural LanguageQuerying of OpenStreetMap.
In Proceedings ofthe North American Chapter of the Association
forComputational Linguistics: Human Language Tech-nologies.
Association for Computational Linguistics,San Diego, California,
pages 740–750.
Michael Hahn and Detmar Meurers. 2011. On deriv-ing semantic
representations from dependencies: Apractical approach for
evaluating meaning in learnercorpora. In Proceedings of the Int.
Conference onDependency Linguistics (Depling 2011). Barcelona,pages
94–103.
Kejun Huang, Matt Gardner, Evangelos Papalex-akis, Christos
Faloutsos, Nikos Sidiropoulos, TomMitchell, Partha P. Talukdar, and
Xiao Fu. 2015.Translation Invariant Word Embeddings. In
Pro-ceedings of the 2015 Conference on Empirical Meth-ods in
Natural Language Processing. Lisbon, Portu-gal, pages
1084–1088.
Max Jakob, Markéta Lopatková, and Valia Kordoni.2010. Mapping
between Dependency Structures andCompositional Semantic
Representations. In Pro-ceedings of the Fifth International
Conference onLanguage Resources and Evaluation.
Zhanming Jie and Wei Lu. 2014. Multilingual Seman-tic Parsing :
Parsing Multiple Languages into Se-mantic Representations. In
Proceedings of Inter-national Conference on Computational
Linguistics.Dublin City University and Association for
Com-putational Linguistics, Dublin, Ireland, pages 1291–1301.
Bevan Keeley Jones, Mark Johnson, and Sharon Gold-water. 2012.
Semantic Parsing with Bayesian TreeTransducers. In Proceedings of
the 50th AnnualMeeting of the Association for Computational
Lin-guistics. Association for Computational
Linguistics,Stroudsburg, PA, USA, pages 488–496.
Laura Kallmeyer and Aravind Joshi. 2003. Factor-ing predicate
argument and scope semantics: Under-specified semantics with LTAG.
Research on Lan-guage and Computation 1(1-2):3–58.
Ronald M Kaplan and Joan Bresnan. 1982. Lexical-functional
grammar: A formal system for gram-matical representation. Formal
Issues in Lexical-Functional Grammar pages 29–130.
Eliyahu Kiperwasser and Yoav Goldberg. 2016. Sim-ple and
Accurate Dependency Parsing Using Bidi-rectional LSTM Feature
Representations. Transac-tions of the Association for Computational
Linguis-tics 4:313–327.
Jayant Krishnamurthy and Tom M. Mitchell. 2015.Learning a
Compositional Semantics for Freebasewith an Open Predicate
Vocabulary. Transactionsof the Association for Computational
Linguistics3:257–270.
Tom Kwiatkowksi, Luke Zettlemoyer, Sharon Gold-water, and Mark
Steedman. 2010. Inducing Prob-abilistic CCG Grammars from Logical
Form withHigher-Order Unification. In Proceedings of Em-pirical
Methods on Natural Language Processing.pages 1223–1233.
-
Tom Kwiatkowski, Eunsol Choi, Yoav Artzi, and LukeZettlemoyer.
2013. Scaling Semantic Parsers withOn-the-Fly Ontology Matching. In
Proceedings ofEmpirical Methods on Natural Language Process-ing.
pages 1545–1556.
Roger Levy and Galen Andrew. 2006. Tregex and tsur-geon: tools
for querying and manipulating tree datastructures. In Proceedings
of LREC. pages 2231–2234.
Mike Lewis and Mark Steedman. 2013. CombinedDistributional and
Logical Semantics. Transactionsof the Association for Computational
Linguistics1:179–192.
Percy Liang, Michael Jordan, and Dan Klein. 2011.Learning
Dependency-Based Compositional Seman-tics. In Proceedings of
Association for Computa-tional Linguistics. pages 590–599.
Fei Liu, Jeffrey Flanigan, Sam Thomson, NormanSadeh, and Noah A.
Smith. 2015. Toward Ab-stractive Summarization Using Semantic
Represen-tations. In Proceedings of North American Chap-ter of the
Association for Computational Linguistics.pages 1077–1086.
Mitchell P. Marcus, Mary Ann Marcinkiewicz, andBeatrice
Santorini. 1993. Building a large annotatedcorpus of English: The
Penn Treebank. Computa-tional linguistics 19(2):313–330.
Ryan McDonald, Joakim Nivre, Yvonne Quirmbach-Brundage, Yoav
Goldberg, Dipanjan Das, Kuz-man Ganchev, Keith Hall, Slav Petrov,
HaoZhang, Oscar Täckström, Claudia Bedini, NúriaBertomeu
Castelló, and Jungmee Lee. 2013. Univer-sal Dependency Annotation
for Multilingual Pars-ing. In Proceedings of Association for
Computa-tional Linguistics. pages 92–97.
Richard Montague. 1973. The Proper Treatment ofQuantification in
Ordinary English. In K.J.J. Hin-tikka, J.M.E. Moravcsik, and P.
Suppes, editors,Approaches to Natural Language, Springer
Nether-lands, volume 49 of Synthese Library, pages 221–242.
Shashi Narayan and Claire Gardent. 2014. HybridSimplification
using Deep Semantics and MachineTranslation. In Proceedings of
Association for Com-putational Linguistics. pages 435–445.
Rebecca Nesson and Stuart M. Shieber. 2006. SimplerTAG Semantics
Through Synchronization. In Pro-ceedings of the 11th Conference on
Formal Gram-mar. Center for the Study of Language and Informa-tion,
Malaga, Spain, pages 129–142.
Joakim Nivre, Marie-Catherine de Marneffe, Filip Gin-ter, Yoav
Goldberg, Jan Hajic, Christopher D. Man-ning, Ryan McDonald, Slav
Petrov, Sampo Pyysalo,Natalia Silveira, Reut Tsarfaty, and Daniel
Zeman.2016. Universal Dependencies v1: A MultilingualTreebank
Collection. In Proceedings of the Tenth In-ternational Conference
on Language Resources and
Evaluation. European Language Resources Associa-tion (ELRA),
Paris, France.
Joakim Nivre et al. 2016. Universal dependencies
1.3.LINDAT/CLARIN digital library at the Institute ofFormal and
Applied Linguistics, Charles Universityin Prague.
Panupong Pasupat and Percy Liang. 2015. Composi-tional Semantic
Parsing on Semi-Structured Tables.In Proceedings of Association for
ComputationalLinguistics. pages 1470–1480.
Ellie Pavlick, Johan Bos, Malvina Nissim, CharleyBeller,
Benjamin Van Durme, and Chris Callison-Burch. 2015. Adding
Semantics to Data-DrivenParaphrasing. In Proceedings of Association
forComputational Linguistics. pages 1512–1522.
Jeffrey Pennington, Richard Socher, and ChristopherManning.
2014. Glove: Global Vectors for WordRepresentation. In Proceedings
of the 2014 Con-ference on Empirical Methods in Natural
LanguageProcessing. Association for Computational Linguis-tics,
Doha, Qatar, pages 1532–1543.
Barbara Plank, Anders Søgaard, and Yoav Goldberg.2016.
Multilingual Part-of-Speech Tagging withBidirectional Long
Short-Term Memory Models andAuxiliary Loss. In Proceedings of the
54th AnnualMeeting of the Association for Computational
Lin-guistics. Berlin, Germany, pages 412–418.
Siva Reddy, Mirella Lapata, and Mark Steedman. 2014.Large-scale
Semantic Parsing without Question-Answer Pairs. Transactions of the
Association forComputational Linguistics 2:377–392.
Siva Reddy, Oscar Täckström, Michael Collins, TomKwiatkowski,
Dipanjan Das, Mark Steedman, andMirella Lapata. 2016. Transforming
DependencyStructures to Logical Forms for Semantic
Parsing.Transactions of the Association for
ComputationalLinguistics 4:127–140.
Tim Rocktäschel, Sameer Singh, and Sebastian Riedel.2015.
Injecting Logical Background Knowledgeinto Embeddings for Relation
Extraction. In Pro-ceedings of North American Chapter of the
Asso-ciation for Computational Linguistics. pages 1119–1129.
Sebastian Schuster and Christopher D. Manning. 2016.Enhanced
English Universal Dependencies: An Im-proved Representation for
Natural Language Under-standing Tasks. In Proceedings of the Tenth
Interna-tional Conference on Language Resources and Eval-uation.
European Language Resources Association(ELRA), Paris, France.
Kiril Simov and Petya Osenova. 2011. Towards Min-imal Recursion
Semantics over Bulgarian Depen-dency Parsing. In Proceedings of the
InternationalConference Recent Advances in Natural
LanguageProcessing 2011. RANLP 2011 Organising Commit-tee, Hissar,
Bulgaria, pages 471–478.
-
Kathrin Spreyer and Anette Frank. 2005. ProjectingRMRS from
TIGER Dependencies. In Proceedingsof the HPSG 2005 Conference. CSLI
Publications.
Mark Steedman. 2000. The Syntactic Process. TheMIT Press.
Yu Su, Huan Sun, Brian Sadler, Mudhakar Srivatsa,Izzeddin Gur,
Zenghui Yan, and Xifeng Yan. 2016.On Generating Characteristic-rich
Question Sets forQA Evaluation. In Proceedings of the 2016
Con-ference on Empirical Methods in Natural LanguageProcessing.
Austin, Texas, pages 562–572.
Ferhan Türe and Oliver Jojic. 2016. Simple and Ef-fective
Question Answering with Recurrent NeuralNetworks. CoRR
abs/1606.05029.
Lucy Vanderwende, Arul Menezes, and Chris Quirk.2015. An AMR
parser for English, French, German,Spanish and Japanese and a new
AMR-annotatedcorpus. In Proceedings of the 2015 Conference ofthe
North American Chapter of the Association forComputational
Linguistics: Demonstrations. Asso-ciation for Computational
Linguistics, Denver, Col-orado, pages 26–30.
Aaron Steven White, Drew Reisinger, Keisuke Sak-aguchi, Tim
Vieira, Sheng Zhang, Rachel Rudinger,Kyle Rawlins, and Benjamin Van
Durme. 2016. Uni-versal Decompositional Semantics on Universal
De-pendencies. In Proceedings of the 2016 Conferenceon Empirical
Methods in Natural Language Pro-cessing. Association for
Computational Linguistics,Austin, Texas, pages 1713–1723.
Yuk Wah Wong and Raymond Mooney. 2007. Learn-ing Synchronous
Grammars for Semantic Parsingwith Lambda Calculus. In Proceedings
of Associa-tion for Computational Linguistics. pages 960–967.
Kun Xu, Siva Reddy, Yansong Feng, Songfang Huang,and Dongyan
Zhao. 2016. Question Answering onFreebase via Relation Extraction
and Textual Evi-dence. In Proceedings of the Association for
Compu-tational Linguistics. Association for
ComputationalLinguistics, Berlin, Germany.
Xuchen Yao. 2015. Lean Question Answering overFreebase from
Scratch. In Proceedings of NorthAmerican Chapter of the Association
for Computa-tional Linguistics. pages 66–70.
Xuchen Yao and Benjamin Van Durme. 2014. Informa-tion Extraction
over Structured Data: Question An-swering with Freebase. In
Proceedings of Associa-tion for Computational Linguistics. pages
956–966.
Semih Yavuz, Izzeddin Gur, Yu Su, Mudhakar Srivatsa,and Xifeng
Yan. 2016. Improving Semantic Pars-ing via Answer Type Inference.
In Proceedings ofthe 2016 Conference on Empirical Methods in
Natu-ral Language Processing. Association for Computa-tional
Linguistics, Austin, Texas, pages 149–159.
Wen-tau Yih, Ming-Wei Chang, Xiaodong He, andJianfeng Gao. 2015.
Semantic Parsing via StagedQuery Graph Generation: Question
Answering withKnowledge Base. In Proceedings of Association
forComputational Linguistics. pages 1321–1331.
John M. Zelle and Raymond J. Mooney. 1996. Learn-ing to Parse
Database Queries Using Inductive LogicProgramming. In Proceedings
of Association for theAdvancement of Artificial Intelligence. pages
1050–1055.
Luke S. Zettlemoyer and Michael Collins. 2005. Learn-ing to Map
Sentences to Logical Form: StructuredClassification with
Probabilistic Categorial Gram-mars. In Proceedings of Uncertainty
in Artificial In-telligence. pages 658–666.
-
Universal Semantic Parsing: Supplementary Material
Siva Reddy† Oscar Täckström‡ Slav Petrov‡ Mark Steedman†
Mirella Lapata††ILCC, School of Informatics, University of
Edinburgh
‡ Google, [email protected], {oscart, slav}@google.com,
{steedman, mlap}@inf.ed.ac.uk
Abstract
This supplementary material to UniversalSemantic Parsing,
provides and outline ofhow quantification can be incorporated inthe
UDEPLAMBDA framework.
1 Universal Quantification
Consider the sentence Everybody wants to buy ahouse,1 whose
dependency tree in the UniversalDependencies (UD) formalism is
shown in Fig-ure 1(a). This sentence has two possible
readings:either (1) every person wants to buy a differenthouse; or
(2) every person wants to buy the samehouse. The two
interpretations correspond to thefollowing logical forms:(1)
∀x.person(xa)→
[∃zyw.wants(ze)∧ arg1(ze,xa)∧buy(ye)∧xcomp(ze,ye)∧house(wa)∧
arg1(ze,xa)∧ arg2(ze,wa)] ;
(2) ∃w.house(wa)∧ (∀x.person(xa)→[∃zy.wants(ze)∧
arg1(ze,xa)∧buy(ye)∧xcomp(ze,ye)∧
arg1(ze,xa)∧ arg2(ze,wa)]) .
In (1), the existential variable w is in the scope ofthe
universal variable x (i.e. the house is dependenton the person).
This reading is commonly referredto as the surface reading.
Conversely, in (2) theuniversal variable x is in the scope of the
existentialvariable w (i.e. the house is independent of theperson).
This reading is also called inverse reading.Our goal is to obtain
the surface reading logicalform in (1) with UDEPLAMBDA. We do not
aim toobtain the inverse reading, although this is possiblewith the
use of Skolemization (Steedman, 2012).
In UDEPLAMBDA, lambda expressions forwords, phrases and
sentences are all of theform λx. . . .. But from (1), it is clear
that we needto express variables bound by quantifiers, e.g.
∀x,while still providing access to x for composition.This demands a
change in the type system since the
1Example borrowed from Schuster and Manning (2016).
Everybody wants to buy a house
nsubj
xcomp dobj
detmark
root
(a) Original dependency tree.
Everybody wants to buy a house
nsubj
xcomp dobj
detmark
root
Ω Ωbind nsub
j
(b) Enhanced dependency tree.
Everybody wants to buy a house
nsubj:univ
xcomp dobj
detmark
root
Ω Ωbind nsub
j
(c) Enhanced dependency tree with universal quantification.
Figure 1: The dependency tree for Everybodywants to buy a house
and its enhanced variants.
same variable cannot be lambda bound and quanti-fier bound—that
is we cannot have formulas of theform λx . . .∀x . . .. In this
material, we first derivethe logical form for the example sentence
usingthe type system from our main paper (Section 1.1)and show that
it fails to handle universal quantifi-cation. We then modify the
type system slightlyto allow derivation of the desired surface
readinglogical form (Section 1.2). This modified type sys-tem is a
strict generalization of the original typesystem.2 Fancellu et al.
(2017) present an elaboratediscussion on the modified type system,
and how itcan handle negation scope and its interaction
withuniversal quantifiers.
2Note that this treatment has yet to be added to
ourimplementation, which can be found at
https://github.com/sivareddyg/udeplambda.
mailto:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]://github.com/sivareddyg/udeplambdahttps://github.com/sivareddyg/udeplambda
-
1.1 With Original Type System
We will first attempt to derive the logical form in (1)using the
default type system of UDEPLAMBDA.Figure 1(b) shows the enhanced
dependency treefor the sentence, where BIND has been introducedto
connect the implied nsubj of buy (BIND is ex-plained in the main
paper in Section 3.2). Thes-expression corresponding to the
enhanced tree is:(nsubj (xcomp wants (mark
(nsubj (dobj buy (det house a)) Ω) to))(BIND everybody Ω)) .
With the following substitution entries,wants, buy ∈
EVENT;everybody, house ∈ ENTITY;a, to ∈ FUNCTIONAL;Ω =
λx.EQ(x,ω);nsubj= λ f gx.∃y. f (x)∧g(y)∧ arg1(xe,ya);dobj= λ f
gx.∃y. f (x)∧g(y)∧ arg2(xe,ya);xcomp= λ f gx.∃y. f
(x)∧g(y)∧xcomp(xe,ya);mark ∈ HEAD;BIND ∈ MERGE,
the lambda expression after composition becomes:λz.
∃xywv.wants(ze)∧ everybody(xa)∧ arg1(ze,xa)∧
EQ(x,ω)∧buy(ye)∧xcomp(ze,ye)∧ arg1(ye,va)∧ EQ(v,ω)∧
arg1(xe,ya)∧house(wa)∧ arg2(ye,wa) .
This expression encodes the fact that x and v arein unification,
and can thus be further simplified to:(3) λz.∃xyw.wants(ze)∧
everybody(xa)∧ arg1(ze,xa)
∧ buy(ye)∧xcomp(ze,ye)∧ arg1(ye,xa)∧ arg1(xe,ya)∧house(wa)∧
arg2(ye,wa) .
However, the logical form (3) differs from thedesired form (1).
As noted above, UDEPLAMBDAwith its default type, where each
s-expression musthave the type η = Ind×Event→ Bool, cannothandle
quantifier scoping.
1.2 With Higher-order Type System
Following Champollion (2010), we make a slightmodification to
the type system. Instead of usingexpressions of the form λx. . . .
for words, we useeither λ f .∃x. . . . or λ f .∀x. . . ., where f
has type η.As argued by Champollion, this higher-order formmakes
quantification and negation handling soundand simpler in
Neo-Davidsonian event semantics.Following this change, we assign
the followinglambda expressions to the words in our
examplesentence:everybody = λ f .∀x.person(x)→ f (x) ;wants = λ f
.∃x.wants(xe)∧ f (x) ;to = λ f .TRUE ;buy = λ f .∃x.buy(xe)∧ f (x)
;a = λ f .TRUE ;house = λ f .∃x.house(xa)∧ f (x) ;Ω = λ f . f (ω)
.
Here everybody is assigned universal quantifiersemantics. Since
the UD representation does notdistinguish quantifiers, we need to
rely on a small(language-specific) lexicon to identify these.
Toencode quantification scope, we enhance the la-bel nsubj to
nsubj:univ, which indicates thatthe subject argument of wants
contains a universalquantifier, as shown in Figure 1(c).
This change of semantic type for words and s-expressions forces
us to also modify the seman-tic type of dependency labels, in order
to obeythe single-type constraint of DEPLAMBDA (Reddyet al., 2016).
Thus, dependency labels will nowtake the form λPQ f . . . ., where
P is the parent ex-pression, Q is the child expression, and the
returnexpression is of the form λ f . . . .. Following thischange,
we assign the following lambda expres-sions to dependency
labels:nsubj:univ= λPQ f .Q(λy.P(λx. f (x)∧ arg1(xe,ya))) ;nsubj=
λPQ f .P(λx. f (x)∧Q(λy.arg1(xe,ya))) ;dobj= λPQ f .P(λx. f
(x)∧Q(λy.arg2(xe,ya))) ;xcomp= λPQ f .P(λx. f
(x)∧Q(λy.xcomp(xe,ya))) ;det, mark= λPQ f .P( f ) ;BIND = λPQ f
.P(λx. f (x)∧Q(λy.EQ(y,x))) .
Notice that the lambda expression ofnsubj:univ differs from
nsubj. In the for-mer, the lambda variables inside Q have
widerscope over the variables in P (i.e. the universalquantifier
variable of everybody has scope over theevent variable of wants)
contrary to the latter.
The new s-expression for Figure 1(c) is(nsubj:univ (xcomp wants
(mark
(nsubj (dobj buy (det house a)) Ω) to))(BIND everybody Ω)) .
Substituting with the modified expressions, andperforming
composition and simplification leads tothe expression:(6) λ f .∀x
.person(xa)→
[∃zyw. f (z)∧wants(ze)∧ arg1(ze,xa)∧buy(ye)∧ xcomp(ze,ye)∧
house(wa)∧ arg1(ze,xa)∧ arg2(ze,wa)] .
This expression is identical to (1) except for theoutermost term
λ f . By applying (6) to λx.TRUE,we obtain (1), which completes the
treatment ofuniversal quantification in UDEPLAMBDA.
ReferencesLucas Champollion. 2010. Quantification and
negation
in event semantics. Baltic International Yearbook ofCognition,
Logic and Communication 6(1):3.
Federico Fancellu, Siva Reddy, Adam Lopez, and Bon-nie Webber.
2017. Universal Dependencies to Logi-cal Forms with Negation Scope.
arXiv Preprint .
-
Siva Reddy, Oscar Täckström, Michael Collins, TomKwiatkowski,
Dipanjan Das, Mark Steedman, andMirella Lapata. 2016. Transforming
DependencyStructures to Logical Forms for Semantic
Parsing.Transactions of the Association for
ComputationalLinguistics 4:127–140.
Sebastian Schuster and Christopher D. Manning. 2016.Enhanced
English Universal Dependencies: An Im-
proved Representation for Natural Language Under-standing Tasks.
In Proceedings of the Tenth Interna-tional Conference on Language
Resources and Eval-uation. European Language Resources
Association(ELRA), Paris, France.
Mark Steedman. 2012. Taking Scope - The Natural Se-mantics of
Quantifiers. MIT Press.