Rewrite and Understand: a transformational approach to natural language understanding in a dialogue system

Rewrite and Understand:a transformational approach to natural languageunderstanding in a dialogue systemRieks op den Akker, Danny H. Lie and Joris Hulstijn �AbstractNatural language understanding in dialogue systems is taken toconsist of two subsequent processes: rewrite and understand. Therewriting proces is a transformational proces in which natural lan-guage utterances are mapped via a sequence of context-sensitive stringto string transformations onto some semantic normal form. In the un-derstand proces an interpretation of the semantic form is made, withrespect to a particular dialogue state. The rewrite proces is com-pletely independent of the state of the dialogue. The transformationsare speci�ed by a rewriting system. The paper discusses the linguisticand semantic principles of such a rewriting system and presents theway they can be speci�ed in a comprehensive and modular way. Fi-nally it compares the transformational approach with more traditionalgrammatical and parsing approaches. The approach is used in a �rstprototype of a natural language interface for a theatre information andbooking system.1 IntroductionA natural language dialogue system is a system that o�ers its users the pos-sibility to engage in a dialogue with the system using their own (natural)language. By means of such a dialogue the user can ask for informationin a particular domain or specify actions to be performed by the system.The quality of such a system strongly depends (among other things) onthe naturalness of the dialogue. If these systems are built to replace hu-man information providers, users have a high expectation about its natural�Computer Science, University of Twente, PO BOX 217, 7500 AE Enschede, TheNetherlands, (infrieks j lie j joris)@cs.utwente.nl1

language understanding ability. It is hard to meet user expectations. Theproblem is to make the machine behave as if it understands the expressionspeople use in dialogue. It is well-known that in human-computer dialogues,people use di�erent expresssions than in human-human dialogues. However,the type of expressions used, is still very irregular.In some cases the di�culty is that complex linguistic constructions areused, as in the polite request (1). But sometimes the di�culty is ratherthat users simplify their utterances in a way that is not easy for a parser tounderstand. (example (2)) Determiners or verbs are omitted and anaphoricexpressions and ellipsis are used a lot. Background knowledge and expecta-tions about the system play an important role here.(1) Kunt u me misschien ook vertellen of Karin Bloemen ook optreedt?Could you please tell me whether Karin Bloemen is performing at all?(2) Drie alstublieft, voor morgenavond.Three please, for tomorrow night.The challenge is to build a robust system, that on the one hand can deal withcomplex polite expressions like (1), but on the other hand is able to parsealmost any string containing meaningful information for the task domain.The dialogue state can be used to �ll in the missing parameters. By the stateof the dialogue we mean everything that in uences the systems reaction ona particular utterance. It subsumes the contents of the database as well asinformation about the dialogue sofar.We model the proces of `understanding' a natural language expressionas two subsequent processes. The �rst part transforms the expression intosome intermediate semantic form, the second part is an interpretation ofthis form given the state of the dialogue. So, the systems reaction on a userutterance in principle depends on the state of the dialogue, but the way inwhich the intermediate semantic form is constructed does not depend onthis state.Within this model of language understanding a natural language expres-sion is seen as a manner (be it speech or typed or written) of wrapping aformal semantic form. For instance, the expression \please show me thedance performances of next week" is a natural way of uttering a seman-tic form like [ACT: show ; ARGUMENT : performances ; GENRE: dance; DATE : next week]. Another way of uttering the same semantic form is:\show dance performances of next week".If two expressions are transformed into one and the same semantic formwe call these expression semantically equivalent. The pragmatic meaning2

(that is the interpretation) of an expression depends on the context of thedialogue. One and the same expression can have di�erent pragmatic mean-ings. Thus, a semantic form can have di�erent interpretations, dependenton the state of the dialogue. Semantic forms are called pragmatically equiv-alent if in all dialogue contexts they have the same interpretation, i.e. thesystem cannot distinguish them by acting di�erently. Since it is of no useto distinguish pragmatically equivalent semantic forms they are consideredthe same. Which semantic forms we distinguish depends ultimately on thefunctionality of the dialogue system. If an utterance contains pragmaticallyirrelevant information this information can be �ltered out by the �rst proces.On the other hand all pragmatically relevant information should somehowbe re ected in the semantic form that is the result of the �rst proces. Thequestions what kind of expressions users will use in a dialogue with the sys-tem and which parts of these expressions are pragmatically irrelevant canonly be answered by experiments with previous versions of the system beingdeveloped.In this paper we describe the dialogue state independent part of thenatural language understanding proces as a transformation proces in whichthe user utterance is transformed via a sequence of context-sensitive stringto string transformations into a semantic form.In section 2 we discuss the principles that underly the di�erent kinds ofrewriting rules. In section 3 we describe how the rewriting system is spec-i�ed. In section 4 we an example is given of the rewriting process and insection 5 it is shown how it is embedded in a dialogue system. Finally insection 6 we compare this approach with more traditional grammatical ap-proaches to natural language understanding, we discuss some shortcomingsof the current system and ways to handle them.Examples are in Dutch; they are client sentences and rewriting rulesthat are used in the theatre information and booking system SCHISMA(Hoeven et al. 1995; Hulstijn et al. 1996). The rewriting system for thelanguage processing part of this information system is based on an analysisof a corpus of dialogues obtained by Wizard of Oz experiments with anearlier semi-automatic version of the system.The complete system for the �rst state independent proces consists ofthe following parts (�gure 1). In this paper we mainly concentrate on theRewriter and the Parser. The word parser will also be used for the wholeproces of rewriting and parsing.Already Chomsky's well-known \Standard Theory" has a transforma-tional component (see Chomsky 1965). Within this theory however, trans-formations are applications of tree to tree rewriting rules; they relate deep3

semantic form

raw input sentence

parser

rewriter

number tagger

spelling corrector

time & date tagger

name tagger

annotated string

Figure 1: transformation stepsstructures { sometimes seen as a semantic structure { to their surface struc-ture. Here we consider context-sensitive string-to-string rewriting as intro-duced by Thue in the beginning of this century (see Salomaa 1973).The semantic forms output by the parser are feature-value structures,in which features represent particular types of semantic or pragmatic infor-mation and values can be either feature-value structures or atomic values.In the current system semantic forms are at lists of feature value pairs:the �rst element of the list is interpreted as a dialogue function; the otherelements are interpreted as argument values of this function. When a valueof an obligatory arguments is not known the dialogue system asks the userfor the required information. Semantic forms may contain values that are(typed) indices referring to topics of the dialogue. The reference problemis solved by the dialogue manager by searching for appropriate elementsin a context stack. We concentrate here on the rewriting system, which isindependent of the type of semantic forms chosen.2 Principles of rewritingThere are four principles that underly the rewriting system.4

1. Deletion of semantically irrelevant words2. Substitution of words by standard synonymous words or ex-pressions3. Transformation of complex (like polite) forms into simple (im-perative) normal forms4. Changing the order of phrases into a prede�ned normal order1Each of these principles motivate the introduction of a set of transforma-tion rules. Each application of a rewriting rule (a parse step) results ina semantically equivalent sequence. An input sentence is submitted to atransformation system that applies the rules in the order in which they arespeci�ed. A sequence of words S0 is transformed into S1 using rule r1; thenS1 is transformed into S2 using rule r2 and so on until no rule can be appliedany more, �nally resulting in a sequence Sn, which is the semantic form ofS0. S0 r1�! S1 r2�! S2 ! ::: ! Sn�1 rn�! SnGiven a particular order of the rules, S0 will always result in the samesemantic form Sn. Since the rules are used in the order in which theyare speci�ed and at most once in the transformation proces, there is nopossibility that the proces becomes cyclic.In some applications (for some particular domains) two words or con-structs may be considered as synonym, where they should be consideredsemantically di�erent in other domains. But there are words, phrases or pat-terns that are semantically equivalent no matter what the particular domainis. Therefore rules are categorized into a set of global rules which are domainindependent and sets of local rules which are domain dependent and whichare associated with particular semantic forms and pragmatic functions.A speci�cation for the transformational system consists of the followingparts.GlobalStrip in which it is speci�ed how the input sentence is prepared forfurther processing. It contains a list of tokens that should be removed,like punctuation marks and special tags.GlobalRemove in which it is specifed which words should be deleted fromthe sentence.1Notice that in Dutch there is for instance no \place before time"constraint for adver-bial phrases. 5

GlobalAliases speci�es synonyms of wordsGlobalRules this parts contains the global rewrite rulesGlobalMacros in which global macros are de�nedLocalGrammar in which local grammars are speci�ed for particular lan-guage constructs.3 How rules are speci�edGlobal rules obey the following syntax-rules:GlobalRule ::= Pattern -> ResultPattern ::= { Expression }Expression ::= Imp | Opt | Choice | Var | Word | Joker | WildcardImp ::= ( Expression )Opt ::= [ Expression ]Choice ::= { Expression | }Var ::= Varname = ( Exp )Varname ::= "<" Word ">"Joker ::= "*"Wildcard ::= "**"Result ::= { ResultExp }ResultExp ::= Word | VarnameNotation: f Expression g means a sequence of one or more Expressions.[ Expression ] means that the Expression is optional: it matches zeroor one occurrence of the pattern Expression. f Exp1 | Exp 2 g matcheswith patterns that match either with Exp1 or with Exp2. Round bracketsare used to indicate units in compound patterns: For example the pat-tern: date f ( en date ) | ( of date ) g matches \3 maart en 4maart"as well as \3 maart of 4 maart". The joker * matches any wordwhere the wildcard ** matches any sequence of words.A rewrite rule pattern -> result is applicable to any sequence ofcharacters having a subsequence that matches the pattern in the lefthandside of the rule. The result of applying the rule is the original sequence inwhich the string matching the pattern has been replaced by the sequenceon the righthand side of the rule.6

MacrosMacros can be used in patterns in any rewrite rule. They allow for a morecompact speci�cation of the grammar rules. A macro de�nition obeys thefollowing syntax.Macro ::= Macroname "=" MacrovalueMacronames are words enclosed by underscores: like date A Macrovalueis a regular expression that may contain other macronames. However, re-cursive macrode�nitions are not allowed.Example: The following macrode�nitions de�ne patterns for (de voorstelling<voorst> van <naam>)._det_ = (de | het | een )_pvnw_ = (ik | we | ze )_voorst_ = _det_ voorstelling %play <voorst> play%[van %name <naam> name%]In the pattern: %play <voorst> play% the tags %play and play% are usedto indicate that the pattern in between (a variable name) matches a specialword or phrase. In this case a phrase indicating the name or title of someperformance. These tags are introduced by a tagger proces, that is runbefore the transformation proces.Special featuresTo indicate that a word or sequence of words may only be replaced byanother word or sequence of words when it occurs at the beginning (or end)of sentence, a rule contains the | symbol (bar). For instance the followingrule allows the �nal transformation of : \doe me 4 kaartjes" into \geef vierkaartjes".| doe me --> geefLocal GrammarsLocal Grammars contain rewriting rules and (locally valid) macro de�ni-tions just like the Global Grammar. The Local rules of a Local Grammarare used to �nally transform the result of earlier rewriting steps (in whichonly Global rules were applied) into a �nal semantic form. Local grammarsare associated with particular categories of semantic forms. These categories7

of semantic forms are associated with particular interpretations (pragmaticfunctions or communicative acts). They are domain and application de-pendent. Together with the Global rules they form the complete rewritinggrammar for the transformation system.Each Local Grammar has a permission set of key-words or key-phrases.The rules of a Local Grammar may only be applied to rewrite a sequenceif the sequence contains one or more keywords in its permission set. Alsokey-words or phrases in permission sets can be marked by a | (bar) toindicate that the word should occur at the beginning or end of the sequence.Permission sets of di�erent Local Grammars are not necessarily disjoint. Theparser chooses a permitted Local Grammar and tries to construct a semanticform by applying rules from this grammar. If it doesn't succeed it will tryto apply another permitted Local Grammar Each Local Grammar has a setof forbidden words. If one of these words occurs in a sequences it is notallowed to apply rules of the Local Grammar. This feature speeds up theparser in �nding permitted Local Grammars.Scoring the resulting semantic formThe parser for the Local grammars gives a score to the output semanticform it has constructed. Therefore a penalty is accociated with each LocalRule; it is a number indicating how good the input �ts the result. This�nal parser is robust: it may skip words in its input in order to matchit with the pattern in the lefthand side of the rule it tries to apply. Themore words are deleted for this purpose the higher the penalty given to theresulting semantic form. If the parser has found a semantic form with a zeropenalty it will stop searching for other applicable rules or Local Grammars.Otherwise it will return the result with the best score. To indicate that theparser in applying a �nal rewrite rule may not disgard a word or words inbetween two words the underscore is used in patterns. For instance, the rulevoor datum -> ... is only applicable if the word `voor' is followed bythe word `datum' in the sequence being rewritten.Developing the grammarAn initial set of rewriting rules was based on the analysis of client utterancesin a corpus of dialogues. These dialogues were obtained by logging Wizardof Oz experiments with an earlier version of the theatre information system.The grammars are further tuned and developed on the basis of experimentswith the current completely automized system. In the theatre information8

system there are for example sentences that ask for the date of some perfor-mance, while another category of sentences ask for information about thecontents of some performance, and another one for asking whether there arestill seats free for a particular performance. For this particular informationsystem there are 20 Local Grammars; one for each of the 20 categories ofsemantic forms. These categories are the result of analysing a corpus ofdialogues. The Global Grammar consists of about 200 rules; the 20 LocalGrammars together contain 400 rules. A speci�cation of a rewriting systemconsists of several �les. Parts of speci�cations can be taken together bymeans of include directives.4 Example of the transformation procesWe illustrate the whole transformation proces by means of the followingexample sentence:input:Kunt u mij alstublieft vertellen wanneer er volgende week eenvoorstelling van Karin Bloemen is?(could you please tell me when there is a performance of Karin Bloemennext week?)1. remove endmarks, capitals:"?" -> .result: kunt u mij alstublieft vertellen wanneer er volgende weekeen voorstelling van karin bloemen is2. tag names, dates and performance names:<date>= date macro -> %date <date> date%.result: kunt u mij alstublieft vertellen wanneer er %date volgendeweek date% %play een voorstelling play% van %name Karin Bloemenname% is3. remove irrelevant words:alstublieft -> .result: kunt u mij vertellen wanneer er %date volgende week date%%play een voorstelling play% van %name Karin Bloemen name% is4. rewrite aliases:vertellen -> zeggen.result: kunt u mij zeggen wanneer er %date volgende week date%%play een voorstelling play% van %name KarinBloemen name% is v 9

5. rewrite:mij -> me.je me -> je.kunt u -> kan je.kan je zeggen -> zeg.zeg wanneer -> wanneer.wanneer ** is -> wanneer is **.result: wanneer is er %date volgende week date% %play een voorstellingplay% van %name Karin Bloemen name%6. shu�e sentence parts:<play>=(%play ** play%) ** -> ** <play>. (moves <play> to end)<name>=([van] %name ** name%) ** -> ** <name>.<date>=(%date ** date%) ** -> ** <date>.result: wanneer is er %play een voorstelling play% van %name KarinBloemen name% %date volgende week%7. apply local grammar for sentences that ask for dates:er -> .wanneer is -> wanneer speelt.result: wanneer speelt %play een voorstelling play% van %name KarinBloemen name% %date volgende week %date8. �nal rewrite:wanneer speelt <play>= play [[van] <name>= name [<date>= date ]]-> (PROC: datum, PLAY: <play>, NAME: <name>, DATE: <date>)�nal resulting semantic form:[PROC: datum, PLAY: een voorstelling, NAME: Karin Bloemen,DATE: volgende week]Please note the following: (1) In Dutch \kan je zeggen" and \kan je mezeggen" are more polite forms for asking information than \zeg me". (2)\Zijn er volgende week voorstellingen?" (\Are there any performances nextweek?") en \zijn er voorstellingen volgende week" are equally good Dutchwith no distinction in meaning. (3) In step 7 above the local grammar fordate questions is applied because of the occurrence of the words `wanneer'(`when') and `is' in its permission set of key-words.5 SchismaIn the SCHISMA project the objective is to develop a natural languagedialogue sustem for theatre information and reservations. Users may inquire10

about performances or have tickets reserved. The main problems studied inthis project are:� How to model dialogues?� How to model and proces the natural language input of users?� How to model the context of the dialogue?Starting with a global functional speci�cation of the system we �rst devel-oped a Wizard of Oz environment allowing people to engage in a dialoguewith a database through a human Wizard. The Wizard responded by meansof a restricted set of canned sentences in which slots for database informa-tion could be �lled. These �rst experiments resulted in a corpus of about80 dialogues containing about thousand client sentences. This corpus wasanalysed with respect to the kinds of requests users have and the way theyexpress them in language.Next we started to develop a context-free uni�cation grammar thatshould cover most of the client utterences. This showed to be a very di�culttask. When the grammar had a few hundred of rules it became unmanagable.Moreover the price for keeping the grammar and head-corner chart parserrobust was that it became too ambiguous.Therefore we decided to follow the rewrite approach. It turned out to bemuch easier to make the step from the corpus of client sentences to a string-to-string rewriting system than to a full-blown grammar. The �rst rewritingsystem for the SCHISMA application was developed by adding about 20Local Grammars to an original Global Grammar developed by Lie. It canhandle 80 percent of the client utterances. Most of the utterances that couldnot be processed have functions that do not �t the task domain of the system(for instance questions about public transport to the theatre). They do notcontain keywords that trigger the parser to apply one of the distinguishedLocal Grammars. Some di�culties arose with constructs involving denials,compounds, or ellipsis. These can sometimes be handled by adding new rulesto the rewriting system only, but sometimes they also require extension ofthe understanding back-end part of the system. For example, coordinatedsentence forms like \Ik wil ** of **" (Eng. I want either ** or **) cantransformed into \Ik wil ** or ik wil **" (Eng. I want ** or I want ** ).The ellipsis in \Ik wil 3 kaartjes voor donderdag en twee voor maandag"(Eng. I want three tickets for thursday and two for monday) is not so easyto resolve by the rewriting system. It can only be solved by adding a special11

�nal rewriting rule for the back-end parser: reserveer cards date encards date .A problem is how to handle unknown words. When a proper nameoccurs in the input that is not in the name list (author or city names forinstance) the system skips this information at an early stage before rewritingand so the dialogue manager cannot handle this problem adequately. Thesequence is simply not recognized as probably being a name. Therefore amore structural analysis seems to be necessary. The problem of unknownwords is not typical however for the transformational approach we discussedin this paper.6 Comparison with traditional approachesThe transformational approach presented in this paper is an unorthodoxsoftware engineering approach. It seems to have some advantages if wecompare it with those approaches in which a context-free based uni�cationgrammar is developed. Given a corpus of sentences it is much easier to setup a rewriting system than to develop a context-free uni�cation grammar forthe same language domain. Large context-free uni�cation grammars thatare robust enough tend to be very ambiguous and parsing becomes verytime consuming.Replacement of words or phrases by semantically equivalent ones, re-ordering of phrases, and deletion can all be speci�ed in the same style.The rewriting system is rather easy to maintain and can easily be adoptedby adding or changing rules without the system becoming unpredictable.Moreover it is fast.The large variety in ways people express their questions and answerscauses a serious problem for practical natural language understanding sys-tems. It amounts to the problem of �nding \the appropriate level of granu-larity of language processing for the task at hand" (see Martin and Kehler1994). A simple keyword analysis has shown to be too coarse-grained formost applications (Sijtsma and Zweekhorst 1993), whereas full syntacticanalysis is far too complex and most likely impossible. In SWIFTUS (Mar-tin and Kehler 1994), the `appropriate level of granularity' is obtained by amulti-level �nite state automaton. If an input sequence is not accepted byan automaton it is submitted to a higher level automaton for a more coarsegrained analysis.We tried to solve this problem using a language engineering approach inwhich natural language expressions are transformed using context sensitive12

rewriting rules into a natural standard form that is �nally mapped ontoa semantic form. The application of this approach in a natural languagedialogue system shows the following advantages of this approach when com-pared to more traditional approaches in which a context-free base grammaris developed.1. A grammar for a simpli�ed standard language is much easier to developand more e�cient to implement.2. The transformation from standard natural expressions onto a semanticform is far more easier to specify.3. Some syntactic constructs occurring in natural languages are easier tohandle with a context-sensitive rewriting system than with a context-freeparser.4. It is easier to add rules for treating `ungrammatical sentences' or idiomaticphrases.5. The context-sensitive rewriting system is easier to adapt for a specialdomain.ReferencesChomsky, N.: 1965, Aspects of the theory of syntax, MIT Press, Cambridge,Mass.Hoeven, G. F. v. d. et al.: 1995, Schisma a natural language accessibletheatre information and booking system, in First International Workshopon Applications of Natural Language to Data Bases, Versailles, FranceHulstijn, J., Steetskamp, R., ter Doest, H., van de Burgt, S., and Nijholt, A.:1996, Topics in schisma dialogues, in Luperfoy, Nijholt, and Veldhuijzenvan Zanten (eds.), Dialogue Management in Natural Language Systems,TWLT11, University of TwenteMartin, P. and Kehler, A.: 1994, Speechacts: a testbed for continuous speechapplications, in Proceedings of the AAAI-94 workshop on the integrationof natural language and speech-processing, Seattle, WashingtonSalomaa, A.: 1973, Formal languages, Academic PressSijtsma, W. and Zweekhorst, O.: 1993, Comparison and review of commer-cial natural language interfaces, in F. Jong and A. Nijholt (eds.), Naturallanguage interfaces, TWLT5, University of Twente13