LING/C SC/PYSC 438/538 Computational Linguistics Sandiway Fong Lecture 2: 8/24
Dec 20, 2015
2
Administrivia
• Textbook – Speech and Language Processing by Jurafsky &
Martin. Prentice-Hall 2000 – (2nd edition to come summer 2007)
– One copy on reserve in the library
– Homepage• errata, updated chapters from the forthcoming 2nd
edition etc.• http://www.cs.colorado.edu/~martin/slp.html
– Background Reading Assignment• Chapter 1: Introduction
– background, history
• available on-line• http://www.cs.colorado.edu/%~martin/slp-ch1.pdf
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
3
Today’s Topic
• Gentle introduction to Prolog – (not used in SLP textbook)
– but you’ll need it for the homeworks
– we’ll be using it to encode finite state automata (FSA), grammars, inference rules
Assignment– install SWI-Prolog on your
computer (free)– current version: 5.6.17– www.swi-prolog.org– manual is downloadable from
the website– run today’s exercises
Something to consider– Computer Lab Class today for
LING 388: a hands-on first look at Prolog
– Time: 3:30pm–4:45pm– Place: Social Sciences 224
4
Some Background
• Prolog = Programming in Logic• based on Horn clause logic
– subset of first-order predicate calculus
– meaning: • can’t do everything 1st order
logic can• however, this does not limit
Prolog
• Roots in Mechanical Theorem Proving
– people investigating automated proofs methods for mathematics
– Resolution Rule (Robinson, 65)
• Language invented by Colmerauer (implementor) and Kowalski (theoretician, textbook: Logic for Problem Solving) in the early 70s
– designed to support natural language processing
– … has grammar rules
• There was one experiment to teach Prolog to schoolkids
– 13 year olds in London (early 1980s?)
– yes, you can learn it too !
5
Interesting History
• Prolog was adopted for Japan’s Fifth Generation Computer Project (80s)– 54.2 billion yen project
• ≈ 500 million dollars
– Was supposed to leapfrog the rest of the world
• well, we know what (didn’t) happen
[The term "fifth generation" was intended to convey the system as being a leap beyond existing machines.
Computers using vacuum tubes were called the first generation, transistors and diodes the second, ICs the third, and those using microprocessors the fourth.]
are we really still stuck here?
6
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Interesting History
http://www.icot.or.jp/MUSEUM
see also:http://en.wikipedia.org/wiki/Fifth_generation_computer
The project imagined a parallel processing computer running on top of massive databases (as opposed to a traditional filesystem) using a logic programming language to access the data. They envisioned building a prototype machine with performance between 100M and 1G LIPS,
LIPS = Logical Inference Per Second.
Echoes of this in the morerecent Human Genome Project
W/MIT: LabBase“Queries (and updates) are posed in a non-recursive logic programming language: the syntax and semantics are essentially those of a subset of Prolog”
7
Prolog by Example• Name things (symbols)• State facts (things that are true)• State rules (relations between things
that are true)• Key Concepts
– Facts and rules are stored in Prolog’s database
– Prolog has the ability to do logical inference over its database
– We communicate with this database by posing logic queries
• Closed World Assumption:– things are true only when given positive
evidence– Prolog never says “I don’t know”– Prolog only knows what it can infer from
the database– if it cannot prove a query, it says the
answer is “No”
Example• Medals: gold, silver, bronze• Facts:
medal(gold).
medal(silver).
medal(bronze).
• Database query:?- medal(silver). Yes
?- medal(aluminum). No
8
Prolog by Example
• Database query with variables?- medal(X). X
= gold ; X = silver ; X = bronze ;No
Notation• There are no variable declarations in
Prolog
• How do we know a variable from a symbol? By convention:
– Variable names begin with an uppercase letter, e.g. X
– Ordinary symbols begin with a lowercase letter, e.g. gold
Other things you need to know:• ; (disjunction) here it means “next
answer please”• ?- is the Prolog interpreter’s “prompt”
meaning it’s ready to receive a query• . the period terminates a query or
fact
9
Prolog by Example
• Facts can be relations:better(gold,silver).better(silver,bronze).better(bronze,nothing).
• Encoding:“gold is better than silver” ... etc.
• Database query:?- better(silver,bronze).
Yes?- better(bronze,X).
X = nothing
• Another query:?- better(gold,nothing).
No (WHAT!!)– Prolog can’t infer this from mere facts– We have to explicitly state the rule
involved in making this natural deduction
• Rule: (transitivity)– X is better than Y if X is better
than (some) Z, and Z is better than Y
better(X,Y) :- better(X,Z), better(Z,Y).
• Notation: – :- means “if” and – , means “and”
10
Prolog by Example
• Retry query:?- better(gold,nothing).
Yes (OK!!)
• How did Prolog prove it via database matching??- better(gold,nothing).
– Tries to match query to the facts (fails)
– Tries to match query to the rule (succeeds)
– better(X,Y) :- better(X,Z), better(Z,Y).
– when X = gold and Y = nothing
• Original query reduces to two subproblems or subqueries?- better(gold,Z).
?- better(Z,nothing).
• Both subqueries must succeed for original query to succeed
• Notes:– Prolog takes one query at a
time (serial not parallel)– in chronological order (of
definition)• behavior: depth-first search
11
Prolog by Example
• Computation tree
better(gold,silver).better(silver,bronze).better(bronze,nothing).better(X,Y) :- better(X,Z), better(Z,Y).
• Knowledge Base
better(gold,nothing)
better(gold,Z) better(Z,nothing)
better(gold,silver) better(silver,nothing)
better(silver,Z’) better(Z’,nothing)
better(silver,bronze) better(bronze,nothing)
12
Prolog by Example• Let’s trace the sequences of database queries for
?- better(gold,nothing).?- better(gold,Z).
Z = silver (by database fact)?- better(Z,nothing).?- better(silver,nothing).
?- better(silver,Z’).Z’ = bronze (by database fact)
?- better(Z’,nothing).?- better(bronze,nothing). (database fact)
Yes
• Notation: Z’used to distinguish this instance Z in this subquery from the main Z
• Computation tree can be explored mechanically in this fashion– Simple strategy lies at the heart of Prolog– Each inference step takes microseconds or less– Computers can make millions of inferences a second– Exploring long chains of inference is not a problem
13
Prolog by Example
• Prolog’s simple mechanical exploration strategy has its limitations• Query:
– ?- better(silver,gold). No (EXPECTED ANSWER)
• What actually happens with Prolog?– Doesn’t work!!!
14
Prolog by Example• Computation tree:
– ?- better(silver,gold).• ?- better(silver,Z).• ?- better(Z,gold).
– ?- better(silver,Z). (FIRST SUBQUERY FROM ABOVE)• Z = bronze
– ?- better(bronze,gold). (SECOND SUBQUERY WITH Z = BRONZE)• ?- better(bronze,Z’).• ?- better(Z’,gold).
– ?- better(bronze,Z’). (FIRST SUBQUERY FROM SECOND SUBQUERY ABOVE)
• Z’ = nothing
– ?- better(nothing,gold). (SECOND SUBQUERY WITH Z’ = NOTHING)• ?- better(nothing,Z”).• ?- better(Z”,gold).
– ?- better(nothing,Z”). (FIRST SUBQUERY FROM IMMEDIATELY ABOVE)
• ?- better(nothing,Z”’).• ?- better(Z”’,Z”).
– ?- better(nothing,Z”’). (OH NO!! WE’RE REPEATING OURSELVES AND GOING ROUND IN CIRCLES)
15
Prolog by Example
• What is happening to the computation tree?
– We’re in an infinite loop
• What does Prolog do with infinite loops?
– Nothing
– It just keeps going round and round generating sub-query after sub-query of the form ?- better(nothing,Zn). until memory is exhausted
• Zn n representing an ever increasing number of primes (‘)
– Then declares an error
• What would a more intelligent system do?
– Detect infinite loops automatically
– And declare failure to prove when one is encountered
Well...• Prolog has no loop detector
– Computational tradeoff: • too expensive computationally
speaking• to check on every inference whether
we are in a loop
– Besides, we can shift the burden to the programmer (you!)
• i.e. rely instead on the programmer to be smart enough to write rules that don’t generate infinite loops
16
Prolog by Example
• How can a programmer rewrite this rule to fix the problem?
– X is better than Y if X is better than (some) Z and Z is better than Ybetter(X,Y) :- better(X,Z), better(Z,Y).
• Definition is highly recursive– recursive in the sense that to prove
better, we call better itself as a subquery
– i.e. better is defined in terms of better
Notice: better calls better left recursively (FATAL MISTAKE)
i.e. better if true if better is true, and ...
Idea:
• restate definition without using left recursion
– X is better_than Y if X is better than Y or X is better than Z and Z is better_than Y
better_than(X,Y) :- better(X,Y).
better_than(X,Y) :-
better(X,Z), better_than(Z,Y).
left recursive:means leftmost (i.e. first) call is to the same named predicate
17
Prolog by Example
• New query:– ?- better_than(silver,gold).
• Computation tree:– ?- better_than(silver,gold).
• ?- better(silver,Z).• ?- better_than(Z,gold).
– ?- better(silver,Z). (FIRST SUBQUERY FROM ABOVE)• Z = bronze
– ?- better_than(bronze,gold). (SECOND SUBQUERY WITH Z = BRONZE)• ?- better(bronze,Z’).• ?- better_than(Z’,gold).
– ?- better(bronze,Z’). (FIRST SUBQUERY FROM SECOND SUBQUERY ABOVE)• Z’ = nothing
– ?- better_than(nothing,gold). (SECOND SUBQUERY WITH Z’ = NOTHING)• ?- better(nothing,Z”).• ?- better_than(Z”,gold).
– ?- better(nothing,Z”).
• No
better_than(X,Y) :- better(X,Y).better_than(X,Y) :- better(X,Z), better_than(Z,Y).
18
Recursion
• Difference now is – subquery fails instead of looping (GOOD!!)– [Smarter interpreter (not Prolog): fail when a loop is detected.]
• Another way to think about better_than:better_than(X,Y) :- better(X,Y).better_than(X,Y) :- better(X,Z), better_than(Z,Y).
startstate
final state
betterFact database:better(gold,silver).better(silver,bronze).better(bronze,nothing).
finite statemachine diagram
19
Recursion
• Yet another way to look at it– Given a relation R
• (transitivity)
• xRy & yRz xRz
Transform definition into:– Given relations R and R
• xRy xRy• xRy & yRz xRz• (non-left recursive
version)
• This is exactly what we did with “better”
• Givenbetter(X,Y) :- better(X,Z),
better(Z,Y).
• We transformed it into:better_than(X,Y) :- better(X,Y).
better_than(X,Y) :- better(X,Z), better_than(Z,Y).
Note:• the different directions for and :-
antecedent consequent
consequent :- antecedent
20
Recursion
• Recursion is a powerful concept– not unique to Prolog
• or to programming languages in general
– compact and powerful way to write certain kinds of definitions
• a way to code infinite sets in a finite manner
– use it carefully or your programs (or grammars) may not terminate
• given Prolog’s depth-first strategy
21
Prolog Resources (slide from Lecture 1)
• Useful Online Tutorials– An introduction to Prolog
• (Michel Loiseleur & Nicolas Vigier)• http://invaders.mars-attacks.org/
~boklm/prolog/
– Learn Prolog Now! • (Patrick Blackburn, Johan Bos &
Kristina Striegnitz)• http://www.coli.uni-saarland.de/~kris/
learn-prolog-now/lpnpage.php?pageid=online