111 Artificial Intelligence CS 165A Tuesday, November 6, 2007 Inference in FOL (Ch 9) 1.

111

Artificial Intelligence

CS 165A

Tuesday, November 6, 2007

Inference in FOL (Ch 9)

1

2

George Boole (1815-1864)

• More than 100 years later, he didn’t know about Leibniz, but proceeded to bring to life part of Leibniz’ dream

• His insight: Logical relationships are expressible as a kind of algebra– Letters represent classes (rather than numbers)

– So logic can be viewed as a form of mathematics

• Published The Laws of Thought

British

• He extended Aristotle's simple syllogisms to a broader range of reasoning– Syllogism: Premise_1, Premise_2 Conclusion– His logic: Propositional logic

3

Gottlob Frege (1848-1925)

• He provided the first fully developed system of logic that encompassed all of the deductive reasoning in ordinary mathematics.

• He intended for logic to be the foundation of mathematics – all of mathematics could be based on, and derived from, logic

• In 1879 he published Begriffsschrift, subtitled “A formula language, modeled upon that of arithmetic, for pure thought”

– This can be considered the ancestor of all current computer programming languages

– Made the distinction between syntax and semantics critical

German

• He invented what we today call predicate calculus (or first-order logic)

4

Notes

• HW#3 due Thursday

• Midterm exam one week from today– Short-answer conceptual questions and HW-like questions

– Topics: Reading, lectures, material through chapter 8 What AI is, main issues and areas in AI Problem solving and intelligent agents Blind search, informed search, adversarial search Logic and inference in propositional calculus Basics of first-order logic

– Closed book

– You may bring one 8.5”x11” piece of paper with your notes Some formulas, procedures will be given (posted in advance)

5

Reminder

• Term– Constant, variable, function( )

• Atomic sentence– Predicate( ), term1 = term2

• Literal– An atomic sentence or a negated atomic sentence

• Sentence– Atomic sentence, sentences with quantifiers and/or connectives

6

Simple example of inference in FOL

Bob is a buffalo

Pat is a pig

Buffaloes outrun pigs

Does Bob outrun Pat?

Buffalo(Bob)

Pig(Pat)

Buffalo(x) Pig(y) Outrun(x,y)

KB entails Outrun(Bob, Pat)?

KB0 |– Buffalo(Bob) Pig(Pat)

(And-Introduction)

KB1 |– Buffalo(Bob) Pig(Pat) Outrun(Bob, Pat)

(Universal Instantiation) [coming soon]

KB2 |– Outrun(Bob, Pat)

(Modus Ponens)

KB1

KB0

KB2

KB3

KB0

S

Review

7

Using FOL to express knowledge

• One can express the knowledge of a particular domain in first-order logic

• Example: The “kinship domain”– Objects: people– Properties: gender, family relationships– Unary predicates: Male, Female, MotherOf, FatherOf– Binary predicates: Parent, Sibling, Brother, Sister, Son,

Daughter, Father, Mother, Uncle, Aunt, Grandparent, Grandfather, Grandmother, Husband, Wife, Spouse, Brother-in-law, Stepmother, etc….

– Functions: MotherOf, FatherOf…

• Note: There is usually (always?) more than one way to specify knowledge

8

Kinship domain

• Write down what we know (what we want to be in the KB)– One’s mother is one’s female parent

m, c Mother(m, c) Female(m) Parent(m, c) m, c TheMotherOf(c) = m Female(m) Parent(m, c)

– One’s husband is one’s male spouse w, h Husband(h, w) Male(h) Spouse(h, w)

– One is either male or female x Male(x) Female(x)

– Parent-child relationship p, c Parent(p, c) Child(c, p)

– Grandparent-grandchild relationship g, c Grandparent(g, c) p Parent(g, p) Parent(p, c)

– Etc…

• Now we can reason about family relationships. (How?)

9

Kinship domain (cont.)

Assertions (“Add this sentence to the KB”)

TELL( KB, m, c Mother(c) = m Female(m) Parent(m, c) )

TELL( KB, w, h Husband(h, w) Male(h) Spouse(h, w) )

TELL( KB, x Male(x) Female(x) )

TELL( KB, Female(Mary) Parent(Mary, Frank) Parent(Frank, Ann) )– Note: TELL( KB, S1 S2 ) TELL( KB, S1) and TELL( KB, S2)

(because of and-elimination and and-introduction)

Queries (“Does the KB entail this sentence?”)

ASK( KB, Grandparent(Mary, Ann) ) True

ASK( KB, x Child(x, Frank) ) True– But a better answer would be { x / Ann }– This returns a substitution or binding

10

Implementing ASK: Inference

• We want a sound and complete inference algorithm so that we can produce (or confirm) entailed sentences from the KB

KB KB i

• The resolution rule, along with a complete search algorithm, provides a complete inference algorithm to confirm or refute a sentence in propositional logic(Sec. 7.5)

– Based on proof by contradiction (refutation)

• Refutation: To prove that the KB entails P, assume P and show a contradiction:

(KB P False) (KB P)Prove this!

11

Inference in First-Order Logic

• Inference rules for propositional logic:– Modus ponens, and-elimination, and-introduction, or-introduction,

resolution, etc.

– These are valid for FOL also

• But since these don’t deal with quantifiers and variables, we need new rules, especially those that allow for substitution (binding) of variables to objects– These are called lifted inference rules

12

Substitution and variable binding

• Notation for substitution:– SUBST ( Binding list, Sentence )

Binding list: { var / ground term, var / ground term, … } “ground term” = term with no variables

– SUBST( {var/gterm}, Func (var) ) = Func (gterm) SUBST (, p)

– Examples: SUBST ( {x/Mary}, FatherOf (x) ) = FatherOf (Mary) SUBST ( {x/Joe, y/Lisa}, Siblings (x,y) ) = Siblings (Joe, Lisa)

13

Three new inference rules using SUBST(, p)

• Universal Instantiation

)},/({

gvSUBST

v

• Existential Instantiation

)},/({

kvSUBST

v k – constant that does not appear elsewhere in the knowledge base

g – ground term

• Existential Introduction

)},/({

vgSUBSTvv – variable not in g – ground term in

14

To Add to These Rules

15

Universal Instantiation – examples

)},/({

gvSUBST

v g – ground term

x Sleepy(x)– SUBST({x/Joe}, )

Sleepy(Joe)

x Mother(x) Female(x)– SUBST({x/Mary}, )

Mother(Mary) Female(Mary)– SUBST({x/Dad}, )

Mother(Dad) Female(Dad)

x, y Buffalo(x) Pig(y) Outrun(x,y)– SUBST({x/Bob}, )

y Buffalo(Bob) Pig(y) Outrun(Bob,y)

16

Existential Instantiation – examples

)},/({

kvSUBST

v k – constant that does not appear elsewhere in the knowledge base

x BestAction(x)– SUBST({x/B_A}, )

BestAction(B_A)– “B_A” is a constant; it is not in our universe of actions

y Likes(y, Broccoli)– SUBST({y/Truman}, )

x Likes(Truman, Broccoli)– “Truman” is a constant; it is not in our universe of people

17

Existential Introduction – examples

)},/({

vgSUBSTvv – variable not in g – ground term in

• Likes(Jim, Broccoli)– SUBST({Jim/x}, )

x Likes(x, Broccoli)

x Likes(x, Broccoli) Healthy(x)– SUBST({Broccoli/y}, )

y x Likes(x, y) Healthy(x)

18

What’s our goal here?

• Formulate a search process:– Initial state

KB– Operators

Inference rules– Goal test

KB contains S

• What is a node?– KB + new sentences (generated by applying the inference rules)– In other words, the new state of the KB

• What kind of search to use?– I.e., which node to expand next?

• How to apply inference rules? – Need to match the premise pattern

19

Applying inference rules

• FOL query– ASK(KB, Outruns(Bob, Pat))

– ASK(KB, x BestAction(x, t))

– ASK(KB, x Equals(x, AuntOf(HusbandOf(Mary))))

• Proof – From axioms (KB) to hypothesis (S)– Does the KB entail a sentence S – if so, what steps were taken?

• We need to apply inference rules and appropriate substitutions to reach the desired sentence– Answer: Yes, SUBST({x/Grab}, Action(x, t))

– Answer: Yes, the KB entails S (and here’s how…)

• Reasoning systems may require the solution path, not just the final answer

Path = proof

Path shows reasoning (here’s why I think you have cancer)

20

Concerns

• Is the inference procedure sound and complete?

• Is it efficient?– We need to be concerned about the branching factor

– Generalized modus ponens (GMP) is one way of reducing branching factor

• How do we know which operators (inference rules) are valid at any particular time?– Inference rule: Pig(x) Cuddly(x)

– KB: Pig(Bob)

Pig(x) and Pig(Bob) are unified by the substitution = {x/Bob}

21

Problem of branching factor

• 10 inference rules– Branching factor can be huge!

• Common inference pattern:– And-Introduction

– Universal Instantiation

– Modus ponens

• Let’s introduce a new inference rule which reduces the branching factor by combining these three steps into one– Generalized Modus Ponens

Buffaloes outrun pigs example

22

Generalized Modus Ponens

• For atomic sentences pi , pi' , and q , where there is a substitution such that SUBST(, pi') = SUBST(, pi) for all i, then

),(

)(,,,, 2121

qSUBST

qpppppp nn

• GMP features:– Efficient – it combines several inferences into one

– Sensible – substitutions are purposeful (unification)

– Efficient – when sentences are in a certain canonical form (Horn clauses), this is the only inference rule necessary (!!!)

Specific facts General rule

Instantiated conclusion

23

Generalized Modus Ponens Example

• Go from– Buffalo(Bob), Pig(Pat), (Buffalo(x) Pig(y) Outrun(x,y))

to

– Outrun(Bob, Pat)

• More generally, go from– p1', p2', (p1 p2

q)

to

– SUBST(, q)

where SUBST(, pi') = SUBST(, pi)

• Reminder: What is ?

24

Unification

• Unification takes two atomic sentences p and q and returns a substitution that would make p and q look the same (or else it fails)– UNIFY(p, q) = where SUBST(, p) = SUBST(, q) is the unifier of the sentences p and q

• Examples– p = Knows(John, x) q = Knows(John, Jane)

= {x/Jane}

– p = Knows(John, x) q = Knows(y, Fred) = {x/Fred, y/John}

– p = Knows(John, x) q = Knows(y, MotherOf(y)) = {x/MotherOf(John), y/John}

– p = Knows(John, x) q = Knows(x, Mary) = fail

– p = Knows(John, x1) q = Knows(x2, Mary)

= {x1/Mary, x2/John}

25

From GMP to GR

• GMP requires sentences in Horn clause

• Prolog is based on this: Horn clause form and GMP inference– Horn clause: an implication of positive literals

• It is “reasonably powerful,” but – Cannot handle disjunctions or negations

– Many legal sentences cannot be expressed in a Horn clause

– I.e., this form is incomplete

• It turns out we can do better by putting the sentences of the KB into a different format and using just one inference rule, which is both complete and sound!

– KB Format: Conjunctive normal form (CNF)

– Inference Rule: Generalized (first-order) resolution

26

Note: Completeness

• An inference procedure i is complete iff– KB i whenever KB

– “The procedure finds the needle in the haystack when there is one”

• An inference procedure using GMP is incomplete– There are sentences entailed by the KB that the procedure cannot

infer

• A complete inference procedure exists: Generalized Resolution– Generalized resolution will produce if KB entails – However, what if is not entailed by the KB? Will resolution tell us

this? No, there exists no certain procedure to show this in general

– Related to the famous “Halting problem”

27

Resolution

rp

rqqp

,

Simple resolution

Let’s rewrite it:rp

rqqp

,

ca

cbba

,or:

Remember: p q p q

So resolution is really the “chaining” of implications….

Review

28

Generalized (first-order) resolution

• Generalized resolution is the basis for a complete inference procedure– Can derive new implications (P Q)

GMP can only derive atomic conclusions (Q)

– Any sentence can be put into its normal form

• Just like “regular” resolution except– Uses UNIFY/SUBST rather than =

– Deals with more than 2 disjuncts

• GR is complete, but semidecidable– Not always able to show that a sentence is not entailed by the KB

29

Reminder: Atomic sentences vs. Literals

• Atomic sentences– Predicate and its arguments (terms)

P(x), Q(y), Siblings(Bill, z), Equal(x, y)

• Literals– Atomic sentences and negated atomic sentences

P(x), Q(y), Siblings(Bill, z), Equal(x, y)

30

Two versions of Generalized Resolution

kj qandpexceptnm

nk

mj

qqppSUBST

qqq

ppp

),( 11

1

1

Disjunctions

For literals pi and qi , where UNIFY(pj , qk) =

kj qandpexceptnnnn

nkn

nnj

qqrrssppSUBST

qqqss

rrppp

),(4231

43

21

1111

11

11

Implications

For atomic sentences pi, qi, ri, si , where UNIFY(pj , qk) =