Transcript
8/14/2019 Introduction to ILP
1/32
Introduction to ILP
ILP = Inductive Logic Programming
= machine learning logic programming
= learning with logic
Introduced by Muggleton in 1992
8/14/2019 Introduction to ILP
2/32
(Machine) Learning
he proce!! by which relatively permanent change!occur in behavioral potential a! a re!ult o" e#perience$(%nder!on)
Learning i! con!tructing or modi"ying repre!entation!o" what i! being e#perienced$ (Michal!&i)
% computer program i! !aid to learn"rom e#perience
Ewith re!pect to !ome cla!! o" ta!&! Tandper"ormance mea!ureP' i" it! per"ormance at ta!&! inT' a! mea!ured byP' improve! with e#perienceE$(Mitchell)
8/14/2019 Introduction to ILP
3/32
Machine Learning echniue!
eci!ion tree learning
*onceptual clu!tering
*a!e+ba!ed learning
,ein"orcement learning
-eural networ&! .enetic algorithm!
and/Inductive Logic Programming
8/14/2019 Introduction to ILP
4/32
0hy ILP + tructured data
eed e#ample o" 3a!t+0e!t train! (Michal!&i)
What makes a train to go eastward ?
8/14/2019 Introduction to ILP
5/32
0hy ILP 4 tructured data
Mutagenicity o" chemical molecule!
(5ing' riniva!an' Muggleton' ternberg' 1996)
0hat ma&e! a molecule to be mutagenic
8/14/2019 Introduction to ILP
6/32
0hy ILP 4 multiple relation!
hi! i! related to !tructured data
rain *ar
t1 c11
t1 c12
t1 c17t1 c16
t2 c21
/ /
*ar Length hape %#e! ,oo" /
c11 !hort rectangle 2 none /
c12 long rectangle 7 none /
c17 !hort rectangle 2 pea&ed /c16 long rectangle 2 none /
c21 !hort rectangle 2 "lat /
/ / / / / /
ha!8car car8propertie!
8/14/2019 Introduction to ILP
7/32
0hy ILP 4 multiple relation!
.enealogy e#ample
.iven &nown relation!/father(Old,Young)and mother(Old,Young) male(Somebody)and female(Somebody)
/learn new relation! parent(X,Y) :- father(X,Y).
parent(X,Y) :- mother(X,Y).
brother(X,Y) :-
male(X),father(Z,X),father(Z,Y).
Mo!t ML techniue! can:t u!e more than 1 relation
e$g$ deci!ion tree!' neural networ&!' /
8/14/2019 Introduction to ILP
8/32
0hy ILP 4 logical "oundation
Prolog = Programming with Logic
i! u!ed to repre!ent
4;ac&ground &nowledge (o" the domain) facts43#ample! (o" the relation to be learned) facts
4heorie! (a! a re!ult o" learning) rules
upport! 2 "orm! o" logical rea!oning4eduction
4Induction
8/14/2019 Introduction to ILP
9/32
Prolog + de"inition!
8/14/2019 Introduction to ILP
10/32
Logical rea!oning deduction
rom rule! to "act!/
; >+ 3mother(penelope'victoria)$
mother(penelope'arthur)$
"ather(chri!topher'victoria)$
"ather(chri!topher'arthur)$
parent(?'@) + "ather(?'@)$
parent(?'@) + mother(?'@)$
parent(penelope'victoria)$parent(penelope'arthur)$
parent(chri!topher'victoria)$
parent(chri!topher'arthur)$
8/14/2019 Introduction to ILP
11/32
Logical rea!oning induction
rom "act! to rule!/
; 3 >+ mother(penelope'victoria)$
mother(penelope'arthur)$
"ather(chri!topher'victoria)$
"ather(chri!topher'arthur)$
parent(?'@) + "ather(?'@)$
parent(?'@) + mother(?'@)$
parent(penelope'victoria)$parent(penelope'arthur)$
parent(chri!topher'victoria)$
parent(chri!topher'arthur)$
8/14/2019 Introduction to ILP
12/32
Induction o" a cla!!i"ier
or *oncept LearningMost studied task in Machine Learning
Given
4bac&ground &nowledge B
4a !et o" training e#ample! E
4a cla!!i"ication c C"or each e#ample e
Find a theory T(or hypothesis) !uch that
; >+ c(e)' "or all e 3
8/14/2019 Introduction to ILP
13/32
Induction o" a cla!!i"ier e#ample
3#ample o" 3a!t+0e!t train!
B relation! ha&%arand %arpropertie&
(length, roof, &hape, etc$)
e#$ ha&%ar(t1,%11), &hape(%11,bu%et)
E the train! t1to t1*
C ea&t, +e&t
8/14/2019 Introduction to ILP
14/32
0hy ILP + tructured data
eed e#ample o" 3a!t+0e!t train! (Michal!&i)
What makes a train to go eastward ?
8/14/2019 Introduction to ILP
15/32
Induction o" a cla!!i"ier e#ample
3#ample o" 3a!t+0e!t train!
B relation! ha&%arand %arpropertie&
(length, roof, &hape, etc$)
e#$ ha&%ar(t1,%11)
E the train! t1to t1*
C ea&t, +e&t
Po!!ible Tea&t() :-
ha&%ar(,), length(,&hort), roof(,).
8/14/2019 Introduction to ILP
16/32
Induction o" a cla!!i"ier e#ample
3#ample o" mutagenicity B relation! atomand bond
e#$atom(mol!",atom1,%,1/).
bond(mol!",atom1,atom",0).
E 27A molecule! with &nown cla!!i"ication
C a%ti'eand nona%ti'ew$r$t$ mutagenicity
Po!!ible Ta%ti'e(ol) :-
atom(ol,2,%,!!), atom(ol,3,%,1*),
bond(ol,2,3,1).
%!!
%1*
8/14/2019 Introduction to ILP
17/32
Learning a! !earch
Given
4 ;ac&ground &nowledge ;
4 heory e!cription Language T4 Po!itive! e#ample! P (cla!! B)
4 -egative e#ample! - (cla!! +)
4 % covering relation covers(B,T,e)
Find a theory that cover!
4 all po!itive e#ample! (completene!!)
4 no negative e#ample! (con!i!tency)
8/14/2019 Introduction to ILP
18/32
Learning a! !earch
*overing relation in ILP
cover!(;''e); >+ e
% theory i! a !et o" rule! 3ach rule i! !earched !eparately (e""iciency)
% rule mu!t be con!i!tent (cover no
negative!)' but not nece!!ary complete Separate-and-conquer!trategy
4,emove "rom P the e#ample! already covered
8/14/2019 Introduction to ILP
19/32
pace e#ploration
trategy
,andom wal&
4 ,edundancy' incompletene!! o" the !earch
y!tematic according to !ome ordering
4 ;etter control =C no redundancy' completene!!
4 he ordering may be u!ed to guide the !earch toward!
better rule!
What kind o ordering?
8/14/2019 Introduction to ILP
20/32
.enerality ordering
,ule 1 i! more general than rule 2
=C ,ule 1 cover! more e#ample! than rule 2
4 I" a rule i! con!i!tent (cover! no negative!)then every !peciali!ation o" it i! con!i!tent too
4 I" a rule i! complete (cover! all po!itive!)
then every generali!ation o" it i! complete too
Mean! to prune the !earch !pace 2 &ind! o" move! !peciali!ation and generali!ation
*ommon ILP ordering D+!ub!umption
8/14/2019 Introduction to ILP
21/32
.enerality ordering
parent(X,Y):-
parent(X,Y):- female(X) parent(X,Y) :- father(X,Y)
parent(X,Y) :- female(X), mother(X,Y)
parent(X,Y) :- female(X), father(X,Y)
%on&i&tent rule&pe%iali&ation
8/14/2019 Introduction to ILP
22/32
earch bia!e!
!"ias reers to any criterion or choosing one genera#i$ation
over another other than strict consistency with the
o%served training instances&' (Mitche##)
,e!trict the !earch !pace (e""iciency)
.uide the !earch (given domain &nowledge)
i""erent &ind! o" bia!
4 Language bia!
4 earch bia!
4 trategy bia!
8/14/2019 Introduction to ILP
23/32
*hoice o" predicate!
roof(,flat) 4 roof() 4 flat() 4
ype! o" predicate! ea&t() :- roof(), roof(,")
Mode! o" predicate!
ea&t() :- roof(,flat)ea&t() :- ha&%ar(,), roof(,flat)
i!cretiEation o" numerical value!
Language bia!
8/14/2019 Introduction to ILP
24/32
earch bia!
he move! direction in the !earch !pace
op+down
4!tart the empty rule (c(?) + $)4move! !peciali!ation!
;ottom+up
4!tart the bottom clau!e (F c(?) + ;$)4move! generali!ation!
;i+directional
8/14/2019 Introduction to ILP
25/32
trategy bia!
Geuri!tic !earch "or a be!t rule
Gill+climbing4 5eep only one rule
4 e""icient but can mi!! global ma#imum
;eam !earch4 al!o &eep krule! "or bac&+trac&ing
4 le!! greedy
;e!t+"ir!t !earch4 &eep all rule!
4 more co!tly but complete !earch
8/14/2019 Introduction to ILP
26/32
% generic ILP algorithm
procedure567(Examples)
5nitiali8e(Rules, Examples)
repeat
R9 Sele%t(Rules, Examples)
Rs9 efine(R, Examples)
Rules9 edu%e(Rules;Rs, Examples)untilStoppingriterion(Rules, Examples)
return(Rules)
8/14/2019 Introduction to ILP
27/32
% generic ILP algorithm
< 5nitiali8e(Rules,Examples):initialiEe a !et o"theorie! a! the !earch !tarting point!
< Sele%t(Rules,Examples):!elect the mo!t promi!ingcandidate rule R
< efine(R,Examples):return! the neighbour! o" R(u!ing !peciali!ation or generali!ation)
top related