SRL Approaches: Frame-based Probabilistic models February 11, 2005.
Post on 21-Dec-2015
216 Views
Preview:
Transcript
Today’s Outline
• Finish w/ Graphical Models Introduction• Families of SRL Approaches• Frame-based Probabilistic approaches
– Probabilistic Relational Models (PRMs)– Probabilistic Entity Relation (PERs)
SRL History
• In general, SRL combines logic and probabilities• Historically, there are two general threads of
research– The first takes graphical models or hierarchical
Bayesian models and adds in some form of relational/logical representation
• examples: Probabilistic Relational Models (PRMs), Probabilistic Entity Relation Models (PERs), Object Oriented Bayesian Networks (OOBNs)
• comes largely from the Uncertainty in AI (UAI} community
– The second takes a logical representation (first-order logic, horn clauses, etc) and adds in some form of probabilities
• examples: Bayesian Logic Programs (BLPs), Stochastic Logic Programs (SLPs)
• comes largely from the Inductive Logic Programming (ILP) community
Families of SRL Approaches
1. Frame-based Probabilistic Models• Probabilistic Relational Models (PRMs), • Probabilistic Entity Relation Models (PERs), • Object Oriented Bayesian Networks (OOBNs)
2. First Order Probabilistic Logic (FOPL)• BLOGs• Relational Markov Logic (RML)
3. Stochastic Functional Programs• PRISM• Stochastic Logic Programs (SLPs)• IBAL
SRL Dimensions
• Syntax – ‘logic-based’ vs. ‘schema-based’
• Logical Semantics –– relational vs. first-order– domain closure/closed world vs. open world
• Probabilistic Semantics – – ‘possible worlds’ vs. ‘domain frequencies’– directed vs. undirected models
• … others?
Today: PRMs
• Developed by Daphne Koller’s group at Stanford– representation: Avi Pfeffer
• builds on work in KBMC (knowledge-based model construction) by Haddawy, Poole, Wellman and others…
• Object Oriented Bayesian Networks • Relational Probability Models
– learning: myself, Nir Friedman, Avi• Attribute Uncertainty• Structural Uncertainty• Class Uncertainty• Identity Uncertainty
– undirected models: Ben Taskar
Learning Statistical Models
Traditional approaches– work well with flat representations– fixed length attribute-value vectors – assume independent (IID) sample
Patient
flatten
Problems:– introduces statistical skew– loses relational structure
• incapable of detecting link-based patterns
– must fix attributes in advance
Contact
Roadmap
• Background: » Bayesian Networks (BNs) [Pearl, 1988]– Probabilistic Relational Models (PRMs)
• Learning PRMs w/ Attribute Uncertainty
• PRMs w/ Structural Uncertainty
• PRMs w/ Class Hierarchies
Bayesian Networks
nodes = random variablesedges = direct probabilistic
influence
Network structure encodes independence assumptions: XRay conditionally independent of Pneumonia given Infiltrates
XRay
Lung Infiltrates
Sputum Smear
TuberculosisPneumonia
Bayesian Networks
XRay
Lung Infiltrates
Sputum Smear
TuberculosisPneumonia
• Associated with each node Xi there is a conditional probability distribution P(Xi|Pai:) — distribution over Xi for each assignment to parents
– If variables are discrete, P is usually multinomial– P can be linear Gaussian, mixture of Gaussians, …
0.8 0.2
p
t
p
0.6 0.4
0.010.99
0.2 0.8
tp
t
t
p
TP P(I |P, T )
BN Semantics
• Compact & natural representation:– nodes have k parents 2k n vs. 2n params
conditionalindependenciesin BN structure
+local
probabilitymodels
full jointdistribution
over domain=
t)|sP(i)|P(xt),p|P(iP(t))pP()sx,i,t,,pP(
X
I
S
TP
Roadmap
• Background: • Bayesian Networks (BNs)» Probabilistic Relational Models (PRMs)
• Learning PRMs w/ Attribute Uncertainty
• PRMs w/ Structural Uncertainty
• PRMs w/ Class Hierarchies
Probabilistic Relational Models
• Combine advantages of relational logic & Bayesian networks: – natural domain modeling: objects, properties,
relations;– generalization over a variety of situations;– compact, natural probability models.
• Integrate uncertainty with relational model:– properties of domain entities can depend on
properties of related entities;– uncertainty over relational structure of domain.
Relational Schema
Strain
Unique
Infectivity
Infected with
Interacted with
• Describes the types of objects and relations in the database
ClassesClasses
RelationshipsRelationshipsContact
Close-Contact
Skin-Test
Age
Patient
Homeless
HIV-Result
Ethnicity
Disease-Site AttributesAttributes
Contact-Type
Probabilistic Relational Model
Close-Contact
Transmitted
Contact-Type
Disease Site
Strain
Unique
Infectivity
Patient
Homeless
HIV-Result
POB
Contact Age
Cont.Contactor.HIVCont.Close-Contact
Cont.Transmitted |
P
4.06.0
3.07.0
2.08.0
1.09.0
,
,
,
,,
tt
ft
tf
ff
P(T | H, C)CH
Relational Skeleton
Fixed relational skeleton – set of objects in each class– relations between them
Uncertainty over assignment of values to attributes
PRM defines distribution over instantiations of attributes
Strains1
Patientp2
Patientp1
Contactc3
Contactc2
Contactc1
Strains2
Patientp3
A Portion of the BN
P1.Disease Site
P1.Homeless
P1.HIV-Result
P1.POB
C1.Close-Contact
C1.Transmitted
C1.Contact-Type
C1.Age
C2.Close-Contact
C2.Transmitted
C2.Contact-Type
truefalse
true
4.06.0
3.07.0
2.08.0
1.09.0
,
,
,
,,
tt
ft
tf
ff
P(T | H, C)CH
4.06.0
3.07.0
2.08.0
1.09.0
,
,
,
,,
tt
ft
tf
ff
P(T | H, C)CH
C2.Age
PRM: Aggregate Dependencies
sum, min, max, avg, mode, count
Disease Site
Patient
Homeless
HIV-Result
POB
Age
Close-Contact
Transmitted
Contact-Type
Contact
Age
.
.
PatientJane Doe
POB US
Homeless no
HIV-Result negative
Age ???
Disease Site pulmonary
A
.
Contact#5077
Contact-Typecoworker
Close-Contact no
Agemiddle-aged
Transmitted false
Contact#5076
Contact-Typespouse
Close-Contact yes
Agemiddle-aged
Transmitted true
Contact#5075
Contact-Typefriend
Close-Contact no
Agemiddle-aged
Transmitted false
mode
6.03.01.0
2.06.02.0
2.04.04.0
o
m
yomym
PRM with AU Semantics
)).(|.(),S,|( ,.
AxparentsAxPP Sx Ax
I
AttributesObjects
probability distribution over completions I:
PRM relational skeleton + =
Strain
Patient
Contact
Strain s1
Patient p1
Patient p2
Contactc3
Contactc2
Contactc1
Strain s2
Patient p3
Next Time 2/18• Structural Uncertainty
– Learning Probabilistic Models of Link Structure, L. Getoor, N. Friedman, D. Koller, B. Taskar. Journal of Machine Learning Research, 2002.
– http://www.cs.umd.edu/class/spring2005/cmsc828g/Readings/jmlr02.pdf
• Class Uncertainty– PRMs with Class Hierarchies, chapter 5 of Learning Statistical
Models of Relational Data, Lise Getoor, PhD Thesis, Stanford University, 2001.
– http://www.cs.umd.edu/class/spring2005/cmsc828g/Readings/thesis-ch5.pdf
• PERS– Probabilistic Models for Relational Data, David Heckerman,
Christopher Meek and Daphne Koller– ftp://ftp.research.microsoft.com/pub/tr/TR-2004-30.pdf
• Background on Learning Graphical Models available in CS Library
Today’s Outline 2/18
• Frame-based Probabilistic approaches– Probabilistic Relational Models (PRMs)
• Learning PRMs• PRMs w/ Structural Uncertainty• PRMs w/ Class Hierarchies
– Probabilistic Entity Relation (PERs)
Learning PRMs w/ AU
Database Patient
Strain
Contact
Relational
Schema
PatientContact
Strain
• Parameter estimation• Structure selection
Parameter Estimation in PRMs• Assume known dependency structure S• Goal: estimate PRM parameters
– entries in local probability models,
• is good if it is likely to generate the observed data, instance I .
• MLE Principle: Choose so as to maximize l
),|(log),:( SPSl II
).(|. AxparentsAx
As in Bayesian network learning,
crucial property: decomposition
separate terms for different X.A
ML Parameter Estimation
ContactCloseContact
Transmitted
PatientHIV
DiseaseSite
Count
Query for counts:
Patienttable
Contacttable
ctCloseContaC
HIVP
dTransmitteC
.
.
.
).,.().,.,.(
tCCfHPNtCCfHPfTCN
P
??
??
??
??
,
,
,
,,
tt
ft
tf
ff
P(T | H, C)CH
Cont.Contactor.HIVCont.Close-Contact
Cont.Transmitted |
P
Structure Selection
• Idea: – define scoring function – do local search over legal structures
• Key Components:– legal models – scoring models– searching model space
Structure Selection
• Idea: – define scoring function – do local search over legal structures
• Key Components:» legal models– scoring models– searching model space
Legal Models
author-of
• PRM defines a coherent probability model over a skeleton if the dependencies between object attributes is acyclic
How do we guarantee that a PRM is acyclic for every skeleton?
ResearcherProf. Gump
Reputationhigh
PaperP1
Accepted yes Paper
P2Accepted
yes
sum
Attribute Stratification
PRMdependency structure S
dependencygraph
Paper.Accecpted
Researcher.Reputation
if Researcher.Reputation depends directly on Paper.Accepted
dependency graph acyclic acyclic for any Attribute stratification:
Algorithm more flexible; allows certain cycles along guaranteed acyclic relations
Structure Selection
• Idea: – define scoring function – do local search over legal structures
• Key Components:– legal models» scoring models– searching model space
Scoring Models
• Bayesian approach:
])()|(log[)|(log):(
priorlikelihoodmarginal
SPSPSPSScore
III
• Standard approach to scoring models; used in Bayesian network learning
Structure Selection
• Idea: – define scoring function – do local search over legal structures
• Key Components:– legal models – scoring models» searching model space
Searching Model Space
Contact
Strain Patient
score
Delete C.CC.T Contact
Strain Patient
scoreAdd S.IS.U
Strain Contact
Patient
Phase 0: consider only dependencies within a class
Contact
Strain Patient scoreAdd S.IP.D
score
Add P.HC.TContact
Strain Patient
Contact
PatientStrain
Phase 1: consider dependencies from “neighboring” classes, via schema relations
Phased Structure Search
Phased Structure Search
scoreAdd S.IC.T
score
Add C.PS.I
Phase 2: consider dependencies from “further” classes, via relation chains
Contact
Strain Patient
Contact
Strain Patient
Contact
Strain Patient
Synthetic Data
• Simple ‘genetic’ domain• Construct training set of various sizes• Compare the log-likelihood of test set of
size 100,000– ‘gold’ standard model– Learn parameters (model structure given)– Learn model (learn both structure and
parameters)
Blood Type
M-chromosome
P-chromosome Person
Result
Contaminated
Blood Test
Blood Type
M-chromosome
P-chromosome
Person Blood Type
M-chromosome
P-chromosome
Person
(Father)
(Mother)
Error on Test Set
-3
-2.5
-2
-1.5
-1
-0.5
0
0 1000 2000 3000 4000
Dataset Size
Av
g L
og
-Lik
elih
oo
d
Gold
Learned Parameters
Learned Models
Error Variance
0
0.5
1
1.5
2
2.5
0 1000 2000 3000 4000
Dataset Size
Av
g E
rro
rLearned Parameters
Learned Models
Errors in Learned Structure
0
2
4
6
8
10
12
500 1300 1800 2500 3000 3800 4300
Dataset Size
Nu
mb
er
of
Le
arn
ed M
od
els
too simple
correct
too complex
TB Cases in SF
Patient (2300)Ethnicity
Homeless
Age @ diagnosis
HIV result
Disease-site
X-ray
Contact (20000)Contact-type
Age
Care
Infected
Strain (1000)
Unique
Drug-Resistance
hivres
# contacts
result
transmitted
infectivity
smrpos
care
closecont
ageatdx
closecont
hh_oohh
ethnic
# infected
% infected
hh_oohh
contype
homeless
gender
contype
disease site
contage
xray
pob
ContactStrain
Subcase
Patient
TB PRM
total assets
# roles
rtn earn assets
age
rtn assets
fired
# employees
top_roletop_role
total_assets
retired retired
salary salary
Company
Role
Prev-Role
Person
SEC PRM
20,000
120,000
40,000
Your turn…
• Describe your focus problem
• What would a PRM for (an aspect of) your focus problem look like?
Roadmap
• Motivation and Background
• PRMs w/ Attribute Uncertainty
» PRMs w/ Structural Uncertainty
• PRMs w/ Class Hierarchies
An Example
Topic
Theory AI
Agent
Theory papers
Cornell
Scientific Paper
Topic
Theory AI
•Attributes of object•Attributes of linked objects
•Attributes of heterogeneous linked objects•Collective Classification
Structural Uncertainty
• Motivation: relational structure provides useful information for density estimation and prediction
• Construct probabilistic models of relational structure that capture structural uncertainty
• Two new mechanisms:– Reference uncertainty– Existence uncertainty
PRMs w/ AU: another example
Vote
Rank
Movie
Income
Gender
Person
AgeGenre
PRM consists of:
Relational Schema
Dependency Structure
Vote.Person.Gender,Vote.Person.Age
Vote.Movie.Genre,Vote.Rank |
P
Local Probability Models
Fixed relational skeleton :– set of objects in each class– relations between them
Movie m1
Vote v1 Movie: m1 Person: p1
Person p2
Person p1
Movie m2
Uncertainty over assignment of values to attributes
PRM w/ Attribute Uncertainty
Vote v2 Movie: m1 Person: p2
Vote v3 Movie: m2 Person: p2
Primary Keys
Foreign Keys
PRM w/ AU Semantics
)).(|.(),S,|( ,.
AxparentsAxPP Sx Ax
I
AttributesObjects
Ground BN defining distribution over complete instantiations of attributes I:
PRM relational skeleton + =
Patient p2
Vote
Movie Person Movie
Vote
Vote
Person
Person
Movie
Vote
Issue
• PRM w/ AU applicable only in domains where we have full knowledge of the relational structure
Next we introduce PRMs which allow uncertainty over relational structure…
PRMs w/ Structural Uncertainty
Advantages:– Applicable in cases where we do not have full
knowledge of relational structure– Incorporating uncertainty over relational structure
into probabilistic model can improve predictive accuracy
Two approaches:– Reference uncertainty– Existence uncertainty
• Different probabilistic models; varying amount of background knowledge required for each
Citation Relational Schema
Wrote
PaperTopic
Word1
WordN
…Word2
PaperTopic
Word1
WordN
…Word2Cites
CountCiting Paper
Cited Paper
AuthorInstitution
Research Area
Attribute Uncertainty
Paper
Word1
Topic
WordN
Wrote
Author
...
Research Area
P( WordN | Topic)
P( Topic | Paper.Author.Research Area
Institution P( Institution | Research Area)
PRM w/ Reference Uncertainty
CitesCitingCited
Dependency model for foreign keys
PaperTopicWords
PaperTopicWords
Naïve Approach: multinomial over primary key• noncompact• limits ability to generalize
Reference Uncertainty Example
PaperP5
Topic AI
PaperP4
Topic AI
PaperP3
Topic AI
PaperM2
Topic AI
Paper P1Topic Theory
CitesCitingCited
Paper P5Topic AI
PaperP3
Topic AI
Paper P4Topic Theory
Paper P2Topic Theory
Paper P1Topic Theory
Paper.Topic = AIPaper.Topic = Theory
P1
P2
PaperTopicWords P1 P2
3.0 7.0
P1 P2
1.0 9.0
Topic
99.0 01.0 Theory
AI
PRMs w/ RU Semantics
PRM-RU + entity skeleton
probability distribution over full instantiations I
Cites
Cited
Citing
PaperTopic
Words
PaperTopic
Words
PRM RU
Paper P5Topic AI
Paper P4Topic Theory
Paper P2Topic Theory
Paper P3Topic AI
Paper P1Topic ???
Paper P5Topic AI
Paper P4Topic Theory
Paper P2Topic Theory
Paper P3Topic AI
Paper P1Topic ???
RegReg
RegRegCites
entity skeleton
Structure Search: New Operators
CitesCitingCited
PaperTopicWords
PaperTopicWords
Cited
Papers
1.0
Paper Paper
Paper Paper
Paper Paper
Paper Paper
Paper Paper
Paper
Topic = AI
ΔscoreRefine on Topic
Paper Paper
Paper Paper
Paper
Paper Paper
Paper Paper
Paper
Paper Paper
Paper Paper
Paper
Paper
Paper Paper
Δscore
Refine on Author.Instition
AuthorInstitution
Institution = MIT
PRMs w/ RU Summary
• Define semantics for uncertainty over foreign-key values
• Search now includes operators Refine and Abstract for constructing foreign-key dependency model
• Provides one simple mechanism for link uncertainty
PRM w/ Exists Uncertainty
Cites
Dependency model for existence of relationship
PaperTopicWords
PaperTopicWords
Exists
Exists Uncertainty Example
Cites
PaperTopicWords
PaperTopicWords
Exists
Citer.Topic Cited.Topic
0.995 0005 Theory Theory
False True
AI Theory 0.999 0001
AI AI 0.993 0008
AI Theory 0.997 0003
PRMs w/ EU Semantics
PRM-EU + object skeleton
probability distribution over full instantiations I
Paper P5Topic AI
Paper P4Topic Theory
Paper P2Topic Theory
Paper P3Topic AI
Paper P1Topic ???
Paper P5Topic AI
Paper P4Topic Theory
Paper P2Topic Theory
Paper P3Topic AI
Paper P1Topic ???
object skeleton
???
PRM EU
Cites
Exists
PaperTopic
Words
PaperTopic
Words
Learning PRMs w/ EU
• Idea: just like in PRMs w/ AU– define scoring function – do greedy local structure search
• Issues:– efficiency
•Computation of sufficient statistics for exists attribute
•Do not explicitly consider relations that do not exist
Experiment I: EachMovie+
thriller
action
horror
gender
theater_status gendervideo_status
age
animationart_foreign
classic
personal_income
comedy
drama
rankhousehold_income
family
romance Movie
Person
Movie
Actor
MOVIE
ROLE
VOTEPERSO
N
ACTOR
education
* © 1999 -2000 Internet Movie Database Limited† http://www.research.digital.com/SRC/EachMovie
Size: 1600
Size: 35,000Size: 50,000
Size: 25,000Size: 300,000
*
†
EachMovie+ PRM-RU
thriller
actionhorror
gender
theater_status
gender
video_status
ageanimation
art_foreign
classic
personal_income
comedy
dramarank
household_incomefamily
romanceMovie
Person
Movie
Actor
MOVIE
ROLE
VOTE PERSON
ACTOR
education
M F
8.0 2.0
Action
7.0 3.0true
false
Typical Voter: male, young adult, college w/o degree, middle income
EachMovie+ PRM-EU
agecomedy
drama rank
gender
family
personal_income
horror
romance
exists
household_income
thriller
exists
gendertheater_status
video_status
action
education
animation
art_foreign
classic
MOVIE
ROLE
VOTEPERSO
N
ACTOR
+
-
Men much more likely to vote on action movies
Experiment II: Prediction
Paper P506
Paper P516Topic Reinforcement LearningWords
…Paper P1309Topic Probabilistic ReasoningWords
…Paper P289Topic Reinforcement LearningWords
…
Cited Papers
Paper P134Topic Reinforcement LearningWords
…Paper P1067Topic Reinforcement LearningWords
…
Citing Papers
Topic ??
w1 wN. . .
Domains
Cites
Exists
PaperTopic
w1 wN. . .
PaperTopic
w1 wN. . .
cited paper citing paper
Cora Dataset, McCallum, et. al
Link
Exists
Web PageCategory
w1 wN. . .
Category
w1 wN. . .
From Page To Page
Web Page
WebKB, Craven, et. al
Prediction Accuracy
Naïve-bayesRU Citing RU Cited ExistsCora 0.75 0.81 0.79 0.85WebKB 0.74 0.78 0.77 0.82
0.65
0.7
0.75
0.8
0.85
0.9
Cora WebKB
Acc
ura
cy
Naive-Bayes
RU Citing
RU Cited
Exists
Experiment III: Collective Classification
Paper#2 Topic Paper#3Topic
WordN
Paper#1Word1
Topic... ... ...
Author#1
Area Inst
#1-#2
Author#2
Area Inst
Exists
#2-#3
Exists
#2-#1
Exists
#3-#1
Exists
#1-#3
Exists
WordN
Word1WordN
Word1
Exists
WordNWord1
WordN
Word1WordN
Word1
ExistsExists Exists ExistsExists Exists
Inst Inst
TopicTopicTopic
Area Area
TopicTopicTopic
Area Area
Topic TopicTopic
Area Area
#3-#2
Inference in Unrolled BN
• Prediction requires inference in “unrolled” network– Infeasible for large networks– Use approximate inference for E-step
• Loopy belief propagation (Pearl, 88; McEliece, 98)– Scales linearly with size of network– Guaranteed to converge only for polytrees– Empirically, often converges in general nets
(Murphy,99)
• Local message passing– Belief messages transferred between related instances– Induces a natural “influence” propagation behavior
• Instances give information about related instances
...
From-Page Category
Word1 WordN
Exists
From
To
Link
Hub
To-Page
Word
AnchorHas
...
Category
Word1 WordN
Hub
Web Domain
WebKB Results*
0.54
0.56
0.58
0.6
0.62
0.64
0.66
0.68
0.7
cornell texas wisconsin washington
School
Acc
ura
cy
Naive-Bayes
Exists
Ex+Hubs+Anchors
* from “Probabilistic Models of Text and Link Structure for Hypertext Classification”, Getoor, Segal, Taskar and Koller in IJCAI 01 Workshop Text Learning: Beyond Classification
Roadmap
• Motivation and Background
• PRMs w/ Attribute Uncertainty
• PRMs w/ Structural Uncertainty
» PRMs w/ Class Hierarchies
From Instances to Classes in Probabilistic Relational Models
• Compare two approaches – Probabilistic Relational Models (PRMs)– Bayesian Network (BNs)
• PRMs with Class Hierarchies (PRM-CH)– bridge gap between BNs and PRMs
• Learning PRM-CHs– hierarchy supplied– discovering hierarchy
VoteProgram
Voter
Ranking
PRM for Collaborative Filtering
VoteProgram
Voter
Ranking IncomeIncomeIncome
1.06.03.0
5.04.01.0
4.05.01.0
1.04.05.0
bs
hssitcom
bsdoc
hsdoc
hmlEG
sitcom
+ Dependency Model
TV-ProgramGenre
Budget
Time-slot
Network
TV-ProgramGenre
Budget
Time-slot
Network
Relational Schema
PersonAge
Gender
Education
BN for Collaborative filtering
Law & Order
Frasier
NBC MondayNight Movies
Mad about you
Beverly Hills 90210
Seinfeld Friends
Melrose Place
Models Inc.
Breese, et al. UAI-98
Limitations of PRMs
• In PRM, all instances of the same classmust use the same dependency mode,it cannot distinguish:– documentaries and sitcoms – “60 Minutes” and Seinfeld
• PRM cannot have dependencies that are“cyclic”– ranking for Frasier depends on ranking for
Friends
Limitations of BNs
• In BN, each instance has its own dependency model, cannot generalize over instances– If John tends to like sitcoms, he will probably like
next season’s offerings– whether a person enjoys sitcom reruns depends
on whether they watch primetime sitcoms
• BN can only model relationships between atmost one class of instances at a time– In previous model, cannot model relationships
between people – if my roommate watches Seinfeld I am more
likely to join in
Desired Model
Allows both class and instance dependencies
WWWF
Person
Age
Gender
Education
Income
Soap Genre
Budget
Time-slot
Network
Genre
Budget
Time-slot
Network
Documentary
Sitcom-VoteProgram
Voter
Ranking
Doc-Vote
Program
Voter
Ranking
Vote
Program
Voter
Ranking
TV-Program Genre
Budget
Time-slot
Network
PRMs w/ Class Hierarchies
Allows us to:• Refine a “heterogenous” class into
more coherent subclasses• Refine probabilistic model along class
hierarchy– Can specialize/inherit CPDs– Construct new dependencies that were
originally “acyclic”Provides bridge from class-based model
to instance-based model
PRM-CH
PersonAge
TV-ProgramGenreBudgetTime-slotNetwork
GenderEducationIncome
VoteProgramVoterRanking
Relational Schema
Class Hierarchy
SoapOpera
TV-Program
SitCom DocumentaryDrama
Legal-Drama Medical-Drama
Dependency Model
BudgetSoapOpera
BudgetTV -Program
BudgetSitCom BudgetDocumentaryBudgetDrama
Budget Legal-Drama
BudgetMedical-Drama
Learning PRM-CHs
Relational
Schema
Database:
TVProgram Person
Vote
Person
Vote
TVProgram
Instance I
• Class hierarchy provided
• Learn class hierarchy
Structure Selection
• Idea: – define scoring function – do phased local search over legal
structures
• Key Components:– scoring models
– searching model space
PRM w/ CHs
new operators
unchanged
• Scenario 1: Class hierarchy is provided
• New Operators– Specialize/Inherit
Learning PRM-CH
BudgetSoapOpera
BudgetTV -Program
BudgetSitCom BudgetDrama
Budget Legal-Drama
BudgetMedical-Drama
BudgetDocumentaryBudgetDocumentar
y
Learning Class Hierarchy • Issue: partially observable data set• Construct decision tree for class defined over
attributes observed in training set
TV-Program.Genre
sitcomdrama
class1 class3
documentary
class2
class4
English
TV-.Network.Nationality
class5
French
class6
American
• New operator
– Split on class attribute– Related class attribute
MOVI
E
Animation
Family
Drama
Comedy
Romance
Action
Horror
Thriller
Theater Status
Video Status
Art/Foreign
Classic
VOTE
Rating PERSON
Gender
Age
Personal Income
Household Income
Education
EachMovie+ PRM
1400 Movies5000 People
240,000 Votes
http://www.research.digital.com/SRC/EachMovie
Theater Status
Video Status
Art/Foreign
ClassicDrama
ROMANCE-MOVIE
Animation
Family
Horror
Thriller
Gender
Age
Personal Income
Household Income
Education
PERSON
ROMANCE-VOTERating
OTHER-VOTERating
COMEDY-VOTERating
ACTION-VOTE
Rating
Theater Status
Video Status
Art/Foreign
ClassicDrama
ACTION-MOVIE
Animation
Family
Horror
Thriller
Theater Status
Video Status
Art/Foreign
ClassicDrama
COMEDY-MOVIE
Animation
Family
Horror
Thriller
PRM-CH
Theater Status
Video Status
Art/Foreign
Classic
OTHER-MOVIE
ThrillerDrama
Horror
Animation
Family
Comparison
• 5 Test Sets: 1000 votes, ~100 movies, ~115 people– PRM Mean LL: -12,079, std 475.68– PRM-CH Mean LL: -10558, std 433.10
• Using standard t-test, PRM-CH model outperforms PRM model with over 99% confidence
PRM-CH Summary
• PRMs with class hierarchies are a natural extension of PRMs:– Specialization/Inheritance of CPDs – Allows new dependency structures
• Provide bridge from class-based to instance-based models
• Learning techniques proposed– Need efficient heuristics – Empirical validation on real-world domains
Roadmap
• Motivation and Background
• PRMs w/ Attribute Uncertainty
• PRMs w/ Structural Uncertainty
• PRMs w/ Class Hierarchies
Next Time 2/25• Focus Problems
– Please add your focus problem to the class wiki• Give a PRM for the problem• Give a PER for the problem• Give at least one of the logical-based methods (BLP, LPRM, LBN)
– For each representation, discuss some modeling issue, or some novelty you used – e.g. structural uncertainty, constraints, etc.
• Readings for next three weeks– 2/28 – Logic-based approaches– 3/4 – Advanced Logic-based approaches– 3/11 – Undirected Models
• Please sign up to lead the discussion for one of the papers 2/28 – 3/11
• For each paper, please post your comments for each paper on the wiki by midnight Wed before the class in which they are assigned to be discussed. This gives the discussion leader some time to synthesize the comments.
top related