Bayesian Artificial Intelligence 1/40 History Bayesian Networks Extensions Bayesian Net Tools Causal Discovery Applications Conclusion References Bayesian AI Introduction for IEEE Computational Intelligence Society IEEE Computer Society Kevin Korb Monash University& Bayesian Intelligence Pty Ltd [email protected]www.bayesian-intelligence.com
40
Embed
Bayesian AI - Introduction for IEEE Computational ... · Bayesian Networks Extensions Bayesian Net Tools Causal Discovery Applications Conclusion ... Probability and Causality For
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Decision nodes (�) :Represent choices between actions.
Utility nodes (♦) :Represent the utility function. Parents are anyvariables that directly affect utility, whether chanceor decision nodes. Has an associated tablerepresenting multi-attribute utility function.
Bayesian ArtificialIntelligence
18/40
History
BayesianNetworks
Extensions
Bayesian Net Tools
Causal Discovery
Applications
Conclusion
References
Decision Networks
There are now two kinds of updating possible:
(1) Add observational evidence and update to getposterior probabilities.
(2) Select an alternative decision value. Update to getthe decision’s consequences.
(Of course, these can be combined.)• We can now find the expected utilities.
If we iterate through all alternative decisions, we canmaximize expected utility.
Bayesian ArtificialIntelligence
19/40
History
BayesianNetworks
Extensions
Bayesian Net Tools
Causal Discovery
Applications
Conclusion
References
ExampleNeapolitan’s Car Buyer
Bayesian ArtificialIntelligence
20/40
History
BayesianNetworks
Extensions
Bayesian Net Tools
Causal Discovery
Applications
Conclusion
References
Dynamic Belief Networks
• One node for each variable for each time step.• Intra-slice arcs FluT −→ FeverT
• Inter-slice (temporal) arcs1 FluT −→ FluT+1
2 AspirinT −→ FeverT+1
Bayesian ArtificialIntelligence
21/40
History
BayesianNetworks
Extensions
Bayesian Net Tools
Causal Discovery
Applications
Conclusion
References
Fever DBN
React t Reactt+1
Flu Flut+1
t+1Th
At
t
tTh
Fevert t+1Fever
At+1
Bayesian ArtificialIntelligence
22/40
History
BayesianNetworks
Extensions
Bayesian Net Tools
Causal Discovery
Applications
Conclusion
References
DBN reasoning
• Can calculate distributions for St+1 and further:probabilistic projection.
• Reasoning can be done using standard BN updatingalgorithms
• This type of DBN gets very large, very quickly.• Usually only keep two time slices of the network.
Bayesian ArtificialIntelligence
23/40
History
BayesianNetworks
Extensions
Bayesian Net Tools
Causal Discovery
Applications
Conclusion
References
Dynamic Decision NetworkPlanning Under Uncertainty
Decision Networks can be expanded dynamically. . .
U
Th
Flu
Th
React t t+1React
t+1t
t t+1Flu
t
t t+1
t+1Fever Fever
A A
Bayesian ArtificialIntelligence
24/40
History
BayesianNetworks
Extensions
Bayesian Net Tools
Causal Discovery
Applications
Conclusion
References
BN Tools
GeNie: Bayesian and decision networks; programmableinterface; Windows GUI; hierarchical Bayesiannetworks. Free.genie.sis.pitt.edu
Hugin: Bayesian and decision networks; programmable;GUI; PC learning algorithm; handles largenetworks. Expensive.www.hugin.com
Netica: Bayesian and decision networks; programmable;easy-to-use Windows GUI. Medium cost; free forsmall networks.www.norsys.com
Bayesian ArtificialIntelligence
25/40
History
BayesianNetworks
Extensions
Bayesian Net Tools
Causal Discovery
Applications
Conclusion
References
Causal Discovery
• Parameterization• Linear models: see path modeling
CaMML (Korb & Nicholson, 2004; ch 8)A Bayesian information-theoretic scoringfunction with MCMC (sampling search);returns dags and patterns. Performancesimilar to BDe/BGe. Supports priors andhybrid local structure.
Bayesian ArtificialIntelligence
32/40
History
BayesianNetworks
Extensions
Bayesian Net Tools
Causal Discovery
Applications
Conclusion
References
Recent Extensions to CaMML
Two significant enhancements have been added in thelast few years.Expert priors (O’Donnell et al., 2006b)• Being Bayesian, it is relatively easy to incorporate
non-default priors into CaMML. We’ve done this invarious ways, specifying strengths for:
• A prior dag, computing a prior distribution via editdistance
• Arc densities• Topological orders, total or partial
Hybrid model learning (O’Donnell et al., 2006a)• Allowing varying representations of local structure
(CPTs, d-trees, logit model) throughout the network
Bayesian ArtificialIntelligence
33/40
History
BayesianNetworks
Extensions
Bayesian Net Tools
Causal Discovery
Applications
Conclusion
References
Selected Monash Applications
• Bureau of Meterology: Fog forecasting: Nicholson,Korb, Boneh PhD project, 2004-2009
• Intelligent tutoring for decimal understanding:Nicholson, Boneh, University of Melbourne(1999-2003)
• NAG (Nice Argument Generator): Zukerman, Korb,1997-2000
• Tropical seagrass in great barrier reef: Nicholson,Thomas (Monash Centre for Water Studies),2004-2006
Bayesian ArtificialIntelligence
35/40
History
BayesianNetworks
Extensions
Bayesian Net Tools
Causal Discovery
Applications
Conclusion
References
Selected Monash Applications(cont.)
• Predicting cardiovascular risk from epidemiologicaldata: Korb, Nicholson, Twardy, John McNeil(Department of Epidemiology and PreventiveMedicine, Monash University), 2004-2006
• Change impact analysis in software architecturedesign: Nicholson, Tang, Jin, Han (Swinburne)
• Victorian DSE (Bayesian Intelligence)• BN Tool for modeling and reporting on threats to
• Australian Conf on Artificial Lifewww.infotech.monash.edu.au/about/news/conferences/acal09/
Bayesian ArtificialIntelligence
39/40
History
BayesianNetworks
Extensions
Bayesian Net Tools
Causal Discovery
Applications
Conclusion
References
References I
G.F. Cooper and E. Herskovits (1991) “A Bayesian Method forConstructing Bayesian Belief Networks from Databases,” inD’Ambrosio, Smets and Bonissone (eds.) UAI 1991, 86-94.
D. Heckerman and D. Geiger (1995) “Learning Bayesian networks: Aunification for discrete and Gaussian domains,” in Besnard andHanks (eds.) UAI 1995, 274-284.
K.B. Korb and A.E. Nicholson (2004) Bayesian Artificial Intelligence.CRC/Chapman Hall.
Meek, C. (1996). Graphical models: Selectiong causal and statisticalmodels. PhD disseration, Philosophy, Carnegie MellonUniversity.
R. Neapolitan (1990) Probabilistic Reasoning in Expert Systems.Wiley.
J. Pearl (1988) Probabilistic Reasoning in Intelligent Systems,Morgan Kaufmann.
F.P. Ramsey (1931). The Foundations of Mathematics, edited by R.B.Braithwaite. Routledge.
Bayesian ArtificialIntelligence
40/40
History
BayesianNetworks
Extensions
Bayesian Net Tools
Causal Discovery
Applications
Conclusion
References
References II
D. Spiegelhalter & S. Lauritzen (1990) “Sequential Updating ofConditional Probabilities on Directed Graphical Structures,”Networks, 20, 579-605.
P. Spirtes, C. Glymour and R. Scheines (1993) Causation, Predictionand Search: Lecture Notes in Statistics 81. Springer Verlag.
J. H. Wigmore (1913). The problem of proof. Illinois Law Journal 8,77-103.
S. Wright (1921). Correlation and causation. Journal of AgriculturalResearch, 20, 557-585.