Bayes Belief Network

Machine LearningBayesian Belief Network

Oleh :嗗 Aldy Rialdy Atmadja (23512031)嗗 Arif Syamsudin (23512099)嗗 Taufiq Iqbal Ramdhani (23512062)嗗 Mahar Faiqurahman (23512028)嗗 Hendri Karisma (23512060)嗗 Jupriyadi (23512029)

Review Bayes

嗗Metodologi Bayesian reasoning嗗Pendekatan probabilistik untuk menghasilkan inferensi.

嗗Quantity of interest -> Distribusi probabilitas.

嗗Pemilihan yang optimal -> Reasoning (Probabilitas dan observasi data).

嗗Pendekatan kuantitatif, menimbang bukti yang mendukung alternatif hipotesis.

Bayesian Learning

嗗Bayesian Learning merupakan suatu metode pembelajaran yang dikenal dalam machine learning.嗗Dua alasan bayesian learning dipelajari dalam machine learning yakni :

–Bayesian Learning menghitung secara eksplisit probabilitas untuk setiap hipotesis, seperti klasifikasi pada Naive Bayes.–Bayesian Learning memberikan perspektif dalam memahami algoritma pembelajaran lainnya

Teorema Bayes

Teorema Bayes menyediakan cara untuk menghitung probabilitas dari suatu hipotesis berdasarkan probabilitas sebelumnya, probabilitas mengamati berbagai data yang diberikan hipotesis, dan data yang diamati itu sendiri.

Penggunaan Teorema Bayess

B

G

S

SC

S

P(B)

P(G)

P(S|B)

SC

P(SC|B)

P(SC|G)

P(S|G)

P(SnB) => P(B).P(S|B)

P(ScnB) => P(B).P(Sc|B)

P(SnG) => P(G).P(S|G)

P(ScnG) => P(G).P(Sc|G)

嗗P(B) = Boys

嗗P(G) = Girls

嗗P(S) = Soccer

Penggunaan Teorema Bayess

B

G

S

SC

S

0.40

0.60

0.30

SC

0.70

0.60

0.40

P(SnB) = 0.12

P(ScnB) = 0.28

P(SnG) = 0.24

P(ScnG) = 0.36

P(B) = 0.40P(G) = 0.60P(S|B) = 0.30P(S|G) = 0.40

Possibility of Girls Playing Soccer ?P(G|S) = ???

Kemampuan Bayesian Method

Menangani data set yang tidak lengkap.

Pembelajaran mengenai Causal Networks

Memfasiitasi kombinasi dari domain knowledge dan data.

Efisien dan mempunyai prinsip untuk menghindari overfitting data.

Bayes Optimal Classifier

Klasifikasi ini diperoleh dengan menggabungkan prediksi dari semua hipotesis

Naive Bayes Classifier

Klasifikasi ini diperoleh dengan probabilitas conditional independence.

Naive Bayes Classifier

嗗Keuntungan–Mudah diimplementasikan.–Hasil yang baik bila diimplementasikan pada beberapa kondisi.

嗗Kekurangan–Asumsi : Conditional independence, loss acuracy.–Tidak dapat memodelkan dependensi atribut.

嗗Untuk menjawab kekurangan pada Naive Bayes ini digunakan Bayes Belief Network.

Intro Bayes Belief Network

Naive Bayes didasarkan pada asumsi conditional independence (berdiri sendiri).

Bayesian Network (tractable method) untuk menentukan ketergantungan antar variabel.

Objective & Motivation

嗗Objective: Explain the concept of Bayesian Network.嗗Reference: www.cse.ust.hk/bnbook

Predisposing factors symptoms test result diseases treatment outcome.

Class label for thousands of superpixels.

Outline

1.Probabilistic Modeling with Joint Distribution2.Conditional Independence 3.Bayesian Networks4.Manual Construction of Bayesian Networks5.Inference6.Some example

The Probabilistic Approach to Reasoning Under Certainty

嗗Domain Variable: X1, X2, X3, …, Xn嗗Knowledge about the problem domain is

represented by a Joint Probability P(X1, X2, X3, …, Xn)

The Probabilistic Approach to Reasoning Under Certainty

Example : Alarm (Pearl 1988)嗗hnCalls (J), MaryCalls (M)嗗Knowledge required by the probabilistic approach in order to solve this problem: P(B,E,A,J,M)嗗Problem: Estimate the probability of a burglary based who has or has not called.嗗Variables: Burglary (B), Earthquake (E), Alaram (A), JohnCalls (J), MaryCalls (M)嗗Knowledge required by the probabilistic approach in order to solve this problem: P(B,E,A,J,M)

Join Probability Distribution (JPD)

Inference with Joint Probability Distribution

± What is probability of Burglary given that Mary Called, P(B=y|M=y)?± Steps:1.Compute Marginal Probability

2.Compute answer (reasoning by conditioning):

Outline

1.Probabilistic Modeling with Joint Distribution2.Conditional Independence 3.Bayesian Networks4.Manual Construction of Bayesian Networks5.Inference6.Some example

Conditional Independence

Conditional Probability Tables (CPT)

Conditional Probability Tables (CPT)

Conditional Probability Tables (CPT)

Outline

1.Probabilistic Modeling with Joint Distribution2.Conditional Independence 3.Bayesian Networks4.Manual Construction of Bayesian Networks5.Inference6.Some example

Bayesian Network

嗗 Each node represent a random variable

嗗 Between nodes as influences

Recall in introduction嗗 Bayesian Networks are

networks of random variables.嗗 The topology of network

determines the relationship between attributes

Independence

Burglary and Earthquake are independentP(B,E) = P(B)P(E)P(B|E) = P(B) P(E|B) = P(E)

P(B|E) = P(E|B)P(B) = P(B)P(E)P(E|B) = P(B|E)P(E) = P(E)P(B)

Conditional Independent

MeryCalls isindependent ofBurglary dan EarthquakeGiven Alarm.P(M|B,E,A) = P(M|A)

Dependent Vs Independent

嗗JohnCalls dan MeryCalls are Dependent嗗JohnCalss is Independent of MeryCalss given Alarm

嗗Burglary and Earthquake are Independent嗗Burglary is dependent of Earthquake given Alarm

Causal Independence

嗗Burglary causes Alarm if motion sensor clear

嗗Earthquake causes Alarm iff wire loose

嗗Enabling factors are independent of each other

Bayesian network topology

Serial Connection

嗗C depend on B, and B depend on A

嗗If the value of B is known, then A should be independent from C (then A d-separated with C)

Divergen Connection

嗗B, C, D.., F depend on A

嗗if the value of A is known, B, C, D,..F should be independent each others (d-separated)

嗗otherwise B, C, D,.. dependent

Bayesian network topology

Convergen Connection

嗗A depend on B, C, D,,... F

嗗if value of A is unknown, then B, C, E, ... F should be independent each others (d-separated)

嗗Otherwise B,C,E,...F dependent each others

Outline

1.Probabilistic Modeling with Joint Distribution2.Conditional Independence 3.Bayesian Networks4.Manual Construction of Bayesian Networks5.Inference6.Some example

Outline

1.Probabilistic Modeling with Joint Distribution2.Conditional Independence 3.Bayesian Networks4.Manual Construction of Bayesian Networks5.Inference6.Some example

Bayesian Network Building

(A)

(B)

Expert

13 Nopember 2012

Bayesian Network Building

Komponen Bayesian Network

嗗Kualitatif → Berupa directed acyclic graph (DAG)

dimana atribut direpresentasikan oleh node sedangkan

edge menggambarkan kausalitas antar node

嗗Kuantitatif → Berupa Conditional Probabilitas Table

(CPT) yang memberikan informasi besarnya probabilitas

untuk setiap nilai atribut berdasarkan parent dari atribut

bersangkutan

13 Nopember 2012

Excercise Diet

Heart Disease

Heartburn

Chest PainBlood Pressure

HD = Yes

E = YesD = Healthy

0,25

E = YesD = Unhealthy

0,45

E = NoD = Healthy

0,55

E = NoD = Unhealthy

0,75 CP = Yes

HD = YesHb = Yes

0,8

HD = YesHb = No

0,5

D = NoHb = Yes

0,4

HD = YesHb = No

0,1

Hb = Yes

D = Healthy 0,8

D = Unhealthy

0,85

Hb = Yes

HD = Yes 0,85

HD = No 0,2

E = Yes

0,7

D = Healthy

0,25

13 Nopember 2012

Contoh Bayesian Network

Tahapan yang dilakukan:

嗗Konstruksi struktur atau tahap kualitatif, yaitu

mencari keterhubungan antara variabel-variabel yang

dimodelkan

嗗Estimasi parameter atau tahap kuantitatif, yaitu

menghitung nilai-nilai probabilitas

13 Nopember 2012

Bayesian Network Building

Bayesian Network Building

Ada dua pendekatan yang digunakan untuk mengkonstruksi

struktur Bayesian Network yaitu

1.Metode Search and Scoring (Scored Based)

Menggunakan metode pencarian untuk mendapatkan struktur yang

cocok dengan data, di mana proses konstruksi dilakukan secara iteratif

2. Metode Dependency Analysis (Constraint Based)

Mengidentifikasi/menganalisa hubungan bebas bersyarat (conditional

independence test) atau disebut juga CI-test antar atribut, dimana CI

menjadi “constraint” dalam membangun struktur Bayesian Network.

13 Nopember 2012

Algoritma BN building

嗗Search & Scoring Based (Chow-Liu Tree

Construction, K2, Kutato, Benedict, CB, dll)

嗗Dependency Analysis Based ( TPDA, Boundary

DAG, SRA, SGS, PC, dll)

13 Nopember 2012

Bayesian Network Building

MMutual Information

Mutual InformationMI dari dua variabel acak merupakan nilai ukur yang

menyatakan keterikatan/ketergantungan (mutual

dependence) antara kedua variabel tersebut.

13 Nopember 2012

Bayesian Network Building

(1)

(2)

(3)

(4)

13 Nopember 2012

Bayesian Network Building

Persamaan yang digunakanLog2

(5)

13 Nopember 2012

Bayesian Network Building

Tabel data rekam medik

13 Nopember 2012

Bayesian Network BuildingCase study

13 Nopember 2012

Teknik Pembobotan

Bayesian Network Building

13 Nopember 2012

Teknik Pembobotan (cont’d)

Bayesian Network Building

Tabel hasil pembobotan data rekam medik

13 Nopember 2012

Bayesian Network Building

Tabel hasil perhitungan Mutual Information

(3) (4) (2) (1)

13 Nopember 2012

Bayesian Network Building

Tabel hasil perhitungan prob. Dependency 2 node

(5) (5)

13 Nopember 2012

Bayesian Network Building

Contoh struktur network yang terbentuk

13 Nopember 2012

Bayesian Network Building

Contoh Tabel Conditional probability yang terbentuk

13 Nopember 2012

Bayesian Network Building

Gradient ascent training

嗗Mirip seperti neural networks–Asumsi bahwa setiap entry dalam CPT adalah sebuah wight–Bentuk gradient dalam likelihooda, P(D|h), with respect to

the weight.–Update weights in the direction of the gradient

Gradient ascent training

Gradient ascent training

嗗Let wijk denote one entry in the conditional probability table for variable Yi in the network

wijk = P(Yi = yij |Parents(Yi ) = the list uik of values) e.g., if Yi = Campfire, then uik might be (Storm = T, BusTourGroup = F)

嗗Perform gradient ascent by repeatedly

1.update all wijk using training data D

1.then, renormalize the wijk to assure

Outline

1.Probabilistic Modeling with Joint Distribution2.Conditional Independence 3.Bayesian Networks4.Manual Construction of Bayesian Networks5.Inference6.Some example

Inference

嗗Suatu metode yang ada dalam bayesian network yang digunakan untuk mengambil suatu keputusan

嗗Inferensi berangkat dari suatu target variabel jika diketahui variabel yang lain (observed variable)

嗗P(A | X) - dimana A adalah target variabel (question), dan X adalah observed variable (evidence)

Inference (cont'd)

嗗Suatu relasi antar atribut (question and evidence) dapat berupa dependent atau conditionaly independent

Inference

嗗Probabilistic inference

嗗Exact inference

嗗Approximate inference

Inference dalam Bayesian Network

嗗Probabilistic Inference

–Diagnostic inference

–Causal inference

–Inter-causal inference

–Mixed inference嗗Exact inference

–Inference by enumeration

–Variable elemination algorithm嗗Approximate inference - digunakan apabila terdapat

unobserved variable

Probabilistic Inference

嗗Suatu proses untuk mencari / menghitung nilai dari distribusi probabilitas posterior jika diketahui beberapa evidence yang ada

嗗Evidence yang diketahui dapat berupa dependent atribute, maupun conditional dependent attribute

Probabilistic Inference

嗗Diagnostic Inference (from effect to cause)–P(B|J) = P(J, B) / P(J) –Mencari suatu

kesimpulan dimana evidence yang diberikan berupa effect (Q=burglary, E=john calls)

Probabilistic Inference

嗗Causal Inference (from cause to effect)–P(J|B) = P(J,B) / P(B)

–Mencari suatu kesimpulan dengan evidence berupa cause (Q = john calls, E=burglary)

Probabilistic Inference

嗗Inter-causal Inference (between causes of the common effect)–Contoh: P(B|A) =

P(B,A)/P(A)–Karena A dependent

terhadap B dan E, maka P(B,A) = P(B,A,E) + P(B,A,E')

Probabilistic Inference

嗗Mixed Inference (combining causes and effects)–merupakan kombinasi

antara inferensi model diagnostic dan inferensi model causal

–contoh: P(A|E,M)

嗗Inference by Enumeration–Untuk menghitung nilai dari probabilitas dari variable Q

dengan evidence E (E1, E2,...Ek) dapat menggunakan aturan conditional independentPersamaan tersebut dapat dihitung dengan dengan menjumlahkan

– persamaan dari full joint distribution

Exact Inference

Exact Inference

嗗Inference by Enumeration (cont'd)

Exact Inference

嗗Variable Elemination Algorithm

Exact Inference

The Algorithm

Approximate inference

嗗Digunakan apabila terdapat atribut yang unobserved

嗗Beberapa metode digunakan–Direct sampling

–Markov chain monte carlo sampling

TERIMA KASIH