NaΓ―ve Bayes π πΆ - Kangwoncs.kangwon.ac.kr/.../2015_MachineLearning/07_naive_bayes.pdfΒ Β· 2016. 6. 17.Β Β· NaΓ―ve Bayes β’Bayes ruleμμ μ©νλ©΄λͺ¨λ λ°μ΄ν°μλνμ¬κ³ λ €ν΄μΌν¨
Post on 05-Oct-2020
3 Views
Preview:
Transcript
π ππππ πΆ
Machine Learning
π ππππ πΆ
2015.08.01.
NaΓ―ve Bayes
π ππππ πΆ 2
Probability Basics
β’ Prior, conditional and joint probability for random variables
β’ Prior probability: π(π)
β’ Conditional probability: π π1 π2 , π(π2|π1)
β’ Joint probability: πΏ = π1, π2 , π πΏ = π(π1, π2)
β’ Relationship: π π1, π2 = π π2 π1 π π1 = π π1 π2 π(π2)
β’ Independence: π π2|π1 = π π2 , π π1|π2 = π π1 ,
π π1, π2 = π π1 π(π2)
β’ Bayesian Rule
π ππππ πΆ 3
Probabilistic Classification
β’ Establishing a probabilistic model for classification
β’ Discriminative model
),, , )( 1 n1L X(Xc,,cC|CP XX
),,,( 21 nxxx x
Discriminative
Probabilistic Classifier
1x 2x nx
)|( 1 xcP )|( 2 xcP )|( xLcP
π ππππ πΆ 4
Probabilistic Classification
β’ Establishing a probabilistic model for classification (cont.)
β’ Generative model
β’ Dataλ€μ ν¨ν΄μΌλ‘ λΆλ₯
β’ Labelμ΄ μ£Όμ΄μ‘μ λ dataλ€μ νμΈ dataμ label κ΄κ³ νμ
π ππππ πΆ 5
Bayes`s Theorem
β’ Bayes' theorem (alternatively Bayes' law or Bayes' rule) describes the probability of an event, based on conditions that might be related to the event.
β’ λ νλ₯ λ³μμ μ¬μ νλ₯ κ³Ό μ¬ν νλ₯ μ¬μ΄μ κ΄κ³λ₯Ό λνλ
β’ μλ‘μ΄ κ·Όκ±°κ° μ μλ λ μ¬ν νλ₯ μ΄ μ΄λ»κ² κ°±μ λ μ§ κ΅¬ν¨
β’ π π΄ = πππππ ππππππππππ‘π¦ ππ βπ¦πππ‘βππ ππ π¨
β’ π π΅ = πππππ ππππππππππ‘π¦ ππ π‘πππππππ πππ‘π π©
β’ π π΄ π΅ = ππππππππππ‘π¦ ππ π¨ πππ£ππ π©
β’ π π΅ π΄ = ππππππππππ‘π¦ ππ π© πππ£ππ π¨
π· π¨ π© =π·(π©|π¨)π· π¨
π· π©
π ππππ πΆ 6
Bayes`s Theorem
β’ MAP classification ruleβ’ MAP: Maximum A Posterior
β’ Assign π₯ to πβ if
π πΆ = πβ π = π₯ > π πΆ = π π = π₯ π β πβ, π = π1, β¦ , ππΏ
β’ Generative classification with the MAP ruleβ’ Apply Bayesian rule
π πΆ = ππ π = π₯ =π π = π₯ πΆ = ππ π πΆ = ππ
π π = π₯
β π π = π₯ πΆ = ππ π πΆ = ππ β ππ
π ππππ πΆ 7
NaΓ―ve Bayes
β’ Bayes ruleμ μ μ©νλ©΄ λͺ¨λ λ°μ΄ν°μ λνμ¬ κ³ λ €ν΄μΌ ν¨ learning the joint probability π(π1, β¦ , ππ|πΆ) : Difficulty
β’ 10κ°μ Binary feature 210κ°μ data
β’ Thus, assumption that all input features are conditionally independent NaΓ―ve Bayes rule
β’ κ° μμ§μ λνμ¬ μ‘°κ±΄λΆνλ₯ μ΄ λ 립μ μ΄λΌ κ°μ
β’ μ‘°κ±΄λΆ νλ₯ μ λν κ²½μ°μ μ: 2π 2π
π ππππ πΆ 8
NaΓ―ve Bayes
β’ NaΓ―ve Bayes
β’ MAP classification rule: π₯ = (π₯1, π₯2, β¦ , π₯π)
π π1, π2, β¦ , ππ πΆ = π π1 π2, β¦ , ππ, πΆ π(π2, β¦ , ππ|πΆ)
= π π1 πΆ π(π2, β¦ , ππ|πΆ)
= π π1 πΆ π π2 πΆ β¦π(ππ|πΆ)
ProbabilityChain rule!
π π₯1 πΆβ β¦π π₯π π
β π πβ > [π π₯1 π β¦π π₯π π)]π(π),
π β π^ β , π = π_1, β¦ , π_πΏ
=
π
π(ππ|πΆ)
π ππππ πΆ 9
Example
β’ Example: Play Tennis
π ππππ πΆ 10
Example
β’ Learning Phase
Outlook Play=Yes Play=No
Sunny 2/9 3/5Overcast 4/9 0/5
Rain 3/9 2/5
Temperature Play=Yes Play=No
Hot 2/9 2/5Mild 4/9 2/5Cool 3/9 1/5
Humidity Play=Yes Play=No
High 3/9 4/5Normal 6/9 1/5
Wind Play=Yes Play=No
Strong 3/9 3/5Weak 6/9 2/5
P(Play=Yes) = 9/14 P(Play=No) = 5/14
π ππππ πΆ 11
Example
β’ Test Phaseβ’ Given a new instance, predict its label
xβ=(Outlook=Sunny, Temperature=Cool, Humidity=High, Wind=Strong)
β’ Look up tables achieved in the learning phrase
β’ Decision making with the MAP rule
P(Outlook=Sunny|Play=No) = 3/5
P(Temperature=Cool|Play==No) = 1/5
P(Huminity=High|Play=No) = 4/5
P(Wind=Strong|Play=No) = 3/5
P(Play=No) = 5/14
P(Outlook=Sunny|Play=Yes) = 2/9
P(Temperature=Cool|Play=Yes) = 3/9
P(Huminity=High|Play=Yes) = 3/9
P(Wind=Strong|Play=Yes) = 3/9
P(Play=Yes) = 9/14
P(Yes|xβ) β [P(Sunny|Yes)P(Cool|Yes)P(High|Yes)P(Strong|Yes)]P(Play=Yes) = 0.0053
P(No|xβ) β [P(Sunny|No) P(Cool|No)P(High|No)P(Strong|No)]P(Play=No) = 0.0206
Given the fact P(Yes|xβ) < P(No|xβ), we label xβ to be βNoβ.
π ππππ πΆ 12
References
β’ NaΓ―ve Bayes Classifier - Ke Chen
β’ Advanced Algorithm(NaΓ―ve Bayes Classifier) - Leeck
β’ Machine Learning and Its Applications β Harksoo Kim
β’ Wikipedia
β’ http://www.leesanghyun.co.kr/Naive_Bayesian_Classifier
β’ http://darkpgmr.tistory.com/62
π ππππ πΆ 13
QA
κ°μ¬ν©λλ€.
λ°μ²μ, λ°μ°¬λ―Ό, μ΅μ¬ν
π ππππ πΆ , κ°μλνκ΅
Email: parkce@kangwon.ac.kr
top related