Top Banner
Bayesian Nonparametric Poisson Factorization for Recommendation Systems Prem Gopalan, Masashi Sugiyama, Rajesh Ranganath, David M. Blei AISTATS 2014 Presented by Yunchen Pu October 31st, 2014 1 / 17
17

Bayesian Nonparametric Poisson Factorization for ...lcarin/Yunchen10.31.2014.pdf · 31/10/2014  · Bayesian Nonparametric Poisson Factorization for Recommendation Systems...

Jul 30, 2020

Download

Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Bayesian Nonparametric Poisson Factorization for ...lcarin/Yunchen10.31.2014.pdf · 31/10/2014  · Bayesian Nonparametric Poisson Factorization for Recommendation Systems PremGopalan,MasashiSugiyama,RajeshRanganath,DavidM.Blei

Bayesian Nonparametric Poisson Factorization forRecommendation Systems

Prem Gopalan, Masashi Sugiyama, Rajesh Ranganath, David M. BleiAISTATS 2014

Presented by Yunchen Pu

October 31st, 2014

1 / 17

Page 2: Bayesian Nonparametric Poisson Factorization for ...lcarin/Yunchen10.31.2014.pdf · 31/10/2014  · Bayesian Nonparametric Poisson Factorization for Recommendation Systems PremGopalan,MasashiSugiyama,RajeshRanganath,DavidM.Blei

Outline

Recommendation Systems

Bayesian Nonparametric Poisson Factorization

Variational Inference algorithm

Experiment results

2 / 17

Page 3: Bayesian Nonparametric Poisson Factorization for ...lcarin/Yunchen10.31.2014.pdf · 31/10/2014  · Bayesian Nonparametric Poisson Factorization for Recommendation Systems PremGopalan,MasashiSugiyama,RajeshRanganath,DavidM.Blei

Recommendation Systems

Goal: Predict ’ratings’ or ’preferences’ of items for users

A general method: Collaborative filtering

3 / 17

Page 4: Bayesian Nonparametric Poisson Factorization for ...lcarin/Yunchen10.31.2014.pdf · 31/10/2014  · Bayesian Nonparametric Poisson Factorization for Recommendation Systems PremGopalan,MasashiSugiyama,RajeshRanganath,DavidM.Blei

Possion Factorization

Each user is represented with a K -dimensional latent vector ofpositive weights: θT

u = {θu1, ..., θuK}T

4 / 17

Page 5: Bayesian Nonparametric Poisson Factorization for ...lcarin/Yunchen10.31.2014.pdf · 31/10/2014  · Bayesian Nonparametric Poisson Factorization for Recommendation Systems PremGopalan,MasashiSugiyama,RajeshRanganath,DavidM.Blei

Possion Factorization

Each item is also represented with a K -dimensional latent vector ofpositive weights: βT

i = {βi1, ..., βiK}T

K Components

5 / 17

Page 6: Bayesian Nonparametric Poisson Factorization for ...lcarin/Yunchen10.31.2014.pdf · 31/10/2014  · Bayesian Nonparametric Poisson Factorization for Recommendation Systems PremGopalan,MasashiSugiyama,RajeshRanganath,DavidM.Blei

Possion Factorization

Ratings come from a distribution involving the inner product: θTu βi

6 / 17

Page 7: Bayesian Nonparametric Poisson Factorization for ...lcarin/Yunchen10.31.2014.pdf · 31/10/2014  · Bayesian Nonparametric Poisson Factorization for Recommendation Systems PremGopalan,MasashiSugiyama,RajeshRanganath,DavidM.Blei

Possion Factorization

The prior for the weights is Gamma:

θuk ∼ Gamma(a, b) (1)βik ∼ Gamma(c, d) (2)

Each observation is modeled by a Poisson distribtuion

yui ∼ Poisson(θTu βi) (3)

7 / 17

Page 8: Bayesian Nonparametric Poisson Factorization for ...lcarin/Yunchen10.31.2014.pdf · 31/10/2014  · Bayesian Nonparametric Poisson Factorization for Recommendation Systems PremGopalan,MasashiSugiyama,RajeshRanganath,DavidM.Blei

Bayesian Nonparametric Possion Factorization

How to determine the dimensionality K of the latent vectors?

Three properties of Bayesian nonparametric:Allow K →∞E(θT

u βi) <∞The item weights βi should be shared among all users.

Stick-breaking construction for the user weights

8 / 17

Page 9: Bayesian Nonparametric Poisson Factorization for ...lcarin/Yunchen10.31.2014.pdf · 31/10/2014  · Bayesian Nonparametric Poisson Factorization for Recommendation Systems PremGopalan,MasashiSugiyama,RajeshRanganath,DavidM.Blei

Bayesian Nonparametric Possion Factorization

The generative process for the Bayesian nonparametric Poissonfactorization model with N users and M items can be formed as:1. For each users u = 1, ...,N :

(a) Draw su ∼ Gamma(α, c)

(b) Draw vuk ∼ Beta(1, α), k = 1, ...,∞

(c) Set θuk = su︸︷︷︸User scaling factor

· vuk

k−1∏i=1

(1− vui)︸ ︷︷ ︸Stick proportions

, k = 1, ...,∞

2. Draw βik ∼ Gamma(a, b), k = 1, ...,∞, i = 1, ...,M.

3. Draw yui ∼ Poisson(∑∞

k=1 θukβik), u = 1, ...,N, i = 1, ...,M.Guarantee a sparse observation matrix:

p(yui = 0) ≥ exp(−aαbc ) (4)

9 / 17

Page 10: Bayesian Nonparametric Poisson Factorization for ...lcarin/Yunchen10.31.2014.pdf · 31/10/2014  · Bayesian Nonparametric Poisson Factorization for Recommendation Systems PremGopalan,MasashiSugiyama,RajeshRanganath,DavidM.Blei

Variational Inference

Variational algorithm to minimizing KL divergence between jointlyposterior and mean-field family:

q(z , β, s, v) =N∏

u=1q(su)

∞∏k=1

N∏u=1

q(vuk)∞∏

k=1

M∏i=1

q(βik)n∏

u=1

M∏i=1

q(zui)

(5)

Auxiliary variable to obtain a conditionally conjugate model:

yui =∞∑

k=1zui ,k , zui ,k ∼ Poisson(θukβik) (6)

To handle the infinite variational factors, the variational families aretruncated on T

q(vuk) = p(vuk), q(βik) = p(βik), for k ≥ T + 1

10 / 17

Page 11: Bayesian Nonparametric Poisson Factorization for ...lcarin/Yunchen10.31.2014.pdf · 31/10/2014  · Bayesian Nonparametric Poisson Factorization for Recommendation Systems PremGopalan,MasashiSugiyama,RajeshRanganath,DavidM.Blei

Variational Inference

Resort a degenerate delta distribution

q(vuk) = δτuk (vuk) (7)q(vuk) ln p(vuk , zuik |su, βik , α) = I(vuk = τuk) ln p(τuk , zui |su, βik , α)

(8)

Variational family:

q(su) = Gamma(su|γu,0, γu,1) (9)

q(vuk) =

{δτuk (vuk), for k ≤ Tp(vuk), for k ≥ T + 1

(10)

q(βik) =

{Gamma(βik |λik,0, λik, 1), for k ≤ Tp(βik), for k ≥ T + 1

(11)

q(zui) = Multinomial(zui |yui , φui) (12)

11 / 17

Page 12: Bayesian Nonparametric Poisson Factorization for ...lcarin/Yunchen10.31.2014.pdf · 31/10/2014  · Bayesian Nonparametric Poisson Factorization for Recommendation Systems PremGopalan,MasashiSugiyama,RajeshRanganath,DavidM.Blei

Experiment Results

Three databases:Movielens1M: 1 million ratings (0 to 5 stars), 6,040 users, 3,980movies.Movielens10M: 10 million ratings (0 to 10 stars), 71,567 users, 10,681movies.Netflix: 100 million ratings (0 to 5 stars), 480,000 users, 17,770 movies.

Metrics:Predictive log-likelihood (on a test set).Mean precision (retrieve the top 100 items): Fraction of recommendeditems that are relevant to the user.Mean recall (retrive the top 100 items): Fraction of relevant items thatare recommended.

12 / 17

Page 13: Bayesian Nonparametric Poisson Factorization for ...lcarin/Yunchen10.31.2014.pdf · 31/10/2014  · Bayesian Nonparametric Poisson Factorization for Recommendation Systems PremGopalan,MasashiSugiyama,RajeshRanganath,DavidM.Blei

Experiment Results

13 / 17

Page 14: Bayesian Nonparametric Poisson Factorization for ...lcarin/Yunchen10.31.2014.pdf · 31/10/2014  · Bayesian Nonparametric Poisson Factorization for Recommendation Systems PremGopalan,MasashiSugiyama,RajeshRanganath,DavidM.Blei

Experiment Results

14 / 17

Page 15: Bayesian Nonparametric Poisson Factorization for ...lcarin/Yunchen10.31.2014.pdf · 31/10/2014  · Bayesian Nonparametric Poisson Factorization for Recommendation Systems PremGopalan,MasashiSugiyama,RajeshRanganath,DavidM.Blei

Supplementary MaterialThe objective function can be formed as:

Eq

[ ∞∑k=1

N∑u=1

ln p(vuk |α) +∞∑

k=1

N∑u=1

M∑i=1

ln p(zuik |θuk , βik)

]

= Eq

[ ∞∑k=1

N∑u=1

(α− 1) ln(1− τuk) +∞∑

k=1

N∑u=1

M∑i=1

(zuik ln τuk

+k−1∑j=1

zuik ln(1− τuj)− βiksuτuk

k−1∏j=1

(1− τuj))

+ const

(13)

For each τuk , the objective function can be formed as:

Eq

(α− 1) ln(1− τuk) +M∑

i=1zuik ln τuk +

M∑i=1

∞∑j=k+1

zuij ln(1− τuk)

−su

M∑i=1

∞∑l=k+1

τul(l−1∏

j=1,j 6=k(1− τuj))βil

(14)

15 / 17

Page 16: Bayesian Nonparametric Poisson Factorization for ...lcarin/Yunchen10.31.2014.pdf · 31/10/2014  · Bayesian Nonparametric Poisson Factorization for Recommendation Systems PremGopalan,MasashiSugiyama,RajeshRanganath,DavidM.Blei

Supplementary Material

∂L∂τuk

=α− 11− τuk

− E [∑M

i=1 zuik ]

τuk+

E [∑M

i=1∑∞

j=k+1 zuij]1− τuk

− Auk

=α− 11− τuk

−∑M

i=1 yuiφui ,kτuk

+

∑Mi=1 yui(1−

∑kj=1 φui ,j)

1− τuk− Auk

= 0

(15)

⇔ Aukτ2uk +

α− 1+M∑

i=1yuiφui ,k +

M∑i=1

yui(1−k∑

j=1φui ,j)

τuk

−M∑

i=1yuiφui ,k = 0

(16)

16 / 17

Page 17: Bayesian Nonparametric Poisson Factorization for ...lcarin/Yunchen10.31.2014.pdf · 31/10/2014  · Bayesian Nonparametric Poisson Factorization for Recommendation Systems PremGopalan,MasashiSugiyama,RajeshRanganath,DavidM.Blei

Supplementary Material

Auk = E [su]∞∑

l=k+1τul

l−1∏j=1,j 6=k

(1− τuj)

× ( M∑i=1

E [βil ]

)

− E [su]

k−1∏j=1

(1− τuj)

( M∑i=1

E [βik ]

)+ E [su]M

ab

∞∏j=1,j 6=k

(1− τuj)

(17)

17 / 17