Predicting Dynamic Embedding Trajectory in Temporal ...srijan/pubs/jodie-kdd2019-slides.pdfPredicting Dynamic Embedding Trajectory in Temporal Interaction Networks Srijan Kumar Stanford
Post on 13-Mar-2020
2 Views
Preview:
Transcript
1
Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks
Srijan KumarStanford University
Georgia Institute of Technology
Jure LeskovecStanford University
Xikun ZhangUIUC
Code and Data: https://snap.stanford.edu/jodie
2
Temporal Interaction Networks
Time
[KDD’19]
Flexible way to represent time-evolving relations
Users Items
Feature
interaction user item time features
Represented as a sequence of interactions,
sorted by time:
3
Temporal Interaction Networks[KDD’19]
E-commerce Social media
Finance
WebEducation
IoT
Application domains Accounts Posts
…...
4
Temporal Interaction Networks[KDD’19]
E-commerce Social media
Finance
Web
Students Courses
Education
IoT
Application domains
…...
5
Problem Setup
Given a temporal interaction network
where
generate an embedding trajectory of every user
and an embedding trajectory of every item
[KDD’19]
interaction user item time features
6
Goal: Generate Dynamic Trajectory
Output: Dynamic trajectory in embedding space
Input: Temporal interaction network
[KDD’19]
1
2
4
3
56
7
ChallengesChallenges in modeling:• C1: How to learn inter-dependent user and item
embeddings? • C2: How to generate embedding for every point
in time?
Challenges in scalability: • C3: How to scalably train models on temporal
networks?
[KDD’19]
8
Existing Methods
Deep recommender systems• Time-LSTM (IJCAI 2017)• Recurrent Recommender Networks (WSDM
2017)• Latent Cross (WSDM 2018)
Dynamic co-evolution• Deep Coevolve (DLRS, 2016)
Temporal network embedding• CTDNE (BigNet, 2018)
Our model: JODIE
[KDD’19]
C1Co-
influence
C2Embed
any time
C3Train in batches
9
Our Model: JODIEJODIE: Joint Dynamic Interaction Embedding• Mutually-recursive recurrent neural network framework
[KDD’19]
ProjectionOperator
ProjectComponent
User RNN Item RNNUpdate Component
10
JODIE: Update Component[KDD’19]
User RNN Item RNN
f =
Weight matrices Ware trainable
• All users share the User-RNN parameters. Similar for items.
11
JODIE: Project Component
How can we predict the next item? • Rank items using distance in the embedding space
[KDD’19]
Projected embedding
Projection operatorTime Δ
Projected embedding
f =
User RNN Item RNN
12
Summary: JODIE Formulation
Update embeddings:
[KDD’19]
Loss:
Predicted next item is close to the real item
embeddingSmoothness in evolving
embeddings
Project user embedding:
Predict next item:
13
Challenges in Dynamic TrajectoriesChallenges in learning:• C1: How to learn inter-dependent user and item
embeddings? Solution: Update component• C2: How to generate embedding for every point in
time? Solution: Project component
Challenges in scalability: • C3: How to scalably train models on temporal
networks?
[KDD’19]
14
Standard Training Processes: N/ATraining must maintain temporal order
[KDD’19]
(1)
(2)
(3)
(4)
.
...
.
.
User 1
User 2
User 3
Split by user (or item): not allowed
Sequential processing: not scalable
1
2
3
4
4
32
1 5
6
Batch 1 Temporal inconsistency
15
T-batch: Temporal data batching algorithm
• Main idea: create each batch as an independent edge set
• Create a sequence of batches– Interactions in each batch are processed in
parallel– Process the batches in sequence to maintain
temporal ordering
[KDD’19]
T-batch: Batching for Scalability
16
T-batch: Batching for Scalability
Batch 2Batch 1 Batch 3
[KDD’19]
1
2
3
4
56
2
1
4
3
5
6
Iteratively select the maximal
independent edge set.
17
Challenges in Dynamic TrajectoriesChallenges in learning:• C1: How to learn inter-dependent user and item
embeddings? Solution: Update component• C2: How to generate embedding for every point in
time? Solution: Project component
Challenges in scalability: • C3: How to scalably train models on temporal
networks? Solution: T-batch Algorithm
[KDD’19]
18
Experiments: Prediction Tasks
• Temporal Link Prediction:–Which item i ∈ 𝐼 will user u interact with at
time t?• Temporal Node Classification:– Does a user u become anomalous after an
interaction?• Settings:– Temporal Splits: 80%, 10%, 10%–Metrics: Mean reciprocal rank, Recall@10,
AUROC
[KDD’19]
Code and Data: https://snap.stanford.edu/jodie
19
Datasets[KDD’19]
Dataset Users Items Interactions Temporal Anomalies
Reddit 10,000 984 672,447 366Wikipedia 8,227 1,000 157,474 217LastFM 980 1,000 1,293,103 -MOOC 7,047 97 411,749 4,066
NEW!
NEW!
Code and Data: https://snap.stanford.edu/jodie
20
Experiment 1: Link Prediction
JODIE outperforms baselines by > 20%
Mean Reciprocal
Rank
0.0
1.0
Latent Cross
0.42
0.18
Time-LSTM
0.60
RRN
0.73
0.39
0.17
CTDNE Deep Coevolve
JODIE
0.2
0.4
0.6
0.8
[KDD’19]
21
Experiment 2: Node Classification
JODIE outperforms all baselines by >12%
AUROC
0.5
1.0
Latent Cross
0.630.58
Time-LSTM
0.65
RRN
0.73
0.65 0.64
CTDNE Deep Coevolve
JODIE
0.6
0.7
0.8
0.9
[KDD’19]
22
Experiment 3: T-batch Speed-up
T-batch leads to 8.5x speed-up in training
5.1 minutes
44 minutes
JODIE without T-batch
JODIE with T-batch
Running Time
0
50
10
20
30
40
8.5x speed-up
[KDD’19]
23
Predicting Dynamic Embedding Trajectory in Temporal Interaction NetworksSrijan Kumar, Xikun Zhang, Jure Leskovec
Code and Data: https://snap.stanford.edu/jodie
JODIE generates and projects embedding
trajectories
• JODIE: a mutually-recursive RNN framework• T-batch: 8.5x training speed-up• Efficient in temporal link prediction and node classification• Extendible to > 2 entity types
24
Open Positions @ Georgia Tech
• Hiring multiple Ph.D. students• Research areas:–Machine Learning for Networks– Safety, Integrity, and Anti-Abuse– Computational Social Science
• Collaborations
Contact: srijan@cs.stanford.edu
25
Predicting Dynamic Embedding Trajectory in Temporal Interaction NetworksSrijan Kumar, Xikun Zhang, Jure Leskovec
Code and Data: https://snap.stanford.edu/jodie
JODIE generates and projects embedding
trajectories
• JODIE: a mutually-recursive RNN framework• T-batch: 8.5x training speed-up• Efficient in temporal link prediction and node classification• Extendible to > 2 entity types
top related