1 Fully Automatic Cross-Associations Deepayan Chakrabarti (CMU) Spiros Papadimitriou (CMU) Dharmendra Modha (IBM) Christos Faloutsos (CMU and IBM)
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
Fully Automatic Cross-Associations
Deepayan Chakrabarti (CMU)Spiros Papadimitriou (CMU)Dharmendra Modha (IBM)Christos Faloutsos (CMU and IBM)
2
Problem Definition
Products
Cus
tom
ers
Cus
tom
er G
roup
s
Product Groups
Simultaneously group customers and products, or, documents and words, or, users and preferences …
3
Problem Definition
Desiderata:
1. Simultaneously discover row and column groups
2. Fully Automatic: No “magic numbers”
3. Scalable to large matrices
4
Closely Related Work
Information Theoretic Co-clustering [Dhillon+/2003] Number of row and column groups must be
specified
Desiderata:
Simultaneously discover row and column groups
Fully Automatic: No “magic numbers”
Scalable to large graphs
5
Other Related Work
K-means and variants: [Pelleg+/2000, Hamerly+/2003]
“Frequent itemsets”: [Agrawal+/1994]
Information Retrieval:[Deerwester+1990, Hoffman/1999]
Graph Partitioning:[Karypis+/1998]
Do not cluster rows and cols simultaneously
User must specify “support”
Choosing the number of “concepts”
Number of partitions
Measure of imbalance between clusters
6
What makes a cross-association “good”?
versus
Column groups Column groups
Row
gro
ups
Row
gro
ups
Good Clustering
1. Similar nodes are grouped together
2. As few groups as necessary
A few, homogeneous
blocks
Good Compression
Why is this better?
implies
7
Main Idea
Good Compression
Good Clusteringimplies
Column groups
Row
gro
ups
pi1 = ni
1 / (ni1 + ni
0)
(ni1+ni
0)* H(pi1) Cost of describing
ni1, ni
0 and groups
Code Cost Description Cost
Σi
Binary Matrix
+Σi
8
Examples
One row group, one column group
high low
m row group, n column group
highlow
Total Encoding Cost = (ni1+ni
0)* H(pi1) Cost of describing
ni1, ni
0 and groups
Code Cost Description Cost
Σi +Σi
9
What makes a cross-association “good”?
Why is this better?
low low
Total Encoding Cost = (ni1+ni
0)* H(pi1) Cost of describing
ni1, ni
0 and groups
Code Cost Description Cost
Σi +Σi
versus
Column groups Column groups
Row
gro
ups
Row
gro
ups
10
Algorithmsk =
5 row groups
k=1, l=2
k=2, l=2
k=2, l=3
k=3, l=3
k=3, l=4
k=4, l=4
k=4, l=5
l = 5 col groups
11
Algorithmsl = 5
k = 5
Start with initial matrix
Find good groups for fixed k and l
Choose better values for k and l
Final cross-association
Lower the encoding cost
12
Fixed k and ll = 5
k = 5
Start with initial matrix
Find good groups for fixed k and l
Choose better values for k and l
Final cross-association
Lower the encoding cost
13
Fixed k and l
Column groups
Row
gro
ups Shuffles:
for each row:
shuffle it to the row group which minimizes the code cost
14
Fixed k and l
Column groups
Row
gro
ups
Ditto for column shuffles
… and repeat …
15
Choosing k and ll = 5
k = 5
Start with initial matrix
Choose better values for k and l
Final cross-association
Lower the encoding cost
Find good groups for fixed k and l
16
Choosing k and ll = 5
k = 5
Split:1. Find the row group R with the maximum entropy per row
2. Choose the rows in R whose removal reduces the entropy per row in R
3. Send these rows to the new row group, and set k=k+1
17
Choosing k and ll = 5
k = 5
Split:
Similar for column groups too.
18
Algorithmsl = 5
k = 5
Start with initial matrix
Find good groups for fixed k and l
Choose better values for k and l
Final cross-association
Lower the encoding cost
Shuffles
Splits
19
Experiments
l = 5 col groups
k = 5 row
groups
“Customer-Product” graph with Zipfian sizes, no noise
20
Experiments
“Quasi block-diagonal” graph with Zipfian sizes, noise=10%
l = 8 col groups
k = 6 row
groups
21
Experiments
“White Noise” graph: we find the existing spurious patterns
l = 3 col groups
k = 2 row
groups
22
Experiments“CLASSIC”
• 3,893 documents
• 4,303 words
• 176,347 “dots”
Combination of 3 sources:
• MEDLINE (medical)
• CISI (info. retrieval)
• CRANFIELD (aerodynamics)
Doc
umen
ts
Words
23
Experiments
“CLASSIC” graph of documents & words: k=15, l=19
Doc
umen
ts
Words
24
Experiments
“CLASSIC” graph of documents & words: k=15, l=19
MEDLINE(medical)
insipidus, alveolar, aortic, death, prognosis, intravenous
blood, disease, clinical, cell, tissue, patient
25
Experiments
“CLASSIC” graph of documents & words: k=15, l=19
MEDLINE(medical)
CISI(Information Retrieval)
providing, studying, records, development, students, rules
abstract, notation, works, construct, bibliographies
26
Experiments
“CLASSIC” graph of documents & words: k=15, l=19
MEDLINE(medical)
CRANFIELD (aerodynamics)
shape, nasa, leading, assumed, thin
CISI(Information Retrieval)
27
Experiments
“CLASSIC” graph of documents & words: k=15, l=19
MEDLINE(medical)
CISI(IR)
CRANFIELD (aerodynamics)
paint, examination, fall, raise, leave, based
28
ExperimentsN
SF
Gra
nt P
ropo
sals
Words in abstract
“GRANTS”
• 13,297 documents
• 5,298 words
• 805,063 “dots”
29
Experiments
“GRANTS” graph of documents & words: k=41, l=28
NS
F G
rant
Pro
posa
ls
Words in abstract
30
Experiments
“GRANTS” graph of documents & words: k=41, l=28
The Cross-Associations refer to topics:
• Genetics
• Physics
• Mathematics
• …
31
Experiments
“Who-trusts-whom” graph from epinions.com: k=18, l=16
Ep
inio
ns.
com
use
r
Epinions.com user
32
Experiments
Number of “dots”
Tim
e (
secs
)
Splits
Shuffles
Linear on the number of “dots”: Scalable
33
Conclusions
Desiderata:
Simultaneously discover row and column groups
Fully Automatic: No “magic numbers”
Scalable to large matrices
34
Cross-Associations ≠ Co-clustering !Information-theoretic
co-clustering Cross-Associations
1. Lossy Compression.
2. Approximates the original matrix, while trying to minimize KL-divergence.
3. The number of row and column groups must be given by the user.
1. Lossless Compression.
2. Always provides complete information about the matrix, for any number of row and column groups.
3. Chosen automatically using the MDL principle.
35
Experiments
36
Experiments
“Clickstream” graph of users and websites: k=15, l=13
Use
rs
Webpages
37
Fixed k and ll = 5
k = 5
Start with initial matrix
Choose better values for k and l
Final cross-associations
Lower the encoding cost
Find good groups for fixed k and l
swaps swaps
38
Experimentsl = 5 col groups
k = 5 row
groups
“Caveman” graph with Zipfian cave sizes, no noise
39
Aim
Given any binary matrix a “good” cross-association will have low cost
But how can we find such a cross-association?
l = 5 col groups
k = 5 row
groups
40
Main Idea
sizei * H(pi) +Cost of describing cross-associations
Code Cost Description Cost
Σi Total Encoding Cost =
Good Compression
Better Clusteringimplies
Minimize the total cost
41
Main Idea
How well does a cross-association compress the matrix? Encode the matrix in a lossless fashion Compute the encoding cost Low encoding cost good compression good
clustering
Good Compression
Better Clusteringimplies
Related Documents
