Shebuti Rayana* Leman Akoglu May 2, 2015
Jan 28, 2018
Rayana & Akoglu
Shebuti Rayana* Leman Akoglu
May 2, 2015
Rayana & Akoglu 2Less is More: Building Selective Anomaly Ensembles
Network intrusion
At time point t
Time tick 7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 5 10 15 20
Sco
re
Time tick
Event Detection
Rayana & Akoglu 3Less is More: Building Selective Anomaly Ensembles
Emerging Topic in Social Media
Nepal Earth Quake 2015tweets, retweets with• #Nepal• #NepalEarthQuake• #NepalEarthQuakeRelief• …
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 2 4 6 8 10 12 14 16 18 20
Sco
re
Time tick
Event Detection
25th April 2015
Rayana & Akoglu 4
Given a sequence of graphs {G1, G2, … , Gt, …, GT}
Find time points t’ at which Gt’ changes significantly from Gt’-1
Less is More: Building Selective Anomaly Ensembles
time
similarity/distance scores
Rayana & Akoglu 5Less is More: Building Selective Anomaly Ensembles
Numerous algorithms for event detection
no “winner” algorithm across datasets Idea: ensemble approach
Combine strength of accurate detectors
Alleviate weakness of inaccurate detectors
Improved accuracy, reduced noise
More robust performance
Better than individual base detectors
T. G. Dietterich. Ensemble methods in machine learning. Springer, 2000
J. Ghosh and A. Acharya. Cluster ensembles: Theory and applications. 2013.
Rayana & Akoglu 6
Idea: ensemble approach
Challenge: building anomaly ensembles –a fully unsupervised task
No labels to guide for detector accuracy
No objective function inherent to task
Combining all the results may deteriorate the overall ensemble accuracy [Rayana&Akoglu’14]
▪ some detectors may be inaccurate
Less is More: Building Selective Anomaly Ensembles
We build SELECTive anomaly ensembles - identify (in)accurate detectors- in unsupervised fashion
Rayana & Akoglu 7Less is More: Building Selective Anomaly Ensembles
Even
t Dete
ction
Rayana & Akoglu 8Less is More: Building Selective Anomaly Ensembles
Eigen-behaviors
Parametric modeling
SPIRIT
Z-score
1 – norm.
(sum
p-value)
projection
Subspace Method
Moving Average
SPE
Agg.
p-value
time ticks
Even
t Dete
ction
(Cyb
ern
et)
feature: degree
Rayana & Akoglu
Even
t Dete
ction
(Enro
n)
feature:
weighted in-degree
Z-score
1 – norm.
(sum
p-value)
projection
SPE
Agg.
p-value
9
Rayana & Akoglu 10Less is More: Building Selective Anomaly Ensembles
Graphs over time node feature time series
Base detectors Anomalous Subspace (ASED) [Lakhina et al. ’04] SPIRIT [Papadimitriou et al. ’05] Eigen-behavior based (EBED) [Akoglu et al. ’10] Parametric modeling (PTSAD) [Rayana&Akoglu ’14]▪ Models: Poisson, ZIP, Bernoulli+ZTP, Markov+ZTP▪ Model selection: likelihood ratio test
Moving average (MAED)
Nodes
Features(egonet)
Time
Rayana & Akoglu 11Less is More: Building Selective Anomaly Ensembles
ASED SPIRIT EBED PTSAD MAED
Base detector SELECTion
Rank based
• Inverse Rank• Kemeny-Young [Kemeny’59]
•RobustRankAggregation[Kolde+ ‘12]
Score based
• Unification [Zimek+ ‘11]
- avg & max• Mixture Model [Gao+ ‘06]
- avg & max
Consensus SELECTion & final ensemble
Rayana & Akoglu 12
Vertical SELECTion (SELECT-V)
Exploits correlation among the rank lists
Horizontal SELECTion (SELECT-H)
Exploits element wise order statistics to filter out inaccurate detectors
Less is More: Building Selective Anomaly Ensembles
Rayana & Akoglu 13Less is More: Building Selective Anomaly Ensembles
S1 S2 S3 S4 S5P1 P2 P3 P4 P5
Unification
Rayana & Akoglu 14Less is More: Building Selective Anomaly Ensembles
P1
target
avg
P2 P3 P4 P5
Pseudo ground truth
P3 is most correlated to the target
Rayana & Akoglu 15Less is More: Building Selective Anomaly Ensembles
P1
target
avg
P2 P3 P4 P5
P3
Ensemble
avg
p
Rayana & Akoglu 16Less is More: Building Selective Anomaly Ensembles
P1 P2
P3
P4 P5
Ensemble
avg
p
P1 is most correlated to p
If corr(avg(E,P1), target) > corr(p, target)accept P1
elsediscard P1
Rayana & Akoglu 17Less is More: Building Selective Anomaly Ensembles
P1 P2
P3
P4 P5
Ensemble
avg
p
P1Update until this list is empty
Rayana & Akoglu 18Less is More: Building Selective Anomaly Ensembles
P2P3
P4 P5
Ensemble
P1
Discarded
Rayana & Akoglu 19Less is More: Building Selective Anomaly Ensembles
S1 S2 S3
…
Sm
1110..
1010..
0011..
1010..
M1 M2 M3
…
Mm
Mixture Modeling• 1 (outliers)• 0 (inliers)
1010..
Majority Voting
O
Order statistics to choose accurate lists
Given m lists, for each pseudo outlier:
r = [r(1), …,r(m)], s.t. r(1) ≤ … ≤ r(m)
Under uniform null, prob. r ̂(l) ≤ r(l):
(at least l ranks drawn uniformly from [0, 1] must be ϵ [0, r(l)])Pseudo
outliers
Rayana & Akoglu 20
Example with 20 detectors
last 5 likely inaccurate
Less is More: Building Selective Anomaly Ensembles
Rayana & Akoglu 22
Full Ensemble (Full) [Rayana&Akoglu‘14]
Assemble all the detector/consensus results
Diversity-based Ensemble (DivE) [Schubert et al. 2012]
Select diverse (less correlated) detector/ consensus results to assemble
Less is More: Building Selective Anomaly Ensembles
Rayana & Akoglu 23
Data Set names duration #nodes #edges rate
1. EnronInc 4 years ~80K ~350K 1 day
2. RealityMining 50 weeks ~18K ~33k 1 week
3. TwitterSecurity 4 months ~130K ~441K 1 day
4. TwitterWCup 1 month ~54K ~274K 5 mins
5. NYTNews 7.5 years ~320K ~2980K 1 week
Less is More: Building Selective Anomaly Ensembles
• Ground truth for datasets 1-4• Qualitative evaluation for NYTNews
Rayana & Akoglu 24Less is More: Building Selective Anomaly Ensembles
Rayana & Akoglu 25Less is More: Building Selective Anomaly Ensembles
Rayana & Akoglu 26Less is More: Building Selective Anomaly Ensembles
Rayana & Akoglu 27Less is More: Building Selective Anomaly Ensembles
Rayana & Akoglu 28Less is More: Building Selective Anomaly Ensembles
Rayana & Akoglu 29
Rayana & Akoglu 30Less is More: Building Selective Anomaly Ensembles
Performance comparison
Rayana & Akoglu 31Less is More: Building Selective Anomaly Ensembles
Performance comparison
Rayana & Akoglu 32Less is More: Building Selective Anomaly Ensembles
Performance comparison
Rayana & Akoglu 33Less is More: Building Selective Anomaly Ensembles
Performance comparison
Rayana & Akoglu 36Less is More: Building Selective Anomaly Ensembles
Rayana & Akoglu 37
Feature: Weighted Degree
Rayana & Akoglu 38
Columbia Disaster
9/11 attack
New York City
World Trade Center
Washington (DC)
Afghanistan
Bin Laden, Osama
Al Qaeda
Manhattan (NY)
Bush, George W
White House Congress
New York City
World Trade Center
Washington (DC)
Afghanistan
Bin Laden, Osama
Al Qaeda
Manhattan (NY)
Bush, George W
White House Congress
Time tick 89 Time tick 90
Less is More: Building Selective Anomaly Ensembles
Rayana & Akoglu 39Less is More: Building Selective Anomaly Ensembles
A new Anomaly Ensemble SELECTive:▪ Discard inaccurate detectors▪ unsupervised
Heterogeneous ▪ different detectors▪ different consensus
2-phases:▪ No bias towards detectors & consensus
SELECT outperforms▪ Full (no selection)▪ DivE (diversity ensemble)
5 large datasets (4 w/ ground truth)
Hurt by inaccurate detectors
Rayana & Akoglu 40Less is More: Building Selective Anomaly Ensembles
Event Detection
http://www.cs.stonybrook.edu/~datalab/