Detecting Attacks in Routers Using Sketches Dhiman Barman Piyush Satapathy Gianfranco Ciardo
Dec 21, 2015
Detecting Attacks in Routers Using Sketches
Dhiman Barman Piyush Satapathy Gianfranco Ciardo
Network Attack Detection
Network anomalies are prevalent Flash crowds, DoS, failures, worms, …
Detect Anamolies quickly and accurately Two basic approaches
Statistics-based: looking for abnormal behavior E.g., heavy hitters, big changes Prior knowledge not required
Signature based: Looking for known patterns Port scan, address scan, Mal wares
Problem Addressed Accurate, memory efficient and scalable
techniques to detect attacks worms, viruses, superspreaders and DoS.
Enabling change detection in the routers by looking only at the IP headers
A general methodology to use sketches to recognize attacks in the routing architectures
Example of Attack
Domain W to Domain J
Domain X to Domain J
Domain Y to Domain J
Domain Z to Domain K
Domain J
Domain K
RouterRouter
Router
Router
Router
Router A
Attacker
2
2
2
3
4
Link C
Link B
10 Mbps
15 Mbps
15 Mbps
35 MbpsLink Capacity: 50 Mb/s
1
1. Attacker Compromises Router A2. Update Message Sent Setting Link B’s cost to 10,0003. Traffic rerouted around link B (lower cost)
4. Congestion occurs on link C DoS on domains W, X, Y and Z
Outline
Introduction Attack Definitions Sketch Background Proposed Methods Simulation and Evaluation Conclusion
Background• Worm spreads and gains control of hundreds of computers in
a few minutes– Example: Code Red version 1 and 2, Nimda etc
State of the art:– PCA (offline) [Lakhina et. al SIGCOMM’04],– Data mining [KrishnaMurthy et. al IMC’03]– Time series analysis [Estan SIGCOMM’03]
• Our Focus: Detecting and filtering attacks at short time scale by online Methods
Architecture
• Put Sketch in the Router Line Card• Sketches will update the flow cache for each given time interval• Data Collection and Analysis part will analyze the change detection
based on the algorithm and distance calculation functions.
Data CollectionAnd
Analysis Over
ForwardingHard Ware
Software
NetFlowFlow Cache
Processor
TerminatedFlow Records
SketchesBus
UpdateAlarm / No Alarm
Attack Definitions
• S is a sequence of packets identified by (<i, j, p, q>, v) where
– i and j denote packet source IP address and port
– p and q denote the destination IP address and port
– v denotes size of the packet
• Flow given by Cisco's NetFlow, R(i, j, p, q) is the multi set containing all the packets corresponding to a given (i, j, p, q) combination
Port Scans
• Attacks where a particular IP address and port pair connects to a destination on several ports
PortScan(i, j, p) |{q : | R(i, j, p, q)| > 0}| > δPS
srcIP srcPort
destPort
destIP
Address Scans• Attacks where a particular IP address
connects to multiple destination IP addresses on a particularly vulnerable port
AddrScan(I, j, q) |{p : | R(i, j, p, q) > 0}| > δAS
or
AddrScan(i, q)| |{(j, p) : |R(i, j, p, q)| > 0}| > δAS
srcIP srcPort destPortdestIP
Malware
• Attacks where a number of sources try to connect to a particular destination or a set of destinations on any ports.WormMalwSpam (i) |{(j, p, q) : | R(i, j, p; q)| > 0}| > δWMS
srcIP srcPort destPort destIP
Sketch Background A sketch of a data stream is a compact
summarization Sketched summary is much smaller than the
data stream itself Sketching is randomized projection of a signal
(using hashing)• Examples:
• Count Min Sketch [Cormode 2003]• Count Bloom Filter [Broder Internet Mathematics
2004]• Multi Count Bloom filter [Broder IM 2004]• Flajolet Martin Sketch [Flajolet CS Journal1985]
Count Min Sketch CM sketch maintains an array of width (K) and
depth (H). Hashed by choosing a hash function uniformly at
random from a pair wise independent family of hash functions.
Update (Keyt, Valuet): Tj [ hj(keyt)] += Valuet (for all j)
1
j
H
0 1 K-1…
……
+Ct
+Ct
+Ct
hj(k)
hH(k)
h1(k)
Key t
Bloom Filter• An array of (m) bits initially all set to 0
• An incoming element is hashed through (k) hash functions and bit positions returned are set to 1
• Update (Keyt): Tj [ Hj(keyt)] = 1 (for all j)
• Update (Keyt, Valuet): Tj [ hj(keyt)] += Valuet (for all j) for Counting Bloom Filter
H1
H2
H3
Hk
1
1
1
1
1
m bit Vector
K hash Functions
Key t
Multi Count Bloom Filter
• Counting Bloom Filter with m counters and divided into k groups of size (m / k) each
• ith Hash function maps from [m*(i-1)/k]+1 to [m*i/k]
• Update (keyt, valuet): Tj [ hj(keyt)] += value t (for all j)
Key t
H1
H2
H3
Hk
m countersDivided intoK groups
K hash Functions
FM Sketch
• Estimates the number of distinct items in a stream of values from [0,…, M-1]
• Assume a hash function h(x) that maps incoming values x in [0,…, M-1] uniformly across [0,…, 2L-1], where L = O(logM)
• Let lsb(y) denote the position of the least-significant 1 bit in the binary representation of y– A value x is mapped to lsb(h(x))
• Maintain FM Sketch = BITMAP array of L bits, initialized to 0– For each incoming value x, set BITMAP[ lsb(h(x)) ] = 1
Data stream: 3 0 5 3 0 1 7 5 1 0 3 7
Number of distinct values: 5
x = 5 h(x) = 101100 lsb(h(x)) = 2 0 0 0 001
BITMAP5 4 3 2 1 0
Space-Time ComplexitySketches Space Update Time Query Time
Count-Min 1/ε ; O(K) 1 1
Bloom Filter m ; Constant k k
Count Bloom Filter
mC; O(m) k k
Multi Counting Bloom filter
mC; O(m) k k
FM ML O(M) M M
C = Number of Bits in the Counter in Bloom FilterM = Number of Bit Maps used in FM Sketch
L = Number of Bits in FM SketchAll Notations Described Earlier
Our proposals Linear Combination Approach
Change between two sketches S1(α1; β1) and S2(α2; β2) is given as Sd = S2 - kS1 (sketches are closed under linear combination)
If Sd > δ then there is an anomaly and Alarm raised Use Count Min sketch, Count and Multi Count Bloom Filter
Change Detection By Sliding Window Method Using two windows, one static and one moving Parallel Execution of all the given window lengths. Use FM, Change Detection Algorithm and Distance function
Change Detection using FM Sketch
A bit vector of length equal to kK different windows sizes
Forming 2 diff. windows
M
5 10 0.05
6 8 0.04
7 4 0.03
Set the bit to 1
Inputs: (A,M); A is a mX1 matrix and M is a KX3 matrix1: c0 02: for i = 1 do3: si FMi
4: Windowx,i mx,i intervals from time c0
5: Windowy,i my,i intervals from in coming data streams6: end for7: while more flow counts to process do8: slide windowy,i by 1 sample9: if distance (windowx,i,windowy,i) ≥ αi then10: c0 current time11: Output change at time c0
12: clear all windows and GOTO step 113: end if14: end whileOutput: c0
Distance Function
• Difference between two windows calculated using:– Probability Distribution [Kifer VLDB 2004]
– L1 Distance
– KL Distance (Empirical Distribution Function)
Sketches in Action
Traces ofgivenTime slots
SketchingChange detectionmodule
(k,u) … Alarms
Pre Processing
sketches
W, d, m
K
A, M
δ
HeaderPayload
Time Key Count
LBNL &NLANRTraces
CAIDA’sCoral Software
Sketch Library
Functions
Experimental Issues
Experiments Run on Intel Xeon 1.4Ghz processor with 512 MB of RAM
Implementation of Sketches Hash functions are created randomly Random numbers are generated with the max number of 261-1
Following Hash Functions for all the 4 sketches Update (key,value) getFrequency (key)
Synthetics traces generated where attack flows are injected
Evaluation (Accuracy)
Sketches of 10 heavy ranked flowsSketching changes in heavy ranked flows between two time intervals
Evaluation (Accuracy)
FM sketch of 10 Intervals of normal traffic
FM sketch of 10 Intervals of malicious traffic trace
Scatter Plot (Accuracy)
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 100000
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
Exact Flow Size (Bytes)
Est
. Cou
ntB
loom
Flo
wS
ize(
Byt
es)
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 100000
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
Exact Flow Size (Bytes)
Est
. M
ultC
ou
ntB
loo
m S
ize
(Byt
es)
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 100000
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
Exact Flow Size (Bytes)
Est
Co
unt
Min
Flo
w S
ize
(B
yte
s)
Scatter plot of Exact Vs Est. CountMin
Count Min Sketch in Attack Detection
0 1 2 3 4 5 60
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
Number of Hash Functions (K)
Fre
qu
en
cy E
rro
r (%
)
Port Scan Attack Detection By CountMin Sketch
0 1 2 3 4 5 6
0.05
0.1
0.15
0.2
0.25
0.3
Number of Hash Functions (K)
Fre
quen
cy E
rror
(%
)
Address Scan Attack Detection By CountMin Sketch
0 1 2 3 4 5 60
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
Number of Hash Functions (K)
Fre
qu
en
cy E
rro
r (%
)
Malware Attack Detection By CountMin Sketch
Attacks Vs Sketches(Efficiency)
1 1.5 2 2.5 3 3.5 4 4.5 50
0.05
0.1
Number of Hash Functions (K)
Fre
qu
en
cy E
rro
r (%
)
Port Scan Attack Detection By Different Sketches
CountMin
CountingBloom
MultiCountingBloom
1 1.5 2 2.5 3 3.5 4 4.5 50
0.02
0.04
0.06
0.08
0.1
0.12
0.14
Number of Hash Functions (K)
Fre
quen
cy E
rror
(%
)
Address Scan Attack Detection By Different Sketches
CountMinCountingBloomMultiCountingBloom
1 1.5 2 2.5 3 3.5 4 4.5 50
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Number of Hash Functions (K)
Fre
quency E
rror
(%)
Malware Detection By Different Sketches
CountMinCountingBloomMultiCountingBloom
Evaluation by FM Sketch (Accuracy)
Change Detection Accuracy by FM sketch using probability distribution
distance function
Comparison of two different distance functions on exact
flow counts
Evaluation by FM Sketch (Efficiency)
Accuracy of two distance functions on synthetic traces. The x-axis represents values such as threshold is mean(X)+ std(X) where X is the flows counts.
Conclusions
• Sketches can detect heavy changes quickly and accurately
• Compact sketch-like data structures can be implemented on-chip hardware
• Reverse hashing of sketches to identify malicious flows is challenging
Thank you!