Network-based Intrusion Detection and Prevention in Challenging and Emerging Environments: High-speed Data Center, Web 2.0, and Social Networks Yan Chen Lab for Internet and Security Technology (LIST) Department of Electrical Engineering and Computer Science Northwestern University
41
Embed
Network-based Intrusion Detection and Prevention in Challenging and Emerging Environments: High-speed Data Center, Web 2.0, and Social Networks Yan Chen.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Network-based Intrusion Detection and Prevention in Challenging and
Emerging Environments: High-speed Data Center, Web 2.0, and Social
NetworksYan Chen
Lab for Internet and Security Technology (LIST)
Department of Electrical Engineering and Computer Science Northwestern University
Chicago
2
Northw
estern
3
4
Statistics
• Chicago: 3rd largest city in US
• NU: ranked #12 by US News & World Report– Established in 1851– ~8000 undergrads
• McCormick School of Engineering: ranked #20– 180 faculty members– ~1400 undergrads and similar # of grad students
5
Statistics of McCormick
• National academy memberships: – National Academy of Engineering (NAE): 12 active,
7 emeriti– National Academy of Science (NAS): 3 active– Institute of Medicine (IoM): 1 emeritus– American Academy of Arts and Sciences (AAAS): 5
active, 3 emeriti– National Medal of Technology: 1 active
Observation 1: Only need to parse the fields related to signatures (mostly leaf nodes)
Observation 2: Traditional recursive descent parsers which need one function call per node are too expensive
3030
Efficient Parsing with State Machines
• Studied eight protocols: HTTP, FTP, SMTP, eMule, BitTorrent, WINRPC, SNMP and DNS as well as their vulnerability signatures
• Common relationship among leaf nodes
• Pre-construct parsing state machines based on parse trees and vulnerability signatures
Varderive
Sequential Branch Loop Derive(a) (d)(c)(b)
VarVar
Outline
• Motivation
• High Speed Matching for Large Rulesets.
• High Speed Parsing
• Evaluation
• Research Contributions
31
Evaluation Methodology
• 26GB+ Traces from Tsinghua Univ. (TH), Northwestern (NU) and DARPA
• Run on a P4 3.8Ghz single core PC w/ 4GB memory• After TCP reassembly and preload the PDUs in memory• For HTTP we have 794 vulnerability signatures which
cover 973 Snort rules.• For WINRPC we have 45 vulnerability signatures which
cover 3,519 Snort rules
Fully implemented prototype 10,000 lines of C++ and
3,000 lines of PythonDeployed at a DC in TsinghuaUniv. with up to 106Mbps
32
Parsing Results
Trace TH DNS
TH WINRPC
NU WINRPC
TH HTTP
NU HTTP
DARPA HTTP
Avg flow len (B) 77 879 596 6.6K 55K 2.1K
Throughput (Gbps)
Binpac
Our parser
0.31
3.43
1.41
16.2
1.11
12.9
2.10
7.46
14.2
44.4
1.69
6.67
Speed up ratio 11.2 11.5 11.6 3.6 3.1 3.9Max. memory per connection (bytes)
16 15 15 14 14 14
33
Parsing+Matching Results
Trace TH WINRPC
NU WINRPC
TH HTTP
NU HTTP
DARPA HTTP
Avg flow length (B) 879 596 6.6K 55K 2.1K
Throughput (Gbps)
Sequential
CS Matching
10.68
14.37
9.23
10.61
0.34
2.63
2.37
17.63
0.28
1.85Matching only time
speedup ratio4 1.8 11.3 11.7 8.8
Avg # of Candidates 1.16 1.48 0.033 0.038 0.0023Avg. memory per connection (bytes)
32 32 28 28 28
11.08-core
34
Scalability Results
0 200 400 600 800
01
23
4
# of rules used
Th
rou
gh
pu
t (G
bp
s)
Performancedecreasegracefully
35
36
Accuracy Results
• Create two polymorphic WINRPC exploits which bypass the original Snort rules but detect accurately by our scheme.
• For 10-minute “clean” HTTP trace, Snort reported 42 alerts, NetShield reported 0 alerts. Manually verify the 42 alerts are false positives
Research Contribution
Regular Expression Exists Vul. IDS NetShield
Accuracy Poor Good Good
Speed Good Poor Good
Memory Good ?? Good
• Multiple sig. matching candidate selection algorithm
• Parsing parsing state machine
Tools at www.nshield.org
Make vulnerability signature a practical solutionfor NIDS/NIPS
37
38
Q&A
Q&A
4. Vulnerability Signature Matching for Large Ruleset Complexity Analysis
Three HTTP traces: avg(|Si|)<0.04 Two WINRPC traces:
avg(|Si|)<1.5
Merging complexity Need k-1 merging iterations For each iteration
Merge complexity O(n) the worst case, since Si can have O(n) candidates in the worst case rulesets
For real-world rulesets, # of candidates is a small constant. Therefore, O(1)
For real-world rulesets: O(k), which is the optimal case
4040
Example for WINRPC• Rectangles are states• Parsing variables: R0 .. R4
• 0.61 instruction/byte for BIND PDU
1 rpc_ver_minor
R4
20*R4
R2++R2£R3
R2 ‹- 0R3 ‹- ncontext
Header BindR0
R0
R1-16
Bind
Bind-ACK
R1
Bind-ACK
1 rpc_vers
1 pfc_flags
1 ptype
2 frag_length
4 packed_drep
6 merge1
1 n_tran_syn
2 ID
16 UUID
1 padding
tran_syn4 UUID_ver
1 ncontext
8 merge2
3 padding
merge3
41
Parser generator
• We reuse the front-end of BinPAC (a Yacc like tool for protocol parsing)
• Redesign the backend to generate the parsing state machine based parser