WebTrafMon II POSTECH DP&NM Lab. Distributed Network Traffic Monitoring and Analysis using Load Balancing Technology Soon-Hwa Hong, Jae-Young Kim, Bum-Rae Cho and James W. Hong DP&NM Lab. Dept. of Computer Science and Engineering POSTECH, Pohang Korea Email: {padosori, jay, brcho, jwkhong}@postech.ac.kr http://dpnm.postech.ac.kr/
16
Embed
Distributed Network Traffic Monitoring and Analysis …dpnm.postech.ac.kr/papers/APNOMS/01/padosori.pdf · Distributed Network Traffic Monitoring and Analysis using Load Balancing
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
WebTrafMon II POSTECHDP&NM Lab.
Distributed Network Traffic Monitoring and Analysis using Load Balancing Technology
Distributed Network Traffic Monitoring and Analysis using Load Balancing Technology
Soon-Hwa Hong, Jae-Young Kim, Bum-Rae Choand James W. Hong
DP&NM Lab. Dept. of Computer Science and Engineering
Introduction?Network traffic (text, image, software, audio, video) is
increasing continuously both on the Internet and Intranet.? A simple, accurate and efficient network traffic monitoring
and analysis is required to understand the current usage as well as to plan for future.
?Many shortcomings exist in currently available monitoring systems.? cannot analyze long-term traffic.? do not have monitoring capability from multiple network points.? capture, analysis and presentation all in one machine.? cannot prevent packet drops from the system overload.
?WebTrafMon II attempts to overcome these shortcomings using distributed architecture and load balancing technique.
total Ethernet frame size(Ethernet header + data + CRC)
Dst addr
Srcaddr Type
4 1 IP data4Dst addrSrc addr Protocol
dst portsrc port
WebTrafMon II POSTECHDP&NM Lab.
(9/16)
Packet Analysis Module (Analyzer)
… … … … …
log file Log Transformerlog data
log file
log file
database
IP-based, non IP-based data
DB Analyzer
IP-based, non IP-based data
statistical real-time, hourly, daily, monthly and yearly data.
?An Analyzer is divided into a Log Transformer module and DB Analyzer module. ?Log Transformer assorts log files into IP-based data, and non IP-based data (e.g., ARP, RARP, IPX). ?Log Transformer saves these assorted data to database. ?DB Analyzer analyzes assorted data in database and makes statistical real-time,hourly, daily, monthly and yearly data.
WebTrafMon II POSTECHDP&NM Lab.
(10/16)
Data Translation by Analyzer for Long-Term Traffic
Log files
hourly_application
hourly_data_sent
hourly_data_received
hourly_data_exchanged
hourly_ network
hourly_transport
daily_*_table monthly_*_table yearly_*_table
raw_ip_table raw_non_ ip_table
hourly
monthly yearly
daily
WebTrafMon II POSTECHDP&NM Lab.
(11/16)
DB Schema
SMALLINT UNSIGNEDdst_port
SMALLINT UNSIGNEDsrc_port
TINYINT UNSIGNED
protocol
INTdst_ip
INTsrc_ip
SMALLINT UNSIGNED
frame_type
SMALLINT UNSIGNEDlength
INTtime
Type Name
SMALLINT UNSIGNED
ether_type
SMALLINT UNSIGNED
length
INTtime
TypeName
BIGINTcount
BIGINTbytes
VARCHAR(100)frame_name
VARCHAR(20)frame_type
TINYINT UNSIGNEDday
TypeName
BIGINTcount
BIGINTbytes
VARCHAR(100)frame_name
VARCHAR(20)frame_type
TINYINT UNSIGNEDyear
TypeName
BIGINTcount
BIGINTbytes
VARCHAR(100)frame_name
VARCHAR(20)frame_type
TINYINT UNSIGNED
month
TypeName
BIGINTcount
BIGINTbytes
VARCHAR(100)frame_name
VARCHAR(20)frame_type
TINYINT UNSIGNED
hour
TypeName
raw_ip_table
raw_non_ ip_table
hourly_network_table
daily_network_tablemonthly_network_table
yearly_network_table
hourly
dailyyearly
monthly
This is a *_network_table example. network_table has IP information from raw_ip_table and ARP and so on information from raw_non_ip_table.
WebTrafMon II POSTECHDP&NM Lab.
(12/16)
Web Viewer Module Design
raw_ip, non_ip table
data_sent table
data_received table
data_exchanged table
network table
transport table
application table
Database
Web Browser(user)
Web Viewer
Database
Client
Web
Serverquery
reply
CGI
requestresponse
WebTrafMon II POSTECHDP&NM Lab.
(13/16)
http
Database
Implementation
Binary Log File
packet capture using libpcap
probe
analyzer
save packet header information into log file
save packet header information into database raw ip, non-ip, table