EP2300 SNMP Project Report Amy Skinner ([email protected]) - Laili Aidi (aidi@kthse) 1 1. Summary This project aims to design and implement a system that is able to monitor the network using SNMP and identify the specific possible attacks (DoS and port scan) using a cluster analysis. In the first task, the program discovers the topology of the network. After successful discovery phase, it will be able to monitor the link utilization (network link-states) for a specified period of time, and then detect the anomaly, using k-means clustering scheme [1]. These anomalies will be analyzed to recognize the attack. Moreover, this program also provides an advance feature, which is defined as optional task, as it executes online monitoring and detects the attacks using Davies-Bouldin Index as quality scoring measurement [2]. 2. Software Design and MIB objects A. The MIB objects which are used in this system are: i. During network crawling System Group and Interface Group (Interfaces table), as listed below: sysName, OID 1.3.6.1.2.1.1.5. This MIB object is used to get the administratively assigned name of the router ifIndex, OID 1.3.6.1.2.1.2.2.1.1. This MIB object is used to get the interface value of the router ifDescr, OID 1.3.6.1.2.1.2.2.1.2. This MIB object is used to get the description of the specific interface that is discovered previously from the ifIndex MIB object request. ipAdEntIfIndex, OID 1.3.6.1.2.1.4.20.1.2. This MIB object represent the index that identifies the interface to which it is applicable in the value ifIndex MIB object. Using this MIB Object, we can identify the Interfaces that exist in the IP routing table of the Router. ipAdEntAddr, OID 1.3.6.1.2.1.4.20.1.1. This MIB object represents the IP address of the specific interface of the Router. ii. To discover the network topology, we identified the link level neighbor of each of the identified Router using MIB Objects in Interface Group (IP Routing tables), which is the ipRouteNextHop, OID 1.3.6.1.2.1.4.21.1.7. This MIB object represents the next hop IP address of a route in the router. iii. To identify the attacks, we used two MIB Objects in the Interface Group (Interfaces table) that relate to interface utilization of a route, thus it able to represent the link-states of the network, as listed below: ifInOctets, OID 1.3.6.1.2.1.2.2.1.10. This MIB object represents the total number of octets received on the specific interface of the Router. ifInUcastPkts, OID 1.3.6.1.2.1.2.2.1.11. This MIB object represents the amount of unicast packets delivered to a higher-layer protocol. B. Below is the design of the software in this SNMP-based network management system, including the classes, key data structures and operations. A full-size class diagram is given in Appendix 5A. i. Class Start, the starting point to running the program. It contains the constant variables, used as default parameters to run the specific task, if user has not specified with command line arguments. ii. Class Router, represents the Managed node (Router), which contains: hostname, which is String data type containing the hostname of the node interfaces, which is Map of Integer (interface index) to RouterInf data structure containing the interfaces of a router localIps, which is List of Strings containing the local IP addresses of a router neighborIps, which is List of Strings containing the neighbor (next-hop) IP addresses of a router
15
Embed
SNMP Project: SNMP-based Network Anomaly Detection Using Clustering
This document contains implementation report of a system that is able to monitor the network using SNMP and identify the specific possible attacks (DoS and port scan) using a cluster analysis. In the first task, the program discovers the topology of the network. After successful discovery phase, it will be able to monitor the link utilization (network link-states) for a specified period of time, and then detect the anomaly, using k-means clustering scheme [1]. These anomalies will be analyzed to recognize the attack. Moreover, this program also provides an advance feature, which is defined as optional task, as it executes online monitoring and detects the attacks using Davies-Bouldin Index as quality scoring measurement [2]. --
Please contact trough lailiaidi at gmail.com for download request
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
This project aims to design and implement a system that is able to monitor the network using SNMP and identify the specific possible attacks (DoS and port scan) using a cluster analysis. In the first task, the program discovers the topology of the network. After successful discovery phase, it will be able to monitor the link utilization (network link-states) for a specified period of time, and then detect the anomaly, using k-means clustering scheme [1]. These anomalies will be analyzed to recognize the attack. Moreover, this program also provides an advance feature, which is defined as optional task, as it executes online monitoring and detects the attacks using Davies-Bouldin Index as quality scoring measurement [2].
2. Software Design and MIB objects
A. The MIB objects which are used in this system are: i. During network crawling System Group and Interface Group (Interfaces table), as listed below:
sysName, OID 1.3.6.1.2.1.1.5. This MIB object is used to get the administratively assigned name of the router
ifIndex, OID 1.3.6.1.2.1.2.2.1.1. This MIB object is used to get the interface value of the router
ifDescr, OID 1.3.6.1.2.1.2.2.1.2. This MIB object is used to get the description of the specific interface that is discovered previously from the ifIndex MIB object request.
ipAdEntIfIndex, OID 1.3.6.1.2.1.4.20.1.2. This MIB object represent the index that identifies the interface to which it is applicable in the value ifIndex MIB object. Using this MIB Object, we can identify the Interfaces that exist in the IP routing table of the Router.
ipAdEntAddr, OID 1.3.6.1.2.1.4.20.1.1. This MIB object represents the IP address of the specific interface of the Router.
ii. To discover the network topology, we identified the link level neighbor of each of the identified Router using MIB Objects in Interface Group (IP Routing tables), which is the ipRouteNextHop, OID 1.3.6.1.2.1.4.21.1.7. This MIB object represents the next hop IP address of a route in the router.
iii. To identify the attacks, we used two MIB Objects in the Interface Group (Interfaces table) that relate to interface utilization of a route, thus it able to represent the link-states of the network, as listed below:
ifInOctets, OID 1.3.6.1.2.1.2.2.1.10. This MIB object represents the total number of octets received on the specific interface of the Router.
ifInUcastPkts, OID 1.3.6.1.2.1.2.2.1.11. This MIB object represents the amount of unicast packets delivered to a higher-layer protocol.
B. Below is the design of the software in this SNMP-based network management system, including the classes, key data structures and operations. A full-size class diagram is given in Appendix 5A.
i. Class Start, the starting point to running the program. It contains the constant variables, used as default parameters to run the specific task, if user has not specified with command line arguments. ii. Class Router, represents the Managed node (Router), which contains:
hostname, which is String data type containing the hostname of the node
interfaces, which is Map of Integer (interface index) to RouterInf data structure containing the interfaces of a router
localIps, which is List of Strings containing the local IP addresses of a router
neighborIps, which is List of Strings containing the neighbor (next-hop) IP addresses of a router
iii. Class RouterInf, represents the network interface of the router, which contains:
The IP Address, called ip, which is String data type.
The description, called desc, which is String data type. iv. Class SNMPUtils, is static class that provide the SNMP values and operations that are needed to accomplish the task, which are:
OID, which is Map of Strings from the human readable OID names data type to Strings of OID numeric values, for the MIB objects of which are requested during the run of this program
open() and close(), opens and closes the SNMP session
getVarBind(), returns SNMP getNext MIB variable-value binding (value with its OID)
getVar(), returns SNMP getNext MIB value v. Class SNMPCrawler, responsible for the node and link discovery task of the test-bed network:
createRouter(), creates a router and add to global list of routers operation
addInterfaces(), discovers and adds a list of the Interfaces of a router
addNeigbors(), discovers and adds a list of the link level Neighbors of a router vi. Class SNMPPoller, provides polling operation to capture the link-states of routers:
poll and onlinePoll, operations used in Task 2, respective Task 3, to poll all routers for a specified period of time and quit, or to continuously poll and call Clusterer after w polling rounds.
xRounds and yRounds, which is Hashtable of integer data type to List of Long data structure. This Integer represent the round number, and the List of Long data structure contains the sum of ifInOctets, respective ifInUcastPkts, from every interface of each router in each round
vii. Class PollingTread
This class has composition relationship with Class SNMPPoller, which polls the information of the routers simultaneously in every round. vii. Class Clusterer, is a Thread that provides clustering calculation based on k-means clustering method and/or Davies-Bouldin Index and show the result. This class contains 2 data structures which represent the global-state of the network in every round, and operations which are:
deltXt and deltYt, the delta values of MIB object ifInOctets and MIB object ifInUcastPkts from all routers in every round, calculated from the average value of the sum of MIB object from all interfaces from all routers in round t
cluster(), the cluster formation operation, which is used to perform clustering until the it is convergence or reach the maximum iteration for convergence (10 iterations)
getNewCentroids(), calculates the centroids from a list of type Cluster
calcDbi(), DBI value operation, used to get the Davis-Boulman Index of the clusters in each calculation for the same dataset.
findAttacks(), identifies the DoS and port scan attacks. vii. Class Cluster
This class has composition relationship with Class Clusterer, represent the cluster object, containing
CentroidX and CentroidY, the X, respective Y values of the centroid
Xs and Ys, holds all the X, respective Y values of all the points in this cluster
getNumPoints, returns the number of points in this cluster
while (!haveSameCentroids(clusters, newCentroids)) {
clusters = cluster(newCentroids);
newCentroids = getNewCentroids(clusters);
if (numIterations++ > MAX_CLUSTERING_ITERATIONS)
break;
} }
double dbi = computeDbi(clusters);
Our clustering algorithm is based off the instructions in the project description in sections 2.3b and 2.3c [3]. For all of our calculations we have kept track of the values for the x value (the sum of the ifInOctets MIB values for every interfaces on a given router) and the y value (the sum of the ifInUcastPkts MIB values for every interface on a router) as separate variables, to the data structures as simple as possible, since they both change and are operated on independent of each other. At the end of the polling phase we have two tables that hold all of the polled values, xRounds and yRounds. These tables have the polling round number as keys, and the values are lists of the x or y values from all routers that responded with valid results for the corresponding polling round. This data, along with an integer interval, specifying how often polling should occur, an integer k, to indicate the number of clusters that should be created, and repeats, to indicate how many times we should recalculate the clusters for a different time period, are the inputs to the clustering function. The clustering algorithm begins by determining the average global state for each round, by summing up all values the list for that round, and then dividing by the number of responses in the list. This number can vary, if we have received a timeout when requesting a MIB value from a router. One of the biggest design choices for this project was to decide how to handle these timeouts. If we receive a timeout from a router while trying to get information about one of the interfaces, we do not add the information received from the other interfaces to the list for the round, so there will be one less entry in the list from this round. We have chosen to do this, because we assume that the null responses from the routers occur independently of when an attack occurs (only as a result of too many students executing at one time), so we do not want to bring down the global state average for that round and create something that may look anomalous, but not because of an attack. Another facet of this decision was if we received a null during a poll of the x value from a router, but not during a poll of the y value, should we add the y sum value to the y list, even though we are not adding the x value to the x list? We have decided that because we are only dealing with global averages in this project, and not with the information from specific routers, that there is no reason why we cannot return one MIB sum value to help calculate the average. All of these conditions can be seen in the run() method of the PollingThread class, in SNMPPoller.java.
After averaging, we determine the changes in global state. This is done by finding the differences between the averages, stored as deltXt and deltYt. Since we are storing all data for x and y separately, there is the possibility that this lists could become different lengths. However, this could only occur if we were to get null responses from all routers for one of the values, but not the other, in a given round. We consider this to be incredibly unlikely, so we assume the size of x list to be the same as the size of the y list. In the next step, we run a loop to pick a new random points in the data set to be initial centroids (x(p), y(p)). Then we make a loop though all of the rounds, and for each point (x(t),y(t)), we record the Euclidean distance to all of the initial centroids. After calculating the distances, the point is then added to a cluster with the centroid which is closest to itself. The clusters are stored as lists of type Cluster. Then we compute the new centroid for each cluster. This step is repeated until we get the same centroids after an iteration (convergence), or until a maximum number of iterations (10) is met. After the clusters are created, the Davies-Bouldin index (DBI) is computed for each clustering round. This entire clustering process is repeated repeats number of times, to compare values discovered from clusterings with different random initial centroids.
/** Anomaly detection **/ int largestCentroidCluster = clusters.getClusterWithLargestCentroid();
long largestCentroid = clusters.get(LargestCentroidCluster).getCentroid();
int secondLargestCentroidCluster = clusters.getClusterWithLargestCentroid();
long secondLargestCentroid = clusters.get(LargestCentroidCluster).getCentroid();
The anomaly detection scheme (ADS) works by using the qualities listed in section 2.3d of the project description. First, the top two clusters are picked by their centroid value, which means the two clusters whose centroid is furthest from the origin. After that, we determine if attacks have happened, by testing if the qualities of these two clusters agree with qualities laid out in the project description, that is to say, that if the largest of the two centroids has a smaller size, we call this a DoS Attack, and can therefore call the other cluster a port scan attack. If these two clusters do not share these qualities, we consider that it is indeterminate whether there was an attack or not. This can happen due to poor choice of random initial centroids which prohibit the clusters from forming in predictable ways. Our clusterer takes a variable integer repeats, which controls how many times we repeat the calculations with different initial random centroids, that we are more accurately able to say during which rounds there may have been an attack. We have decided to run the ADS on all clusterings, rather than just the clusterings with the lowest DBI, because we have found that the clusterings with the lowest DBI do not always show the most accurate attack detection (see Section 4A). We do however determine and output which clustering has the lowest DBI, to conform to the requirements of Task 3.