
2/33

Introduction (Patrick Lloyd)

With the vast expansion of the internet and the exponential increase in number of users utilizing the

internet, there came a problem with unseen proportions. The internets original development purpose was

to be a small scale defense network and allow for a mesh to interconnect essential government resources

for use in case of an enemy attack. Once it became part of civilian use, basic structures that meant to

handle very little traffic load and a limited number of users, became overloaded and bogged down. This

has been a piece of discussion during Network Management and therefore sparked our interests as to how

much of a performance degradation occurred as usage increased and more expectations were place on

routers and other networking devices. Therefore, our experiment was to explore the links between the

number of routers and the number of routing tables put on those routers and how this affected the

performance of the network.

Background (Patrick Lloyd)

It is approximated that the internet as we know it, has over 120,000 routes in it

(http://www.employees.org/~tbates/cidr.plot.html). This means that traffic that is forwarded from one

domain or web site to one at any other place one the internet, could be compared to, at most, 120,000

different sites before it reaches its final destination. This also means that routers that are used for moving

traffic from one network to another have to have multiple IP routes programmed into them. IP routes are

lists that the routers read to know where another network is located, based on its IP address, subnet mask

and the interface that the router will have to go through to get to that network. To make system

administrators lives easier, not all routes have to be programmed in manually.

RIP, or router information protocol, is a protocol built into most routers in use today and allows for an

administrator to tell a router to advertise a network on a given interface. With the routers that we used in

this experiment, there were two interfaces and the routers were aligned in a topology that was successive,


3/33

or one router just connected to another. More information on the topology and drawings of it will be

given later in the paper.

Our original experiment was based on the idea of how long each of these routers, when connected

together, would take to converge and allow for traffic to be sent across them. This would solve the

question of how long a network outage would last, in the very least, if a new router had to be

implemented into the network, assuming it already had RIP and its router tables configured. The first

complication was found when we attempted to use the cisco 2514 routers in the Netsys lab, and received

numerous errors while not returning a SNMP Rip settings. This, we later found out, was because the

routers were running ios version 12.1(27b) and the mib import was only compatible with versions 12.4(6)

and higher.

Upon obtaining a flash card compatible with a cisco 2800 and using 12.4(6) we found that the ripv2 mib

was unsupported. This was also the same with 12.4(18), with each time we specified the OID that we

wanted to poll, we received an OID not supported error. The conclusion that we came to was that the

computer which we were using for the connection and SNMP polling abilities of the router, had the MIB

on it and allowed for access by using the computer (we could view the variables in both our translations

and network management viewing application) but it was not supported or available on the router. This

was a large point of confusion due to no documentation not being available to explain this on the Cisco

site, or any other internet site we used for research.

After our previous failure and problems that were caused by trying to poll the cisco routers RIP settings,

we decided that we could still test the impact of routing table size by using static routes. In the appendix,

there is a script we wrote to generate thousands of routes which can be pasted into a router configuration

file and uploaded via TFTP. We generated 5,000 routes to upload to our test routers using this script.

Hypothesis (Patrick Lloyd)


4/33

As mentioned previously, we decided that we wanted to create a script that would monitor network usage

and determine if throughput and utilization changed for a network of routers, with a computer on either

end doing a file transfer, when multiple routes are programmed into the routers. Our prediction was that

once multiple routes were programmed into the router with a script that we made, the router would take a

longer time and utilize less bandwidth on account to send the information between two sources because of

having to check the routing table for each packet and determine the location of the actual recipient.

Therefore, as we changed the network design from 2 to 4 to 7 routers, the utilization, in theory would

change as well, with throughput decreasing measurably with the increase in size.

Program Architecture (Adam Markert)

The script programming for this project has several components. When we first decided to create routing

tables that would be burdensome to our hardware, we knew it would require thousands of routes. The

only way to create those efficiently is with a script. Appendix A shows the commented source code for

this script.

Appendix B shows the commented code for the main program. It is a shell script which uses awk and

snmpget commands. This program provides an interface which can be used to add/remove network

devices to a MySQL database called perfmon. This database also contains a table used to store

performance data. The program interface also allows the user to change polling options such as polling

interval and duration. Statistics calculation and reporting are also done in this program and the fully

commented code is available in the Appendix.

Appendix C is the code for an awk script which is used to determine a total for a given set of MIB object

values. For example, it will determine the total bytes in for a given router or interface using the data

collected in the MySQL database. This script is written to catch counter roll-overs and will give a true

total no matter how long the polling application runs.


5/33

This application was designed to use SNMPv3 security. The sample router configuration in Appendix D

shows how the agent is configured for SNMPv3. Our scripts also use snmpget v3, but there is one more

component to this. Our manager workstation has had the authentication information programmed into

snmp.conf so that it is not necessary to put the full information in at the command line. Appendix E is

our snmp.conf file and should be used on any system to make using SNMPv3 easier.

Testing Methodology (Text: Patrick Lloyd, Diagrams: Adam Markert)

As mentioned previously, we decided that we wanted to create a script that would monitor network usage

and determine if throughput and utilization changed as we added in multiple routers. We started by using

7 routers in succession and configuring each of them to communicate with each other through RIP as

routing paths to multiple networks. So each router, being the connected node between networks, was able

to see the networks that we had set up:

192.168.1.0 255.255.255.0

192.168.2.0 255.255.255.0

192.168.3.0 255.255.255.0

192.168.4.0 255.255.255.0

192.168.5.0 255.255.255.0

192.168.6.0 255.255.255.0

192.168.7.0 255.255.255.0

192.168.8.0 255.255.255.0


6/33

These networks each utilized one port from two routers with an address list looking like the one in figure

1. In addition to the networks that we set up with the routers, we also set up one computer on either side

of the network which would be our test machines. These two computers would be the sender and receiver

of multiple large files sent across the network. We chose to send multiple large files across the network

instead of a single one to ensure that there was a change in packets when the new files were sent, instead

of a single file being sent with the same successive information.

Once all routers were set up on the network and configured to only have the needed routing tables

programmed into them, we assessed connectivity across the network to find that OSPF, or open shortest

path first, if not configured correctly on the network, will prevent the connection between routers for a

significant amount of time. Once we were able to establish total connectivity, we copied down the router

configurations for later modification.

With a connection between computers, a file was sent from computer A to computer B and the utilization

of the network monitored with both our SNMP script, as well as NetMeter, a freeware program which

monitors network traffic utilization as it enters and exits a network interface card.


7/33

Figure 3: 8 Routers and their network utilization

As can be seen in figure 3, network traffic is traveling at a rate of around 738.6 kilobytes per second with

a maximum of 829.1 kilobytes per second. This tended to be the average of the overall file transfer.

Once the first trial was done, we proceeded to scale the network back to 4 routers and see if there was

much difference in the transfer speed and utilization.

Figure 4: 4 routers and their network utilization

When we scaled back to 4 routers, there was a bit of a performance increase in network utilization,

mainly because of the less number of routing tables that the router had to scan through to find the

destination of the packet. As can be seen here, there is a relative average of 942.7 kilobytes per second

when sending the files from one end of the network to another, with a maximum of 972.4 kilobytes per

second. This related to the first trial that we ran by being 204.1 kilobytes faster for the average speed of

the transfer and 143.3 kilobytes faster than the maximum transfer speed.


8/33

Figure 5: 2 routers and their network utilization

The pattern that we found with scaling back from 8 to 4 routers, followed a relatively similar fashion

when we decreased to 2 routers. As can be seen in figure 5, there was an increase in maximum speed

from 972.4 to 1000.3 kilobytes per second, a difference of 27.9 kilobytes per second. Even though this

was not a significant difference as compared to the first two trials, we also only removed two routers from

the mix instead of three, and had two computers connected to two routers (a relatively small network).

The average also changed in this trial, increasing from 942.7 to 973, an increase of 30.3 kilobytes per

second.

Figure 6: 8 routers with multiple routing tables added

For our fourth trial, we decided to start with our full network again, and add in multiple routing tables to

the routers. The routing tables pointed to paths that were actually nonexistent, but still required for the

routers to check multiple paths for the proper one to send the packets to, when allowing them to traverse


9/33

the network. The most surprising result came with the inspection of the data from the second round of

trials and comparing it the first round of trials. Where as in the first round of trials the traversal rate of the

network was 738.6 kilobytes per second with a maximum of 829.1 kilobytes per second, in the second

round of trials, for 8 routers this speed increased. As can be seen in figure 6, there was an average speed

of 859.3 and a maximum of 872.9 kilobytes per second. This was actually an increase over the round 1

trial of 8 routers by 120.7 average and 42.8 maximum. As the following screen shots will show, this

trend continues for all of the trials.

Figure 7: 4 routers with multiple routing tables added

We can see here that there was an overall drop in average download speed when we scaled our network

back to four routers and added multiple IP routes to it, with the average download speed being 921.9

kilobytes per second as compared to the original 942.7 kilobytes per second. On the other hand, there

was an overall increase in maximum download speed with an increase from 972.4 kilobytes per second to

997.5 kilobytes per second, a change of 24.9 kilobytes per second change when we added in the IP routes.


10/33

The minor exception to the rule came with 2 routers, where the average download speed was roughly the

same as it was in the first trial, but there was a slight drop in the maximum download speed, from 1000.3

to 987.9 kilobytes per second. We do not feel that this will affect our conclusion, due to such a minor

change between the two.

Analysis (Patrick Lloyd and Adam Markert)

In our experiment, we tried to replicate a situation that would be of interest to a network or systems

administrator. When it comes to administering a large enterprise size network, or even a small LAN,

when performance is vital to the network operations, this experiment shows that even by increasing the

number of hops that traffic has to travel by a power of 2, there will be very little difference in the speed of

network traffic, even with caching turned off. This means that there is less worry when expanding a

current network that the expansion will be detrimental to the networks traffic load abilities.

[Adam Market]

In our hypothesis, we predicted that the addition of numerous routes would have a cumulative impact on

network throughput. Our data seemed to show otherwise though, with unclear results as to any

performance impact. After doing some additional research, it seems that our understanding of how

routers select routes was rather simplistic.


11/33

Our initial assumption was that an incoming packets destination network was compared, one by one to

every route to find a matching destination. The final next hop address would be determined based on the

addresses found and any metric information. In reality, a router is more sophisticated than this. Exact

operation varies by vendor, but most routers cache commonly used routes. The result here is what we

observed. The first packet from source to destination might experience a slight delay, but once a good

route is established subsequent packets travel much quicker. A 5-minute trial with our setup would not

show any appreciable difference in average throughput.

Conclusion (Adam Markert)

In conclusion, we have determined that while router tables can be memory-intensive and theoretically

detrimental to performance, modern router software is very efficient at finding and caching routes for

traffic. The result is that performance is largely unaffected by a properly configured and equipped router

with a typical traffic load.


12/33

Figure 1 Diagram of network layout

Appendix A tablegen.sh

#!/bin/bash

#Written by: Adam Markert

#Rochester Institute of Technology

#Feb, 2009

#Purpose: Generates ip routes to be pasted into a router config file

#Output to a text file at the command line to make pasting easier.

x=255 #Max. Number of X in ip address where ip = A.B.X.Y

y=4 #Number to increase by after each loop iteration

lcv=2 #Address to start at

lcv2=0

#Here we have a nested loop that basically increments by Y

#Producing a new 30-bit subnet on each line


13/33

while [ $lcv -lt $x ]

do

while [ $lcv2 -lt $x ]

do

echo "ip route " 172.16.$lcv"."$lcv2 " 255.255.255.252

192.168.1.1"

lcv2=`expr $lcv2 + $y`

done

lcv2=0

lcv=`expr $lcv + 1`

done


14/33

Appendix B menu.sh

#!/bin/bash

#Written By: Adam Markert & Patrick Lloyd

#February, 2009

#Network Management, Prof.Shenoy


#Clear out screen for menu

clear

#Set up variables

CHOICE=0

DURATION=600 #Default polling duration in seconds

INTERVAL=10 #Default polling interval in seconds

#Main program loop, provides menu functionality

while [ ! $CHOICE -eq 8 ]

do

echo "\n----------------------------------"


15/33

echo "Network Performance Monitor"

echo "----------------------------------"

echo "1. Add Device"

echo "2. Remove Device"

echo "3. Show Devices"

echo "4. Modify testing parameters"

echo "5. Begin Test"

echo "6. Show Statistics"

echo "7. Clear Database"

echo "8. Quit"

echo "-------------------------------"

echo -n "\nMake Selection: "

read CHOICE

#User makes choice from menu above

case $CHOICE in

1) echo -n "Add Device - Enter Name: "

read DEVICE

echo -n "Add Device - Enter Mgmt IP Address: " #Add Device to file


16/33

read IP

echo -n "Add Device - Enter # of Interface: " #Specify interface to poll

read INT

mysql -u root -ppass123 -e "INSERT INTO perfmon.devices (device_name, ip_address, interface)

VALUES ('$DEVICE', '$IP', '$INT');" #Add device to SQL database

;;

2) echo -n "Delete Device - Enter Name: " #Remove Device from file

read DEVICE

mysql -u root -ppass123 -e "DELETE FROM perfmon.devices WHERE device_name = '$DEVICE'" #Removes device from SQL database

;;

3)clear

echo "\n-----------"

echo "Device List"

echo "-----------"

mysql -u root -ppass123 -e "SELECT * FROM perfmon.devices" #Displays a list of all devices/interfaces

echo "Press Any Key To Continue..."

read ANY

;;

4)echo "\n---------------------"

echo "Select Parameter To Change"


17/33

echo "---------------------"

echo "1. Polling Interval (Currently " $INTERVAL " seconds)" #Allows users to change duration/interval for polling

echo "2. Polling Duration (Currently " $DURATION " seconds)"

echo "3. Back"

echo -n "Make Selection: "

read OPTION

case $OPTION in

1)echo -n "New Interval: "

read INTERVAL

;;

2)echo -n "New Duration: "

read DURATION

;;

3) ;;

esac

;;

5)STARTTIME=$(date +%s) #Records the start time

while [ `expr $(date +%s) - $STARTTIME` -lt $DURATION ] ; do #While the current time minus start time is less than specified duration...

mysql -u root -ppass123 --skip-column-names -e "SELECT ip_address FROM perfmon.devices" > devices.txt #Output all device

ip's to device.txt


18/33

while read LINE #For each ip address

do

#Gather interface information for this device

mysql -u root -ppass123 --skip-column-names -e "SELECT interface FROM perfmon.devices WHERE ip_address = '$LINE'" > interfaces.txt

#For each interface number

while read INT

do

#Collect counter data IP/Interface pair

INBYTES=$(snmpwalk -v 3 $LINE IF-MIB::ifInOctets.$INT | awk ' BEGIN { FS = "Counter32: " } { print $2 }')

OUTBYTES=$(snmpwalk -v 3 $LINE IF-MIB::ifOutOctets.$INT | awk ' BEGIN { FS = "Counter32: " } { print $2 }')

INERR=$(snmpwalk -v 3 $LINE IF-MIB::ifInErrors.$INT | awk ' BEGIN { FS = "Counter32: " } { print $2 }')

OUTERR=$(snmpwalk -v 3 $LINE IF-MIB::ifOutErrors.$INT | awk ' BEGIN { FS = "Counter32: " } { print $2 }')

INDISCARD=$(snmpwalk -v 3 $LINE IF-MIB::ifInDiscards.$INT | awk ' BEGIN { FS = "Counter32: " } { print $2 }')

OUTDISCARD=$(snmpwalk -v 3 $LINE IF-MIB::ifOutDiscards.$INT | awk ' BEGIN { FS = "Counter32: " } { print $2 }')


19/33

INUCAST=$(snmpwalk -v 3 $LINE IF-MIB::ifInUcastPkts.$INT | awk ' BEGIN { FS = "Counter32: " } { print $2 }')

OUTUCAST=$(snmpwalk -v 3 $LINE IF-MIB::ifOutUcastPkts.$INT | awk ' BEGIN { FS = "Counter32: " } { print $2 }')

INBCAST=$(snmpwalk -v 3 $LINE IF-MIB::ifInNUcastPkts.$INT | awk ' BEGIN { FS = "Counter32: " } { print $2 }')

OUTBCAST=$(snmpwalk -v 3 $LINE IF-MIB::ifOutNUcastPkts.$INT | awk ' BEGIN { FS = "Counter32: " } { print $2 }')

#Store variables in database table

mysql -u root -ppass123 -e "INSERT INTO perfmon.perfdata (device_name, errors_in, errors_out, bytes_in,

bytes_out, interface_num, discards_in, discards_out, ucast_in, ucast_out, bcast_in, bcast_out) VALUES ('$LINE','$INERR', '$OUTERR', '$INBYTES', '$OUTBYTES', '$INT', '$INDISCARD',

'$OUTDISCARD', '$INUCAST', '$OUTUCAST', '$INBCAST', '$OUTBCAST')"

done < interfaces.txt

done < devices.txt

echo -n "."

sleep $INTERVAL

done

;;

6) echo "\n--------------"

echo "Reporting Options"

echo "-----------------"


20/33

echo "1. Report on All Devices"

echo "3. Back"

echo -n "Make Selection: "

read OPTION

case $OPTION in

1) mysql -u root -ppass123 --skip-column-names -e "SELECT ip_address FROM perfmon.devices" > devices.txt

while read LINE

do

mysql -u root -ppass123 --skip-column-names -e "SELECT interface FROM perfmon.devices WHERE ip_address = '$LINE'" > interfaces.txt

#For each IP/Interface combo, generate a report

while read INT

do

echo "\n-------------------------------"

echo "Report for Device: " $LINE

echo "Interface number: " $INT

echo "-------------------------------"

#SQL database is queried for values, which are placed in a temporary text file

#The text file is run through an awk script which totals them and outputs a result

#The result is displayed for the user for a total, or divided by the polling duration for an average value


21/33


22/33

mysql -u root -ppass123 --skip-column-names -e "SELECT bcast_out FROM perfmon.perfdata WHERE device_name = '$LINE'" >

temp.txt

echo "Total Broadcast Packets Out: " $(awk -f totaler.awk temp.txt)

echo "Avg. Broadcast Packets Out: " èxpr $(awk -f totaler.awk temp.txt) / $DURATION` " pkts/sec."

mysql -u root -ppass123 --skip-column-names -e "SELECT discards_in FROM perfmon.perfdata WHERE device_name = '$LINE'" >

temp.txt

echo "Total Discarded Packets In: " $(awk -f totaler.awk temp.txt)

echo "Avg. Discarded Packets In: " èxpr $(awk -f totaler.awk temp.txt) / $DURATION` " pkts/sec."

mysql -u root -ppass123 --skip-column-names -e "SELECT discards_out FROM perfmon.perfdata WHERE device_name = '$LINE'"

> temp.txt

echo "Total Discarded Packets Out: " $(awk -f totaler.awk temp.txt)

echo "Avg. Discarded Packets Out: " èxpr $(awk -f totaler.awk temp.txt) / $DURATION` " pkts/sec."

mysql -u root -ppass123 --skip-column-names -e "SELECT errors_in FROM perfmon.perfdata WHERE device_name = '$LINE'" >

temp.txt

echo "Total Errors In: " $(awk -f totaler.awk temp.txt)

echo "Avg. Errors In: " èxpr $(awk -f totaler.awk temp.txt) / $DURATION` " pkts/sec."

mysql -u root -ppass123 --skip-column-names -e "SELECT errors_out FROM perfmon.perfdata WHERE device_name = '$LINE'" >

temp.txt

echo "Total Errors Out: " $(awk -f totaler.awk temp.txt)

echo "Avg. Errors Out: " èxpr $(awk -f totaler.awk temp.txt) / $DURATION` " pkts/sec."

#Here we determine link utilization by polling the device for its interface link speed


23/33

#Then the data rate is calculated based on the information collected above

#To determine utilization we divide the rate by the link speed and multiply by 100 to get the percentage.

echo "\nInterface " $INT " speed: " $(snmpwalk -v 1 -c public $LINE IF-MIB::ifSpeed.$INT | awk 'BEGIN { FS="Gauge32:

" } { print $2 } ' )

LINKSPD=$(snmpwalk -v 1 -c public $LINE IF-MIB::ifSpeed.$INT | awk 'BEGIN { FS="Gauge32: " } { print $2 }')

RATE=`expr ($AVGBYTERATEIN + $AVGBYTERATEOUT) \* 8`

UTILIZATION= expr $RATE / $LINKSPD \* 100`

echo "Interface Avg. Utilization: " $UTILIZATION "%"

echo "------------------------------"

echo "End of Report"

echo "------------------------------\n"

done < interfaces.txt

done < devices.txt

;;

esac

;;

7) echo "Database Cleared." #Clears performance data only, not device information

mysql -u root -ppass123 -e "DELETE FROM perfmon.perfdata"

;;


24/33

8) break ;;

esac

done


25/33

Appendix C totaler.awk

#Written by: Adam Markert


#February, 2009

#Network Management, Prof. Shenoy

#Accepts a text file list of values and computes a total

BEGIN { STARTTOTAL = 0; RUNNINGTOTAL = 0; LASTTOTAL = 0; ITERATION=0}

{

#First iteration, make starttotal equal to value of first line in file

if ( ITERATION == 0)

STARTTOTAL = $1

#increment iteration value

ITERATION++

#If the current value is greater than the last value, simply add the incremental amount

#to the running total

#If it is less, then the counter has wrapped. Subtract the last total from the max value

of


26/33

#a 32-bit counter, then add this to the current runningtotal and the current value from

the text value

#to achieve a new, correct running total

if ( $1 >= LASTTOTAL ) {

RUNNINGTOTAL = RUNNINGTOTAL + ($1 - LASTTOTAL)

} else {

RUNNINGTOTAL = RUNNINGTOTAL + $1 + (4294967294 - LASTTOTAL)

}

LASTTOTAL = $1

}

END { print RUNNINGTOTAL - STARTTOTAL}


27/33

Appendix D sample router configuration

!

version 12.4

service timestamps debug datetime msec

service timestamps log datetime msec

no service password-encryption

!

hostname RouterA

!

boot-start-marker

boot-end-marker

!

enable secret 5 $1$sNC8$75vT0QyuxG.0Rl0.vXHDx0

enable password pass

!

no aaa new-model

no network-clock-participate wic 0

!

!

ip cef


28/33

!

!

ip auth-proxy max-nodata-conns 3

ip admission max-nodata-conns 3

!

!

voice-card 0

no dspfarm

!

!

controller T1 0/0/0

framing esf

linecode b8zs

!

interface FastEthernet0/0

ip address 192.168.2.1 255.255.255.0

duplex half

speed auto

no mop enabled

!

interface FastEthernet0/1

ip address 192.168.1.1 255.255.255.0


29/33

duplex half

speed auto

!

interface FastEthernet0/1/0

shutdown

!


shutdown

!


shutdown

!


shutdown

!

interface Vlan1

no ip address

shutdown

!

router rip

network 192.168.1.0


30/33

network 192.168.2.0

!

ip forward-protocol nd

!

!

ip http server

no ip http secure-server

!

snmp-server community public RO

!

!

!

!

snmp-server group myGroup v3 auth read myView

snmp-server user myUser myGroup v3 auth md5 Password12345

snmp-server view myView ip included

control-plane

!

!

line con 0

line aux 0

line vty 0 4


31/33

password password

login

!

scheduler allocate 20000 1000

!

end


32/33

Appendix E snmp.conf

defContext none

defSecurityName TestUser

defAuthPassphrase Password12345

defVersion 3

defAuthType MD5

defSecurityLevel authNoPriv

Appendix F: Sample output from scripts

-------------------------------Report for Device: 192.168.1.1Interface number: 2-------------------------------

Total Bytes In: 9807533Avg. Bytes In: 16345 bytes/sec.

Total Bytes Out: 407127910Avg. Bytes Out: 678546 bytes/sec.

Total Unicast Packets In: 151431Avg. Unicast Packets In: 252 pkts/sec.

Total Unicast Packets Out: 283820Avg. Unicast Packets Out: 473 pkts/sec.

Total Broadcast Packets In: 67Avg. Broadcast Packets In: 0 pkts/sec.

Total Broadcast Packets Out: 32Avg. Broadcast Packets Out: 0 pkts/sec.

Total Discarded Packets In: 0Avg. Discarded Packets In: 0 pkts/sec.

Total Discarded Packets Out: 0Avg. Discarded Packets Out: 0 pkts/sec.

Total Errors In: 0Avg. Errors In: 0 pkts/sec.

Total Errors Out: 0Avg. Errors Out: 0 pkts/sec.

Interface 2 speed: 100000000Interface Avg. Utilization: 0 %------------------------------


33/33

End of Report------------------------------

Network Management Final

Documents