Dynamic Routing Control in Heterogeneous Tactical Networks with Multiple Traffic Priorities

Dynamic Routing Control in Heterogeneous Tactical Networks with Multiple Traffic Priorities

Mariusz A. Fecko, Larry Wong, Jaewon Kang, Andrzej Cichocki, Vikram Kaul, Sunil

Samtani

Applied Communication Sciences

1 Telcordia Drive, Piscataway, NJ 08854

ABSTRACT † To efficiently use alternate paths during periods of congestion, we have devised prioritized Dynamic Routing Control Agent (pDRCA) that (1) selects best links to meet the bandwidth and delay requirements of traffic, (2) provides load-balancing and traffic prioritization when multiple topologies are available, and (3) handles changes in link quality and traffic demand, and link outages. pDRCA provides multiplatform load balancing to maximize SATCOM (both P2P and multi-point) and airborne links utilization. It influences link selection by configuring the cost metrics on a router’s interface, which does not require any changes to the routing protocol itself. It supports service differentiation of multiple traffic priorities by providing more network resources to the highest priority flows. pDRCA does so by solving an optimization problem to find optimal links weights that increase throughput and decrease E2E delay; avoid congested, low quality, and long delay links; and exploit path diversity in the network. These optimal link weights are sent to the local agents to be configured on individual routers per traffic priority. The pDRCA optimization algorithm has been proven effective in improving application performance. We created a variety of different test scenarios by varying traffic profile and link behavior (stable links, varying capacity, and link outages). In the scenarios where high priority traffic experienced significant loss without pDRCA, the average loss was reduced from 49.5% to 13% and in some cases dropped to 0%. Currently, pDRCA is integrated with an open-source software router and priority queues on Linux as a component of Open Tactical Router (OTR), which is being developed by ONR DTCN program.

1. INTRODUCTION Routing protocols such as OSPF that typically use a single metric to determine shortest paths work well for wired static networks;

however, this behavior is non-optimal for heterogeneous tactical wireless networks. Single metric does not take into account the utilization of the link; as a result, the routing protocol is unable to use alternate paths during periods of congestion. To efficiently utilize network resources, we have devised prioritized Dynamic Routing Control Agent (pDRCA) that selects best links to meet the bandwidth and delay requirements of traffic. The pDRCA heuristic takes as input the traffic loading, link capacities, network topology, and link propagation delay, and computes a set of OSPF link weights that optimize the network performance while satisfying the traffic prioritization requirements.

The primary method pDRCA uses to influence link selection is by configuring the cost metrics on a router’s interface. This mechanism does not require any changes to the routing protocol itself. pDRCA supports service differentiation of multiple traffic priorities by enabling the best service for the highest priority flows. It solves an optimization problem to find the “best” set of links weights. Since both the throughput and delay can be improved by minimizing link loading across all the links, a useful objective is to solve for a set of link costs that minimize the maximum utilization across all links in the network. We use local search heuristics to determine a feasible solution. Each iteration evaluates a candidate solution (i.e., a link costs assignment) and sets the stage for exploring a neighborhood of solutions by changing one or a few link costs.

Currently, pDRCA is integrated with an open-source software router (Quagga OSPFv3 routing suite [7] with Multi-Topology Routing (MTR)) and priority queuing implementations atop Open Tactical Router (OTR). OTR is a Linux-based software platform under development in the ONR Dynamic Tactical Communications Networks (DTCN) program. pDRCA provides a critical OTR capability of multiplatform load balancing to maximize SATCOM (both P2P and multi-point) and airborne links utilization. It has the following unique features:

• Autonomous operations in multi-tier, heterogeneous networks with links of different capacity and delay characteristics; • Discovery of dynamic topologies, plug-and-play interfaces, link capacities, and traffic demands; • Adjustment of routes in response to changes in message-level or application-level priorities;

† This material is based upon work supported by U.S. Navy ONR under Dynamic Tactical Communications Networks (DTCN) program, Contract No. N00173-09-C-4014.

• Formal route-cost derivation consistent with strict priority queuing discipline; • Meaningful route-cost comparison between point-to-point and multiple access media; • Seamless integration with multi-topology routing (MTR) and radio–router standards such as PPPoE. The rest of the paper is organized as follows: Section 2 discusses related work. Section 3 describes the objective functions used in

the link-weight optimization. The pDRCA architecture and implementation are described in Section 4, while Section 5 presents the experimental results. Finally, we conclude and discuss next steps in Section 6.

2. RELATED WORK Ref. [1] shows that the optimum link-weight selection is NP-hard. It also provides worst-case results comparing the performance

of OSPF routing vs. optimal multi-commodity flow routing. A local search heuristic is suggested that approaches in performance the results achieved by the general optimum (OPT) routing. The optimization algorithm in pDRCA is based on this original heuristic; however, we enhanced the solution to optimize aggregate delay, exploit more accurate queuing models, as well as use more efficient search techniques and iterative shortest path algorithm [3] to expedite computation. The problem was further studied in Ref. [2], which suggested that the slopes of the link-cost convex function can be adjusted to obtain better weights, for example to reduce the number of equal-cost multi-paths (ECMP). This achieves single shortest paths for different demand pairs, where possible, without unduly increasing the cost. Recently, the relation between the optimal link weights for shortest path routing and the dual solution associated with the “capacity” constraints of the multi-commodity flow problem gained attention in the context of OSPF routing [4][5].

3. PDRCA OBJECTIVE FUNCTIONS pDRCA chooses a set of link weights with the network-wide objective of minimizing one of two objective functions: (1)

throughput and (2) delay/throughput. We use local search heuristics to determine a feasible solution, where each candidate solution is evaluated using an objective function that quantifies the “goodness” of a solution.

Per pDRCA configuration, one of the two objective functions is applied iteratively for each traffic priority, starting from the highest one. Consistent with strict priority queuing, for every topology pDRCA considers only residual capacity “left” by the aggregated volume of higher priority traffic. The residual capacity available for traffic priority k on link i with the capacity �� is derived as ��� = �� − ∑ ����� , where ��� is the offered load for priority j on link i. The value of ��� is a function of the current link-weights evaluated by pDRCA as a candidate solution for traffic priority j.

3.1 Throughput Objective Function Throughput objective function is typically used for delay-tolerant TCP traffic. It is a convex function that penalizes solutions that

contain heavily loaded links and does not consider link propagation delay. Ref. [1] provides the rationale for the particular step-wise shape of , whose slope grows rapidly with increasing values of link

(over)utilization ρ = l/c (Figure 1). The intended link-cost is closely related to the probability of packet being lost. In general, approximates retransmission delays caused by packet losses (Figure 1), with the derivative corresponding to the multipliers of ρ. (Other definitions of are likely to work well in practice as long as it is a piece-wise linear increasing and convex function.)

The cost of sending traffic increases progressively as the utilization ρ approaches 1 and explodes for ρ>11/10. The rapidly increasing cost above ρ=1 indicates that as shown above will typically minimize the maximum utilization on any given link in the network. This objective is consistent with trying to decrease the likelihood of packet loss in the network, increasing throughput, and thereby being able to support a larger amount of traffic with acceptable QoS in the network.

Function can also be compared with the M/M/1 queuing delay [1][2]. It approximates this delay closely except that it is designed to mathematically handle overloaded links, i.e., the value of is prevented from growing to infinity when the link utilization reaches 100% to minimize the number of overloaded links and the degree of link overload in pDRCA solutions.

3.2 Delay/Throughput Objective Function Delay/Throughput objective function (Table 1) is typically used for

delay sensitive traffic such as UDP or low-bandwidth TCP such as Chat. When links with long propagation delays (e.g., satellite links) are present in the network, failure to consider the impact of the propagation delay could result in long end-to-end delays. To address this issue, this objective function considers both throughput and link delays. Since both these metrics can be improved by minimizing link loading across all the links, pDRCA searches for a set of link costs that minimize the maximum utilization across all links in the network. This function balances propagation/transmission delays with link loading by splitting traffic evenly to reduce the queuing

Figure 1: pDRCA throughput objective function

0

10

20

30

40

50

60

70

0 0.5 1 1.5

f

ρρρρ=l/c

Convex function

Value of f

delay (approximated by M/G/1 model) of each link. It computes the total delay incurred by a packet traversing a link and normalizes the delay using the ratio of the link traffic to total network traffic.

Function 1 in Table 1 computes the total packet delay that is a sum of propagation, transmission, and queuing delays. Additionally, the term l ik/O assigns higher cost to links carrying a large amount of traffic compared to links carrying less traffic. The value O is used to normalize the traffic value.

Propagation delay is the time that signals take to travel from one radio to another. For example, for ground to ground HNW links the delay value can be set to a few dozen milliseconds, alternatively for SATCOM links the delay can be set to 500 milliseconds. Transmission delay is the time it takes for the packet to leave the radio interface. This delay is computed as the packet length divided by the link transmission rate. Queuing delay specifies the amount of time that a packet waits in the queue before being transmitted. It is a function of link utilization (i.e., ratio of link traffic to link capacity). The above delay components are derived as follows:

Table 1: pDRCA delay/throughput objective function

MIN ( )∑∈∀ Ei

iikik Odclf ,,,

l ik – offered load on link i for priority k cik – capacity of link i for priority k

di – propagation delay experienced on link i O – total offered load in the network

ρ = l ik /cik – (over)utilization of link i for priority k

( ) ( )

<+=

=ikikikiik

ikiikik lOclfdOl

lOdclf

0),,1(*)/(

00,,,

<++−<=<++−<=<++−<=<+−

<=<+

=

ρρρρρρρ

ρρρβ α

10.......................................................35.5)95.0(1)1.110(400

101.1.................................................35.5)95.0(1)1.1(400

1.11................................................).........95.0(135.0)1(50

195.0.......................................................).........95.0(1)95.0(7

95.00.../8*866/))/64*47862(*))-(1*/(2 8/c*866*

1

2i

mfm

mfm

fmm

fm

cc

f

ii

where, m = 100 * (f1(0.95) – f1(0.94)), and α and β are tunable parameters set to 0.8 and 1, respectively

3.2.1 Propagation Delay The value is set statically based on the link type.

3.2.2 Transmission Delay It is calculated as � × 866 ∗ 8 �� , where we use the average packet size of 866 bytes. This value was based on measurements from

the simulation using the WIN-T IERs. Since there might be some variation in the processing of the packets, we introduce a tunable parameter � which can be used to tweak the delay. Currently we set the value of � = 1.

Note: The packet size of 866 is multiplied by 8 to get the value in bits, since the unit of capacity � is bits/second.

3.2.3 Queuing Delay We use M/G/1 queue to model the queuing delay, since a closed form solution exists for this model. The mean queuing delay for

M/G/1 is � 2(1 − �)� × ����� , where

• � is the utilization (link traffic/link capacity). • �� is the first moment (mean) of transmission delay • �� is the second moment of the transmission delay: (Mean2 + Variance) or (Mean + Std Deviation2)

The values of variance and mean were derived using simulations using WIN-T IERs. In the simulations the mean and the variance of packet size were computed as 866 bytes and 478628 (bytes)2, respectively. Substituting these values in the above equation we get the following formula:

Queuing Delay =

×

×+×

×−

c

cc8866

886664478628

)1(2

2

2

αρρ

M/G/1 models assume Poisson arrival; however the traffic is likely to be more bursty, leading to a larger queuing delay. To account for this variability we introduce a tuning parameter �: 0 < � ≤ 1. Since � ≤ 1, the value (1 − �)� is smaller than (1 − �), making the value of queuing delay larger.

3.2.4 Breaking down Queuing Delay into Stepwise Function In the function for Queuing Delay, the delay approaches ∞ as � approaches 1. To avoid this problem, we use function f to

compute values for 0 < � < 0.95. For values of � ≥ 0.95, we extrapolate the values using a piecewise linear function. The extrapolated equations are obtained as follows. We take the slope m of with � = 0.95. For values of � >0.95 we create

straight lines using a multiple of m; the values used for multiples of m are derived experimentally. To detect and penalize severely utilized links for values of � >10, we cap � = 10 in the input to f. This is done so that a single link would not overwhelm the values

computed for other links. This situation will occur more often in the case of multiple priorities because, as pDRCA performs the analysis for consecutive priority values, the variable representing residual link capacity is likely to become very small or even negative. In the latter case, the variable value is set to a small positive number, say 1kbps, before the utilization is calculated.

3.3 Accounting for Link Quality In the case where link bandwidth does not change in response to link

quality degradation, pDRCA can use link quality reports to consider link preferences. It computes the total network-wide cost as ∑ #��� , where #� = $%&' + 1 − $� and $� are the cost multiplier and abstract priority for link i, respectively. This mechanism can be used flexibly to (1) reduce the cost of links of advantaged nodes and (2) increase the cost of poor quality links.

For example, pDRCA accounts for link quality by considering Packet Delivery Ratio (PDR). Given the PDR value (100 – no loss, $� = 10; 0 – 100% loss, $� = 1), the value of #� is shown in Figure 2. The $%&' is calculated by iterating over all links and choosing the largest value of $�.

4. PDRCA ARCHITECTURE pDRCA is currently implemented in a semi-centralized architecture. As shown in Figure 3, the Lead pDRCA is activated at a

single network node. It solves a centralized optimization problem to derive a network-wide set of OSPF weights that induce a near-optimal set of routing paths for each traffic priority. A Local pDRCA configured in each node is responsible for providing the required inputs to the Lead pDRCA. It also receives and configures new link weights from the Lead pDRCA. The current implementation assumes that a single instance of Lead DRCA will be used in each OSPF Area to optimize the routing within the area.

4.1 pDRCA Process Overview As shown in Figure 3, Lead DRCA periodically analyzes the network topology, traffic statistics, and link capacities to see if there

are any problems with the network. These could mean congestion, excessive delays for some flows, or any other criterion deemed detrimental to the network. If no problem is detected, no further analysis needs to be done. Otherwise, Lead pDRCA re-computes link weights to amend the problem. The new link weights will induce more optimal traffic distribution such that throughput is increased and E2E delay is decreased; congested, low quality, and long delay links are avoided; and path diversity is exploited.

As a result of the analysis, if the Lead pDRCA is able to compute a “better” set of link weights, then these link weights are sent to Local pDRCA on each router to be eventually configured on the router per MT ID (i.e., traffic priority). If Lead pDRCA is unable to compute a “better” set of link weights, then no action is taken. The new link weights will effect optimal traffic distribution for the current topology, link characteristics, and the traffic being injected to the network.

Figure 3: pDRCA execution flow

4.2 Lead pDRCA Inputs This section defines the dynamic inputs required by Lead pDRCA, and the mechanisms to get the information: Topology Information: Lead pDRCA needs the current topology of the network. The topology information is already

disseminated through OSPF Link State Advertisements. The Lead pDRCA reads the topology from the local text file created by a

Local DRCA(remote)

Lead DRCA

Link capacity and qualityLink propagation delayTraffic statistics per DSCP

Optimal link weights per MTR (traffic priority)

Local DRCA(remote)

Traffic measurement

Effect optimal traffic distributionfor current topology and link characteristics

Local DRCA(collocated)

Topology (from OSPF LSDB)

Dynamic input files(aggregated per traffic priority)

Tactical CT Core

Node B

Node C

Node ANode A

Figure 2: Translating from PDR to link-cost multipl ier

dedicated utility that retrieves this information from the LSDB. The topology can be retrieved for both OSPFv2 and OSPFv3. A separate topology file is created for each priority value (i.e., MT ID).

Link Characteristics: Link characteristics such as capacity, relative quality, and propagation delay are reported periodically by Local pDRCA instances to the Local pDRCA collocated with the Lead pDRCA. In the current implementation, each Local pDRCA can retrieve the link capacity and relative quality information dynamically either from EMANE [8] or from a radio proxy via the PPPoE (RFC4938) interface. In such a case, the default values defined in a local configuration file will be replaced with the dynamic ones in the link characteristics reports.

Traffic Demand: The traffic demand is the aggregate traffic flowing between each source and destination router. This information is collected locally on each router by a special-purpose netStats utility. The utility measures traffic between source and destination host machines. It uses libpcap, which is a portable C/C++ library for network traffic capture widely used in several tools (e.g., Wireshark, tcpdump).

Each Local pDRCA reads the information provided by netStats and smoothes it by applying the averaging or the maximum operation on a number of samples. It then sends periodic reports of measured traffic per DSCP value to the Local pDRCA collocated with the Lead pDRCA.

5. EXPERIMENTAL RESULTS An early pDRCA version, which was able to handle only a single traffic-priority, was tested extensively using PM WIN-T

supplied 15-node OPNET models [6]. The optimization algorithm ran every 30 seconds. WIN-T Traffic types were used: TCP (Data, Imagery), and UDP (Voice, Unidirectional Video, and Bi-directional Video). We randomly selected 32 pairs of traffic source and destination nodes interconnected with ground to ground, ground to air, and SATCOM links. Link capacity was varied with the links in degraded states for about 25% of the time. The average results were as follows: TCP throughput increased by 84%; UDP delay and jitter decreased by 43–45%, and loss decreased by 14% [6].

The latest pDRCA version shown in Figure 3 was tested in ACS DTCN testbeds using network emulators such as EMANE [8] and CORE [9]. Both testbeds contain the implementation of several QoS/routing technologies interworking with pDRCA (e.g., Linux Traffic Control (TC) strict priority queuing, Quagga OSPF routing suite).

5.1 EMANE Testbed – Traffic Overload and Link Outage The objective was to exercise and validate pDRCA logic with respect to multiple traffic priorities in response to traffic overload,

link outage, and link quality degradation. We used EMANE to induce mobility, link outages, and SNR changes.

Table 2: Traffic in multiple-priority experiments

Initial experiments used non-streaming CBR UDP traffic representing a total of 46 flows with randomly selected source-destination pairs and mapped into two priority classes, where the ratio of high priority (HP) to low priority (LP) was 0.25. In a highly loaded 7-node network (125Kbps per flow), the overall packet loss dropped from 15% to 8%. All HP traffic was carried by pDRCA with the packet loss reduced by 80%.

Subsequent experiments used a combination of UDP and TCP traffic mapped into five priorities. All link capacities were set to 512 kbps with 40 msec one-way delays. Traffic flows shown in Table 2 were generated using ixChariotTM from Ixia, which is a test tool for simulating real-world applications and assessing network performance. Each experiment ran for about eight minutes.

Per static configuration, we map one or more DSCP values to the same priority (abstract integer value), which is in turn mapped to a single MT ID. Initially, all link-weights are equal and do not differ across multiple topologies. As shown in Figure 4, in the absence of pDRCA, all the traffic priorities follow the single-hop path from the source (node 1) to the destination (node 4). Since strict priority queues are configured on all interfaces, almost the entire path capacity is used by the highest priority flows – other priorities are able to achieve only small throughput despite the existence of several alternative paths in the network.

Once enabled, pDRCA shifts flows to adjust to link capacities, traffic volume, and traffic priorities. Figure 5 shows the final network configuration after pDRCA actions in response to traffic overload. Spare capacity in diverse paths from node 1 to node 4 is optimally utilized – pDRCA separates top three priorities (0, 1, 2) into three equal-capacity

Traffic Priority

Number and Type of Flows

Offered Load [kbps]

Throughput w/o

pDRCA w/

pDRCA 0 3 audio (UDP) 450 450 450 1 3 video (UDP) 450 46 410 2 39 Chat (TCP) 195–468 27 190 3 1 FTP (TCP) 300 25 270 4 (lowest) 1 FTP (TCP) 300 28 45

Figure 4: Inefficient use of diverse paths by OSPF

1

2

3

4

5

6

7

0

12

3

4

paths so that they would not compete for resources. Throughput is significantly improved across all traffic classes. Lower throughput for the Chat flows, which are of higher priority than FTP flows, is due to some peculiarities of ixChariot Chat scripts – flow rates are sometimes sticky and cannot utilize all of the available capacity.

By analyzing pDRCA actions step-by-step, we concluded that it makes the best flow-rerouting decisions, i.e., pDRCA is optimal in its decisions given the inputs. Ideally, pDRCA should consider the offered load, but TCP dynamics and rate adaptation make it difficult to relate it to the measured traffic. Since pDRCA considers the measured TCP flow rates, they are sometimes lower than the offered load would be without competing flows. Using the maximum rather than average of traffic samples alleviated this problem to a large extent.

Figure 5: pDRCA response to traffic overload Figure 6: pDRCA response to link outage

Given the network state of Figure 5, Figure 6 shows the result of link outage. OSPF reacts to this event by moving the highest priority traffic to path 1→3→6→4. This is clearly suboptimal because the two highest priority traffic classes are now routed through this path; as a result, the throughput of priority 1 is lower than that of priority 2.

pDRCA corrects this default (suboptimal) response provided by OSPF by separating top two priorities (0, 1) into two diverse equal-capacity paths. That is,

• priority 1 flows are moved to path 1→2→7→5→4 to move them away from priority 0 traffic; • priority 2 flows are moved to path 1→3→6→4 because it has the same residual capacity as path 1→2→7→5→4, but fewer

hops. In another experiment, we tested pDRCA response to link

quality degradation. Starting from the state in Figure 5, we decreased Packet Delivery Ratio (PDR) to 82% on the link from node 1 to node 4 through mobility emulation. EMANE uses SNR level to determine whether to probabilistically drop packets. We retrieved packet drop probabilities from EMANE and averaged them into PDR values. As shown in Figure 2, PDR=82% corresponds to the link-cost multiplier 6.2. This increase in cost is sufficient for pDRCA to move priority-0 traffic to path 1→3→6→4 and priority-1 flows to path 1→2→7→5→4. Consider priority-0 traffic (blue line in Figure 7), where packet drops reach 18% at 6:00 time. Thanks to pDRCA rerouting actions, the traffic recovers from losses and continues at original rate at 7:00 time.

Across the above scenarios, we noticed that • pDRCA consistently offers additional protection to high priority traffic by correctly separating higher priority flows if diverse

paths are available. • Flows may be rerouted multiple times, but the highest priority traffic experiences the least disturbance. • When configured to analyze the network every 30 seconds, DRCA is able to respond to link outages in 20–30 seconds and to

link degradation in 1 minute. • pDRCA objective functions are flexible enough to be extended with link quality metrics (e.g., PDR, BER, SNR).

5.2 CORE Testbed – Link Bandwidth Change The objective of this effort was to test pDRCA ability to rebalance traffic in the network in response to a link-bandwidth change

event. The network is emulated using CORE running under Xen PVM, with OPNET hooked up as Software-In-The-Loop (SITL).

1

2

3

4

5

6

7

1

32

0

4

450 kbps

410 kbps

190 kbps

270 kbps

45 kbps

1

2

3

4

5

6

71

0

4

2

3

Figure 7: pDRCA response to link degradation

Figure 8: pDRCA adapts routing on parallel links Figure 9: Capacity changes due to fading of one satellite link

A key element relevant to a real-world Navy scenario is the presence of multiple parallel satellite paths out of a ship leading to shore-based routers (Figure 8). A PPPoE OPNET SATCOM model from MITRE is subject to link fading, which causes the modem to change its coding rate and hence link bandwidth in discrete steps as shown in Figure 9. The PPPoE interface allows the modem to dynamically report bandwidth and link quality changes and reduced flow-control credit upon link degradation. The Local pDRCA has been extended to receive these reports through a netlink interface and send them to Lead pDRCA, which in turn updates the link capacity parameter in the objective function. Reduced link capacity causes the value of the link cost to increase rapidly as per Figure 1 and Table 1.

We created a variety of different scenarios by varying traffic profile and link behavior (stable links, varying capacity, and link outages). In the scenarios where high priority traffic experienced significant loss without pDRCA, the average loss was reduced from 49.5% to 13% and is some cases dropped to 0%.

Figure 10: Traffic profile for "Flat" scenario

Figure 11: Traffic profile for "Declining" scenario

We provide more details on the three scenarios with one of the satellite links degrading over time. The scenarios differed in the level and distribution of the offered traffic. The “Flat” scenario used the same, and constant, amount of data for each traffic priority. The “Declining” scenario had traffic level decreasing as the priority rises, and finally, the “Preemptive” scenario had the traffic similar to the “Declining” one, but with constant low-priority (background) traffic, and bursty high-priority traffic. The traffic profiles for

OTR (pDRCA)

Ship

Traffic sources

OPNETSATCOM modem

Links to shoreOTR router

0.00%

20.00%

40.00%

60.00%

80.00%

100.00%

120.00%

0 100 200 300 400 500 600 700 800

EBEM % peak rateSATCOM %peak rate

these scenarios are presented in Figure 10, Figure 11 and Figure 12. The horizontal axis on these figures denotes time, the vertical axis show the number of packets transmitted per time unit. The traffic volume is proportional to the size of the colored bar, and the color correspond to priority, with orange being the highest priority, and ping the lowest. Note that the traffic of different priorities is sent to different destination ports.

Figure 12: Traffic profile for "Preemptive" scenari o

The results of these experiments confirm that pDRCA is able to handle the change of the capacity of the link. The total number of bytes sent from a node representing a ship to the shore node (in “Flat” traffic scenario) is shown in Figure 13.

Figure 13: Total number of bytes without (left) and with (right) pDRCA, per priority level, in “Flat” scenario

It is immediately visible that without pDRCA only a small percentage of the traffic sent (depicted pink) is actually received (received data is depicted blue), for the average loss of about 85%. However, with pDRCA managing the routing, traffic of the two highest priorities (0 and 1) was received with only transient losses, and the percentage of losses for lower traffic priorities was reduced to the average of about 60%.

A more detailed analysis of the traffic on the parallel satellite links reveals the reason for such improvement in the system performance. The byte count on the two links when the system is not running pDRCA, as a function of time, is shown in Figure 14. On both diagrams, the green line represents the capacity of the link: one link is fading, while the other’s capacity is constant.

The different priorities of traffic are depicted with different color (from blue, the highest priority, to red, the lowest). Traffic that is actually received is shown above the horizontal axis, traffic lost appears below. Traditional routing is trying to send all the traffic through one link, leaving the other significantly underutilized, and leading to a huge loss on the overloaded one.

In contrast, pDRCA can dynamically reconfigure the routers in response to worsening capacity of one link. Moreover, pDRCA utilizes Multi-Topology Routing (MTR), which enables sending traffic of different priorities through different paths. The byte count of traffic on the same two links, when pDRCA is utilized, is depicted in Figure 15.

Figure 14: Traffic on parallel satellite links without pDRCA, in “Flat” scenario

Figure 15: Traffic on parallel satellite links with pDRCA, in “Flat” scenario

We can see that the high priority traffic is sent through the fading link for as long as possible. When the capacity of that link fades below the demand, the traffic is rerouted to the other link. When there is already lower priority traffic on that link, it gets preempted.

The “Declining” scenario produces very similar results. By utilizing the parallel link, pDRCA reduces losses for highest priority traffic from 15% to 0%, and for the rest of the traffic, from 87% to 50%. For brevity, we omit the detailed graphs for this scenario.

The “Preemptive” scenario shows significant improvement as well. The traffic sent and received without and with pDRCA is shown in Figure 16.

Figure 16: Total number of bytes without (left) and with (right) pDRCA, per priority level, in “Preemp tive” scenario

As in other scenarios, the improvement is immediately visible. However, even without pDRCA the loss for the two highest priorities is much smaller than in other scenarios (about 20%), and the improvement is less dramatic (loss is reduced to about 7%). This behavior is due to the fact that the bursty high priority traffic persists only for periods of time, which limits the opportunity window for pDRCA actions, i.e., detecting congestion and reacting by changing the routing paths.

The detailed traffic graphs for the parallel satellite links (without pDRCA in Figure 17, and with pDRCA in Figure 18) seem to confirm this hypothesis. As in the previous case, the line on top of the graphs denotes current capacity of the link. The first high priority burst occurred when there was still enough bandwidth on the original link to accommodate it, and thus the average losses for the high priority traffic were smaller. As in previous scenarios, the lower priority traffic had losses approaching 100%. When pDRCA was active, the delays in processing the network information (that is, the link capacity and the load) are visible in the somewhat irregular shape of the traffic graphs. Nevertheless, pDRCA actions enabled more balanced traffic split between the two parallel links, and brought about overall improvements in the performance of the system, both for the high priority traffic (losses reduced, in average, from 20% to 7%) and the lower priority traffic (with losses reduced from about 85% to about 55%).

Figure 17: Traffic on parallel satellite links without pDRCA, in “Preemptive” scenario

Figure 18: Traffic on parallel satellite links with pDRCA, in “Preemptive” scenario

6. CONCLUSION prioritized Dynamic Routing Control Agent (pDRCA) selects best links to meet the bandwidth and delay requirements of traffic

and provides load-balancing and traffic prioritization when multiple topologies are available. The primary method pDRCA uses to influence link selection is by configuring the optimal cost metrics on a router’s interface. The pDRCA optimization algorithm has been proven effective in improving application performance. We created a variety of different test scenarios by varying traffic profile and link behavior (stable links, varying capacity, and link outages). In the scenarios where high priority traffic experienced significant loss without pDRCA, the average loss was reduced from 49.5% to 13% and in some cases dropped to 0%.

Current work focuses on more efficient signaling (link-weight dissemination through opaque LSAs, an alternate protocol for reporting link statistics), and enhancement for multi-point SATCOM links. Since the lead pDRCA is at present configured statically in the network, we will also provide mechanisms for dynamic pDRCA election and failover.

7. ACKNOWLEDGMENTS We would like to thank Dr. Richard Edell of MITRE Corp. for valuable comments on this paper and technical feedback on the

pDRCA technology.

8. REFERENCES [1] B. Fortz, M. Thorup. Increasing Internet Capacity Using Local Search. Comput. Optimizations and Algorithms 29(1), 2004. [2] S. Srivastava, G. Agrawal, D. Medhi. Dual-Based Link Weight Determination Towards Single Shortest Path Solutions for OSPF Networks. In

Proc. International Teletraffic Congress, 2005. [3] G. Ramalingam and T. Reps. An incremental algorithm for a generalization of the shortest-path problem. J. Algorithms 21(2):267–305, 1996. [4] M. Pioro, A. Szentesi, J. Harmatos, A. Juttner, P. Gajowniczek, and S. Kozdrowski. On OSPF related network optimization problems.

Performance Evaluation 48:201–223, 2002. [5] Y. Wang, Z. Wang, and L. Zhang. Internet traffic engineering without full mesh overlaying. In Proc. IEEE INFOCOM, pp. 565–571, New

York, USA, 2001. [6] S. Khurana, G.T. Kim, M. Elaoud, S. Samtani, A. Staikos. Performance Evaluation of a Unicast Routing Control Agent for Proactive Diverse

Link Selection. In Proc. IEEE MILCOM, 2006. [7] Quagga Routing Suite. http://www.quagga.net [8] Extendable Mobile Ad-hoc Network Emulator (EMANE). http://labs.cengen.com/emane/ [9] Naval Research Laboratory. Common Open Research Emulator (CORE). http://cs.itd.nrl.navy.mil/work/core/index.php .