1 Achieving Correct Hop-by-Hop Forwarding on Multiple ... · Abstract—Algebra has made the tasks of modelling routing algorithms and proving their correct operation very clear.

1

Achieving Correct Hop-by-Hop Forwarding onMultiple Policy-based Routing Paths

Pedro Amaral, Member, IEEE, Paulo Pinto, Member, IEEE, and Luis Bernardo, Member, IEEE

Abstract—Algebra has made the tasks of modelling routing algorithms and proving their correct operation very clear. However,traditional models are only concerned with the convergence to a stable routing solution not covering distributed hop-by-hop forwardingon multiple paths (within a flow or separately per flow). This paper addresses this challenge. Currently, loop-free distributed multipathforwarding is only proven if a strict decrease in the preference of a path occurs with every added link. This is very rigid for multipathrouting, limiting the amount of equal preferred paths. In this work we prove that correct distributed forwarding behaviour is possible,without the strict decrease in preference, if the possible pairs of policy values applied to the directed edges connecting two nodes obeyto a particular condition. If this is not true, correct behaviour is still possible if the policies applied to links forming circuits in the networkgraph are constrained according to two other conditions. Modern technologies are bringing the possibility to build new routing protocolsusing network programmability while distributed operation is still needed in many scenarios due to scaling issues. We show how ourresults can be relevant in this setting and serve as a tool in the design process and/or for formal protocol verification.

Index Terms—multipath routing, algebraic routing models, policy routing, SDN, distributed control

F

1 INTRODUCTION

Classical routing theory [1]–[7] uses graph theory to modelnetwork topology, and algebraic structures to model boththe calculation of path weights (as a composition of its linksweights) and the selection of the best path (based on anorder on the weights). This theory as been used to modelboth policy based and weight based shortest path routingprotocols.

Policy-based routing provides a means to obtain, vali-date, and select paths based on a set of link attributes thatare semantically richer than the traditional metrics. These at-tributes are somewhat abstract in nature and reflect a givenintention towards a link or a path, expressing how that linkor path should be considered in the routing decision. Inpolicy based routing protocols (e.g. BGP) routers apply apolicy p associated with a neighbor to an incoming route r,something that can be modelled with an algebraic operation⊗, resulting in a new candidate route with policy value p⊗r.They then compare the policy values of all available routesto determine the best routes. Traditional routing metricsare, by opposition, well defined characteristics of a link orpath like delay, loss, or bandwidth values that translate toa numeric path weight. In best path weight based routingprotocols (e.g. OSFP, RIP, IS-SI) routers add the weight w,associated with the link to a neighbor, to an incoming router resulting in a new candidate route with weight w+ r. Thesmallest weight route(s) are then chosen for forwarding.

If we look in to the classical theory, we see that weightbased protocols hold the following property, or something

P. Amaral, P. Pinto and Luis Bernardo are with the Dep.o de Eng.a Elec-trotecnica, Faculdade de Ciencias e Tecnologia, Universidade Nova de Lisboa,2829-516, Caparica Portugal and with Instituto de Telecomunicacoes, LisboaPortugal e-mail:[email protected] (P. Amaral), [email protected] (L. Bernardo),[email protected] (P. Pinto).This work was supported by FCT/MEC Project IT PEst-UID/EEA/50008/2013.

equivalent:

Definition 1. Consider S the set of all possible link or pathvalues (weights or policies), used in a routing protocol. Consider� the order of preference between the path values and ⊗ the pathcomposition operation.

∀a, b, c ∈ S a � b⇒ (c⊗ a) � (c⊗ b)

In this work we call this property monotony 1.When the model of a routing protocol holds this property

the classical theory proves that the routing protocol con-vergences to a global optimal set of routes and distributeddestination based forwarding operates correctly.

Policy based protocols however are protocols weremonotony does not hold. A simple example is encounteredin common BGP inter-domain routing polices that indi-cate that peer routes pe are preferred to provider routespr, meaning that pe � pr, and that peer routes are notannounced to providers, meaning that pr ⊗ pe = 0 (aninvalid route). This means that we have pe � pr but(pr ⊗ pr) � (pr ⊗ pe) since pr � 0.

Previous work on policy based routing models rangedfrom BGP specific models [8], that are not applicable to otherprotocols, to generalized routing algebras [3]–[7], [9] thatstudied the convergence of path computation algorithmsto stable routing solutions in the absence of the monotonyproperty. These later works provide a set of convergenceconditions that only require an increasing ⊗ operation (def-inition 2), or in alternative, an non-decreasing ⊗ operation(definition 3) together with some conditions on the policiesapplied to the specific topology.

Definition 2. An ⊗ operation is increasing if

∀a, b ∈ S a ≺ a⊗ b ∧ b ≺ a⊗ b

1. If instead of the preference notation a preference operation is usedthen the this property is called distributivity

2

Definition 3. A ⊗ operation is non decreasing if

∀a, b ∈ S a � a⊗ b ∧ b � a⊗ b

Considering multipath routing, the classical theoryworks that deal with it [4]–[7] do not address how forward-ing on the multiple paths can be done and do not proveunder which conditions it can be done without forwardingloops. They only provide proofs of convergence to a stableset of equally preferred or incomparable paths. Those proofsrequire an increasing ⊗ operation, or at least a non decreasingone, but in this case with restrictions either to the applicablepolicies on a specific topology or to the paths that can be inthe routing solution.

The goal of this work is to further enrich the modelsin the classical theory with aspects that are not covered byit and that we argue that are relevant for modern routingapplications. In concrete terms, we focus on policy basedmodels that only exhibit the non decreasing property definedin definition 3 and on the use of multiple equally preferredrouting paths for distributed destination-based forwarding.Our contributions to the theory are:

• We show that in the multipath routing case. The con-ditions in the previous theory that allow finite timeconvergence in non-decreasing models, namely thepolicy restrictions on the topology graph circuits, areinsufficient to assure loop free multipath forwarding.

• We define a sufficient (although not necessary) con-dition for correct operation, that assures both con-vergence and loop-free distributed multi-path for-warding. That condition uses the notion of label pair,that consists in the pairs of policy values [a− b] thatcan be applied to the two directed edges connectingneighbour nodes and is efficiently verifiable since itis built in the model (definition of the possible policypairs). Simple algorithms can be devised to build setsof policies that comply with the condition and/or toverify if an existing set of policies complies with thecondition.

• We formulate a necessary and sufficient condition onthe policies applied to the links of the circuits in thetopology that assures correct operation without anyrestrictions on the set of policies of a given routingalgorithm.

These contributions are relevant since they add to theexisting algebraic routing model theory and they increaseits generality making it applicable to a wider range ofpolicy based protocols. This generality is important sincetraditional inter-domain BGP (that was the drive of previousalgebraic routing theory) is no longer the only policy-richrouting scenario. SDN technologies like OpenFlow [10], BGPFlowSpec [11] or Path Computation Element (PCE) Commu-nication Protocol (PCEP) [12] bring the possibility to pro-gram routing using software allowing the implementationof new policy based routing protocols in several scenarios.

In this software based scenario we believe that the useof distributed destination-based forwarding is of particularinterest since SDN technologies can suffer from scalingimplementation issues, when more granular forwarding isused, due to the cost of Hardware forwarding rule space[13]. Another aspect is the use of multiple paths that brings

several known advantages in routing and policy routingprovides interesting features in this regard. Paths with thesame policy attributes can all be considered of equal pref-erence (we want to treat them in the same way) even ifthey have quite different metrics (like hops, bandwidth ordelay) opening the possibility to have more path diversity.Multipath policy based protocols can find applications insuch diverse scenarios as Internet routing [14] or layer 2fabrics in data center networks [15] where multipath (evenwithout formal guarantees of mutual exclusion of paths)alleviates several problems.

Our results can be used to verify the correct operationand/or to provide information for the design process ofpolicy-rich protocols implementable with several technolo-gies. This includes new protocols and/or existing ones (e.g.BGP implementations that allow Multiple equally preferredpaths to be installed in the routing table) where our resultscan be used to verify if a specific set of implemented routingpolicies (e.g. a particular set of inter-AS commercial relation-ships) are robust. This is something that come in line withthe increasing importance of formal verification of softwarefor critical infrastructures.

This work started with the results in [16]. The formalmodel is now completely generalized with the introductionof the concept of equivalence classes for path preferences,and a new result is derived and proved for assuring correctbehaviour for all possible networks. The notions of linearand closed circuits, directed cycles and label pairs wereneeded for the extension. Finally, application examples thatillustrate our results and are useful for the understanding ofthis type of work are added.

The paper has the following structure: section 2 brieflydescribes some routing algebra background for multipathrouting models; in section 3 we establish the necessaryconditions to obtain a stable set of routes in finite timeand discuss the synchronous or asynchronous nature ofthe path computation as well as its time bounds; and insection 4 we do the same for loop free forwarding. In section5 we show how the conditions in sections 3 and 4 canbe met, pointing the differences between them and howthe conditions in 4 are not covered in the ones previouslyknown. Section 6 illustrates the results by applying themto an example algebraic model for better understanding. Insection 7 we discuss the implications of our contributionsand the relation to other research in routing models. Section8 concludes the paper.

2 MULTIPATH POLICY ROUTING MODEL

The correctness of a policy based routing protocol is usuallyrelated to how a stable set of acyclic routes is calculated infinite time. Correct forwarding is not usually included inthose definitions. In this work we define correctness withboth aspects in mind. A routing protocol is correct if itcomplies with the following definition.

Definition 4. A routing protocol is correct if and only if

• It converges to a fixed routing solution, meaning that itcalculates a fixed set of routes in finite time.

• It forwards packets through any of the equally pre-ferred paths without causing loops (in our case, usingdestination-based hop-by-hop forwarding).

3

To understand under which conditions a routing proto-col operates correctly we need a routing model.

We have to model: the network, the set of link attributes;the process of obtaining paths by appending links; and theprocess of selecting the paths to be used.

We use an algebraic structure formed by a set S endowedwith a binary operation ⊗ and an order �:

(S,�,⊗)

The elements of S represent values that can be attributedto links as well as to entire paths. 0 models the weight of anon existent or invalid path and is a zero of the⊗ operation.1 models the trivial path, which is the path from one nodeto itself, containing no links and is a neutral element of the⊗ operation.

The network is represented by a weighted directed graphG(V,w), where V is a set of vertices and w : E → S is aset of edges connecting vertices in V and weighted withvalues of S. The weight of the edge connecting va and vbboth belonging to V is represented by w(va, vb) ∈ S.

The � order models the path choice by ordering thepossible path values in S. Since we want the possibility ofhaving equivalent or incomparable paths� is a preorder [1].

We want to be able to assign the values in S to links andcalculate paths with⊗ in as many different ways as possible.This means for example that we can accommodate routingmodels where the values of two edges/paths do not provideenough information on how to ⊗ add them. An examplewould be a routing algorithm where the total value of apath depends on the ordering of the edges [2]. Therefore,we do not impose associativity on the binary operation ⊗although, of course, it may be present. A designation for analgebraic structure composed by a set and a non necessarilyassociative binary operation is not consensual; in this workwe use the term magma to refer to such algebraic structure.Another naming aspect is the use of the term ordered toexpress the existence of an order in algebraic structures.This term is applied in associative algebraic structures calledsemi-groups when they are monotonic [1]. Since we do notrequire our structure to be monotonic nor the order to be acomplete order we use the term preordered instead of ordered.In conclusion, we call our non-associative, non-monotonicstructure (S,�,⊗) a preordered magma.

In a preordered magma with a preorder�we can divideS in equivalence classes. An equivalence class is a maximalset of elements with equivalent preference in �. Any twoequivalence classes are either equal or disjoint so an elementof S belongs to one and only one equivalence class.

The ⊗ operation models the calculation of paths. Itis applied between a link value and a path values, bothbelonging to S (starting with the trivial path) and resultsin another path value also from the set S. It expresses howthe attribute of a path changes with the addition of one link.

Since⊗ can be non-associative we consider two differentways (depending on the nested application of⊗) to calculatethe weight of a path P = v1, v2 · · · , vk, connecting verticesv1 to vk:

Starting from the destination vertex vk.

wl(P ) = (w(v1, v2)⊗ (· · ·⊗ (w(vk−2, vk−1)⊗w(vk−1, vk))))

Or starting from the source vertex v1:

wr(P ) = (((w(v1, v2)⊗ w(v2, v3))⊗ · · ·)⊗ w(vk−1, vk))

For all pairs (vi, vj) with vi, vj ∈ V the weights of allpaths between vi and vj are calculated, and the set σ(vi, vj)of the smallest weight paths is obtained using �. This resultis the routing solution. This routing solution is called theleft routing solution σl(vi, vj) if the used ⊗ nesting is fromdestination to source (resulting in wl weights), or the rightrouting solution σr(vi, vj), if ⊗ is applied from source todestination (resulting in wr weights). If⊗ is associative thenboth solutions are equal. It has been shown in previousworks that only the left routing solution is compatiblewith destination based, hop-by-hop forwarding [3]. For thisreason in this work we always perform the left weight (wl)calculations of path weights and always consider the leftrouting solution. In a preorder, the concept of minimal setsin S can model multipath routing [4], [5]. It represents aset of the smallest path’s weights (with highest preference),meaning that it cannot contain an element together with onefrom a strictly less preferred equivalence class. Both routingsolutions σl(vi, vj) and σr(vi, vj) are minimal sets.

The study of protocol correctness for both statementsof definition 4 in abstract terms is closely related with thepresence of circuits (according to the definitions below) inthe network.

Definition 5. A closed circuit, or simply a circuit, C is asequence of connected vertices such that only the first and thelast are the same:

C = 〈v0, v1, v2, ..., vk = v0〉

Another kind of circuit is important in the sequel: wecan define a circuit where more than one vertex is repeatedby following directed edges. There are two directed edgesconnecting the vertices (one in each direction). We call thesecircuits linear circuits:

Definition 6. A linear circuit, LC, is a sequence of connectedvertices where all nodes but one are traversed twice:

LC = 〈v0, v1, v2, ..., vk−1, vk, vk−1..., v2, v1, v0〉

Each time we refer circuit with no adjective we meanclosed circuit. Figure 1 serves as an example and containsseveral linear circuits.

V1V0

w(V0, V1)

w(V1, V0)

V2

w(V1, V2)

w(V2, V1)

Fig. 1. Three vertices forming linear circuits

The three connected nodes v0, v1, v2 form two linearcircuits spanning the three nodes: one starting and endingin v0, LC1 = 〈v0, v1, v2, v1, v0〉; and the other in v2, LC2 =〈v2, v1, v0, v1, v2〉. Note also in the figure that there are thefollowing two circuits involving each pair: C1 = 〈v0, v1, v0〉and C2 = 〈v1, v2, v1〉. These circuits are both closed circuitsand linear circuits.

To model a full-duplex bi-directional communicationlink connecting va to vb we need two weights in W ,w(va, vb) and w(vb, va) that represent the weight in each

4

direction. Since links in network are usually bi-directionalthis also means that the graph G is strongly connectedvia bi-directional edges. Lets consider a closed circuit withmore than two vertices that are formed exclusively bybi-directional communication links, or in other words ifthe sub-graph C ⊂ G that contains the vertices of theclosed circuit is symmetric (i.e ∀w(vi, vj) ∈ W there is alsow(vj , vi) ∈W for all vj , vi ∈ C) and has more than two ver-tices. In such closed circuits, a path 〈v0, v1, v2, ..., vk = v0〉that starts and ends in the same vertex has two differentcircuits according to the direction in which with travelin the path, C1 = 〈v0, v1, v2, ..., vk = v0〉 and C2 =〈vk, vk−1, ..., v1, v0 = vk〉 we call these different circuitsdirected cycles.

For linear circuits and closed circuits with only twovertices there is only one way to travel around the circuits.In linear circuits if we consider that there is only one bi-directional link between the vertices2 then there is only oneway to travel from vi, around the cycle, back to vi. The sameis valid for a closed circuit with only two vertices va, vb sincethere is also only one way to travel from va to vb and backto va.

Definition 7. Consider a node p in a closed circuit representedby a symmetric directed graph C ⊂ G with more than twovertices. It is possible to define directed cycles from p to itselfaround the circuit following a certain direction: The clockwise(right) direction is represented byCR

p ; The counter-clockwise (left)direction is CL

p .CR

p,q (and CLp,q) denotes the segment from a node q 6= p of the

directed cycle to p (thus not completing the directed cycle).

In the above notation we consider the vertex p as the des-tination of the path when calculating the circuits weights.This means that we always calculate the left weights wl ofthe circuits regardless of their direction. The direction of thecircuit defines the order in which we consider the edgeswhile the direction of the weight has to do with the nestingof the ⊗ operation. This is important since ⊗ is not assumedto be associative.

Definition 8. Consider all linear circuits starting and ending ina node p of a symmetric directed graph C ⊂ G. There is only onepossible direction to traverse those linear circuits and thereforeonly one cycle for each one that we denote by LCp,···,p.

Figure 2 illustrates a circuit formed by k − 1 nodes andone destination node, d, that might be reached (directly, ornot) by any of the k− 1 nodes of the circuit. Some examplesof left weight calculations for cycles are the following.

The weight of the directed cycle CRv2 , in the figure 2 is

given by:

wl(CRv2) = w(v2, v1)⊗ (w(v1, vk−1)

⊗(· · · ⊗ (w(vi+1, vi)⊗ (· · · ⊗ w(v3, v2)))))

The weight of the path corresponding to the cycle segmentfrom vi to v2 (CR

v2,vi ) is:

wl(CRv2,vi) = w(vi, vi−1)⊗ (· · · ⊗ w(v3, v2))

2. If more than one physical link exists a lower layer protocol such asLACP can make it appear as a single link for routing purposes

The weight of the directed cycle LCv2,vi,v2 corresponding tothe linear circuit from v2 to vi to v2 is:

wl(LCv2,vi,v2) = w(v2, v3)⊗ (· · · ⊗ (w(vi−1, vi)⊗

(w(vi, vi−1)⊗ (· · · ⊗ w(v3, v2)))))

V1

Vk-1V2

Vi+1

Vi

w(V1,V2)

d

w(Vi,Vi+1)

w(Vk-1,V1)

V3

w(V1,Vk-1)w(V2,V1)

w(Vi+1,V1)

w(V3,V2)

w(V2,V3)

Fig. 2. Generic cycle

With this model we are now able to address the protocolcorrectness problem, starting with the convergence to astable routing solution.

3 CONVERGENCE TO A STABLE ROUTING SOLU-TION

Both the left and right routing solutions can be obtained viamatrix iteration [1]:

σl : X 7→ (A⊗X) �min

I

σr : X 7→ (X ⊗A) �min

I

where �min

is the minimization in the preorder �, I is

the identity matrix and A the adjacency matrix representingthe directed graph. A contains in each position either theweight of the edge connecting to adjacent nodes or anempty value for the non-adjacent ones. The starting pointof the iteration is I and each iteration consists in a matrixmultiplication with A where ⊗ is the multiplication andthe minimization in � is the addition. This implies theexistence of an infima and suprema defined by � in set Sthat corresponds respectively to the 1, and 0 weights.

In each iteration step the weight of another edge is ⊗added to each element of the set of best weights followedby the weight minimization between all the results and theunion of the resulting best weights to the previous weightset of each position.

We can redefine σl and σr changing the multiplicationand addition in the matrix iteration so that the resultscontain paths instead of weights. For this we can use pathconcatenation as the multiplication (instead of ⊗) and anaddition that minimizes in �

minby calculating the left weight

(for σl) or the right weight (for σr) of the paths at eachiteration.

When we do this, the kth iteration of left routing so-lution, σk

l , contains the set of best paths between i and

5

j(according to the preorder), of length at most k calculatedfrom destination to source.

If the iteration converges to a fixed point, in at most ksteps, the limit of σl for k →∞ is σk

l = A∗ (closure matrix ofA) [1], [2], [5] and contains in each position the set of all thepaths between (i, j) whose weights belong to the minimalset. convergence must occur starting in any arbitrary stateof the iteration.

The iteration can be implemented in a centralized wayor in a distributed approach using algorithms like Bellman-Ford or Dijkstras [3]. In a distributed implementation thetiming of the exchanges of routing updates between nodesmust be considered. The simplest case is to consider syn-chronous updates exchanges between nodes, and that is thecase in most of the algebraic routing literature that eitherassumes central implementation or synchronous messageexchanges when proving convergence of σl for both thesingle and multipath solutions [6] [7] [5] [4]. Asynchronousconvergence considers delayed, reordered, duplicated andlost update messages in distributed implementations, and itis studied in [17].

All these works prove convergence using different for-mal tools but in all of them the proof is tied to theimpossibility of maintaining the weight of a path whenadding the weights of all links of a cycle. This includes theasynchronous convergence proof in [17], that is based ontwo conditions: a non decreasing ⊗ operation and a routingsolution were only simple paths (paths were no node isvisited twice) can exist (which is assured if the weightsaround circuits do not maintain preference).

In this work we formalize (in our proposed model) acondition, presented in its most general case, that assuresthis:

Consider any destination d; we can identify a node vd(might exist more than one) belonging to a circuit whicheither is a vertex connected to d by a path Pvd,d = 〈vd, · · · , d〉that exits the circuit immediately after vd (e.g. paths contain-ing the dotted lines in the figure 2) or is itself the destinationd.

The other vi vertices in the circuit have two paths foreach vd, and then to d, with weights wl(C

Rvd,vi

) ⊗ wl(Pvd,d)and wl(C

Lvd,vi

)⊗ wl(Pvd,d).

Condition 1. Consider a circuit (or linear circuit). For alldestinations d, and all vd’s vertices as defined above, and allvertices vi in the circuit:

wl(CRvd,vi

)⊗wl(Pvd,d) ≺ wl(CRvi)⊗ (wl(C

Rvd,vi

)⊗wl(Pvd,d))

∧

wl(CRvd,vi

)⊗wl(Pvd,d) ≺ wl(CLvi)⊗ (wl(C

Rvd,vi

)⊗wl(Pvd,d))

∧

wl(CLvd,vi

)⊗wl(Pvd,d) ≺ wl(CRvi)⊗ (wl(C

Lvd,vi

)⊗wl(Pvd,d))

∧

wl(CLvd,vi

)⊗wl(Pvd,d) ≺ wl(CLvi)⊗ (wl(C

Lvd,vi

)⊗wl(Pvd,d))

We use only the notation for closed circuits for readabil-ity but condition 1 it is also valid for linear circuits (replacingCR

viand CR

vi for LCvi,···,vi ). We can now state the followingtheorem

Theorem 1. The matrix iteration σx with x ∈ r, l converges toa fixed point in finite time if condition 1 is met

Breaking condition 1 means that consecutively ⊗ addingwl(C

Rvi) or wl(C

Lvi) results in paths with equal preference

that also belong to the multipath routing solution and there-fore the convergence to the routing solution is not assuredin finite time. If condition 1 is met then travelling aroundthe circuit decreases the paths weight in the preferenceorder. This assures the conditions needed for the proofs inthe classical theory both for synchronous and asynchronousconvergence.

In this work we are focused in treating the problemof assuring that packets are forwarded without any loopsif all paths in the multipath routing solution are usedsimultaneously (either within a flow, or the flows chose oneof the paths) and distributed destination-based hop-by-hopforwarding is used. Maintaining the possibility of using allthe preferred paths is not considered in the current literature[4]–[7].

4 LOOP-FREE DESTINATION BASED HOP-BY-HOPFORWARDING

Forwarding loops in destination-based hop-by-hop for-warding are caused by (independent) forwarding decisionsthat are not consistent. Each node forwards based on theminimal set of best paths obtained in the left routing solu-tion or the right routing solution matrix σk

l or σkr (the results

of the matrix iterations at the fixed point k). Let us use figure3 as an example.

5 1 2

4

W(3,4)W(4,5)

W(5,4) W(4,3)

W(1,5)

W(5,1)

W(2,1)

W(1,2)

W(3,2)

W(2,3)

3

Fig. 3. Directed graph

Let us start by considering only nodes 5, 1, 2, and3 and the edges in full lines. They form linear cycleswhere intermediate nodes are traversed twice. For exampleLC2 = 〈2, 1, 5, 1, 2〉, is the linear cycle between nodes 1, 2,and 5. Forwarding loops can occur in linear cycles. Considernode 5 is the source and node 3 is the destination. Considerthe left routing solution σk

l (5, 3) = {w(5, 1) ⊗ (w(1, 2) ⊗w(2, 3))} where the minimal set of weights only has one el-ement corresponding to the weight of the path 〈5, 1, 2, 3〉. Inorder to have loop-free forwarding, node 1 has to prefer thedirect path to 3 meaning that σk

l (1, 3) = {w(1, 2)⊗w(2, 3)}.If, besides this one, we had another equally preferred weight(according to �) in the minimal set of σk

l (1, 3), for examplethe weightw(1, 5)⊗(w(5, 1)⊗(w(1, 2)⊗w(2, 3))), then node1 could forward packets to 3 via node 5 and a forwardingloop could occur.

This later example corresponds to the multipath casewhere we have a minimal set of weights with more than

6

one element corresponding to more than one path. In thiscase, node 1 has two equally preferred weights to node 3:

σkl (1, 3) = {w(1, 5)⊗ (w(5, 1)⊗ (w(1, 2)⊗ w(2, 3)));

w(1, 2)⊗ w(2, 3)}

And packets can be forwarded through any of the twopaths (next-hops) that correspond to those two weights.A forwarding loop can now occur even with the directpath having one of the preferred weights, because the otheralternatives can be chosen.

In linear circuits the only physical alternative path to adestination d at any given node vi of the circuit containsthe linear circuit from vi back to vi. This means that aforwarding loop only occurs if the direct path with weight:wl(C

Rvd,vi

)⊗wl(Pvd,d) has the same preference of the alterna-tive path that contains the linear circuit starting and endingin vi with weight: wl(LCvi,···,vi)⊗ (wl(C

Rvd,vi

)⊗wl(Pvd,d)).This of course breaks condition 1. Therefore condition 1assures loop free forwarding in linear circuits.

In the above example (from node 5 to 3) we have: vd = 2and vi = 1 ; wl(LC1,5,1) = w(1, 5) ⊗ w(5, 1) ; wl(Pvd,d) =w(2, 3) and wl(C

Lvd,vi

) = w(1, 2)⊗w(2, 3). With the solutionσkl (1, 3), we have wl(LC1,5,1) ⊗ (wl(C

Lvd,vi

) ⊗ wl(Pvd,d)) 'wl(C

Lvd,vi

)⊗ wl(Pvd,d) thus breaking condition 1.If we also consider node 4 in figure 3 (connected by the

dotted line edges) we now have closed circuits where nonodes are traversed twice, for example 5, 1, 2, 3, 4, and 5.

In the cycles present in closed circuits, condition 1 isnot sufficient for loop-free multipath forwarding. Considernode 1 as the source and node 4 as the destination, with thefollowing multipath routing solutions:

Node 2 has two equally preferred path weights to node4:

σkl (2, 4) = {w(2, 1)⊗ (w(1, 5)⊗w(5, 4)); w(2, 3)⊗w(3, 4)}

= {wl(CL4,2);wl(C

R4,2)}

Node 1 also has two equally preferred path weights tonode 4:

σkl (1, 4) = {w(1, 2)⊗ (w(2, 3)⊗w(3, 4)); w(1, 5)⊗w(5, 4)}

A forwarding loop can occur in this situation if a packetis forwarded by node 1 to 2 and vice-versa via the next-hopthat corresponds to the longer paths (whose weights belongto the solution). In a single path approach the problem issolved by selecting from the two equally preferred weightsthe one that corresponds to the smallest path. With multi-path, all equal preference weights are part of the solution soall paths must be considered.

Theorem 1 is not enough to prevent these forwardingloops on these cycles because both of the paths causingloops in this example with weights given by:

{w(2, 1)⊗ (w(1, 5)⊗ w(5, 4))} and

{w(1, 2)⊗ (w(2, 3)⊗ w(3, 4))}

comply with condition 1. Theorem 1 refers to further trav-elling an entire circuit, which is not the issue here. Let usexamine the first path – CL

4,2: condition 1 would be brokenif wl(C

L2 ) ⊗ wl(C

L4,2) and wl(C

R2 ) ⊗ wl(C

R4,2) belonged to

the minimal set, which is not true (the other element ofthe minimal set is wl(C

R4,2)). This means that wl(C

L4,2) is

preferred to both of them and therefore the condition holds.The same reasoning can be applied to the paths in node 1.So, the condition holds for both and still a forwarding loopdoes occur.

Let us now consider: the generic circuit of figure 2; thedefinition 7 of directed cycles; the definition 5 of a closedcircuit; and the definition of vd (a vertex that reaches thedestination d by exiting the circuit). We can state the follow-ing condition:

Condition 2. For every circuit C in G(V,w), and all reachabledestinations d, there is at most one vertex vi per circuit such that:

wl(CRvd,vi

)⊗ wl(Pvd,d) = wl(CLvd,vi

)⊗ wl(Pvd,d)

For all other vertices vx 6= vi in C we have:

wl(CRvd,vx

)⊗ wl(Pvd,d) ≺ wl(CLvd,vx

)⊗ wl(Pvd,d)

∀ vx ∈ CRvd,vi

∧

wl(CLvd,vx

)⊗ wl(Pvd,d) ≺ wl(CRvd,vx

)⊗ wl(Pvd,d)

∀ vx ∈ CLvd,vi

That leads us to the following theorem on loop-freeforwarding in closed circuits

Theorem 2. If Condition 2 holds then destination-based hop-by-hop forwarding is loop free.

The reasoning is that only one node vi can have equallypreferred paths to d via the two possible directed cycles. Allthe others must have path preference to d according to theirrelative position to vi so that the packet never returns to vi.A detailed proof of the above theorem can be found in theappendix.

In summary, condition 1 assures loop-free forwarding inlinear circuits and condition 2 in closed circuits. They areboth are necessary to achieve correct multipath destination-based hop-by-hop forwarding on multiple paths.

5 CONDITIONS FOR CORRECT MULTIPATH POLICYROUTING PROTOCOLS

The two conditions in theorems 1 and 2 are computationallycomplex to verify. So we will derive more useful conditionsby analysing if there are specific algebraic properties thatresult in both conditions being met.

Related work has produced results for some algebraicproperties of interest.

For single path protocols, it is known that for an alge-braic structure that holds monotony (definition 1) the matrixiterations σr and σl will converge to the same a result thatcorresponds to a global optimum routing solution [1], [2]that provides loop-free destination-based forwarding [3].

Several proofs [2], [4], [5], [9] can also be found thatfor a non monotonic preordered magma with an increasing⊗ operation (definition 2), the matrix iterations σr and σlstill converge, but in this case to local optimum routingsolutions. To obtain loop-free forwarding in these models,

7

paths must be calculated from destination to source (corre-sponding to σl) [3].

For multipath, the current results [4], [5], [7], [9] onlyassure correct behaviour in the presence of the increasing ⊗property (definition 2), or in strictly monotonic models (i.e.monotony with strict ≺). For a non-decreasing ⊗ operation(definition 3), restrictions have to be applied on how theelements of S are assigned in the network and simultaneousforwarding on all paths is not covered.

In the following text we check under which algebraicproperties conditions 1 and theorem 2 can be assured.

It is easy to see that an increasing ⊗ operation as wellas a strict monotonic ordered semigroup, naturally meetcondition 1, independently of the weight values in S appliedto the edges of any circuits present in the network’s graph.

However, it is interesting to be able to add links to a pathwhile maintaining its preference or, in other words, a non-decreasing ⊗ operation. This opens the possibility to havepaths with a different number of links that have the samepath weight and preference (e.g. if all their links have thesame link weight l ∈ S) potentially increasing the numberof usable paths.

In a non decreasing model, ⊗ adding c can maintainpreference a ' (c ⊗ a). In these conditions we might breakcondition 1 for some circuits of the network since we cannow have

wl(CXvd,vi

)⊗ wl(Pvd,d) '

wl(CXvi )⊗ (wl(C

Xvd,vi

)⊗ wl(Pvd,d))

and/orwl(C

Xvd,vi

)⊗ wl(Pvd,d) '

wl(LCvi,···,vi)⊗ (wl(CXvd,vi

)⊗ wl(Pvd,d))

withX ∈ {R,L}.

This means that to assure finite time convergence and loop-free forwarding in circuits we must then either calculate σlusing an algorithm that stops the iteration when a pathtravels around circuits (i.e. repeats a node), or restrict thenetwork so that there are no circuits (linear or closed) in thegraph where the weight of its links maintains the preferenceof a path.

With this in mind we define the set, L[u] ⊂ S and use itto formulate a condition for the application of weight valuesin circuit links:

Definition 9. Consider a set L[u] ⊂ S for each equivalence class[u] ⊂ S with the following characteristic:

L[u] = {l ∈ S | such that (l ⊗ u) ∈ [u]}

This means that for each equivalence class in S, the correspondingL[u] set contains the values of S that ⊗ added to any value in [u]result in a equivalent preference value (also in [u]).

Condition 3. The ⊗ operation is non-decreasing and a circuitnever has all its links E with weight values l belonging to thesame L[u] set. (For closed circuits this must be true for both the Rand the L directions).

Theorem 3. Convergence to a routing solution in finite time andloop-free forwarding in circuits is assured if condition 3 is met.

Proof. If condition 3 holds then for all possible values of

wl(CXvd,vi

)⊗ wl(Pvd,d) = u ∈ S we have:

wl(CXvi ) 6∈ L[u] and if⊗ is non-decreasing we have:

wl(CXvd,vi

)⊗ wl(Pvd,d) � wl(CXvi )⊗ wl(C

Xvd,vi

)⊗ wl(Pvd,d)

with X ∈ {R,L}, the same is valid replacing wl(CXvi ) by

wl(LCVi,···,Vi) for linear circuits and condition 1 is thereforealways met.

Condition 3 in theorem 3 can be computationally expen-sive to verify since we must check all circuits (closed andlinear) and linear circuits can occur between all vertices ofthe graph. The complexity increases with the number ofelements in the L[u] sets. For linear circuits we can obtaina condition that is simpler to verify by restricting policyapplication in the two directed edges connecting adjacentnodes.

Let us start by defining the concept of weight pairs andthe weight pair set of a protocol model to help us in thedescription of how weight values can be applied to edges(links) in a particular model.

Definition 10. A weight pair denoted by [a, b] consists of twoelements a, b ∈ S such that the weights of the two directededges between two adjacent vertices vi and vj ∈ G(V,w) are:w(vi, vj) = a and w(vj , vi) = b. A weight pair representsone valid weight attribution for the directed edges between twoadjacent vertices in a graph. The set of all valid weight pairs in agiven protocol model is called the weight pair set.

5.1 Verifying finite time convergence and multipathloop-free forwarding in linear circuitsWe can now define a theorem that assumes a sufficient(but not necessary) condition that implies that condition 3needed for theorem 3 is verified for all linear circuits in agraph and therefore condition 1 is also verified. This theo-rem concerns the definition of the protocol itself (restrictingthe possible policies in the directed edges of two adjacentnodes) and in practical terms the condition it assumes ismore simple to verify than condition 3 of theorem 3.

Condition 4. For all a, b ∈ S forming a weight pair [a, b] wehave a, b /∈ L[u]∀u ∈ S (i.e. the weights a and b do not belong tothe same L[u] set).

Theorem 4. If condition 4 is met then with a non-decreasing ⊗operation condition 3 is verified for all linear circuits.

Proof. Consider a linear circuit LC (according to definition6) formed by i nodes. All the directed cycles LCvi,···,vi travelthrough the same edges in both directions. This means thatat some point in theses cycles we have a weight w(b)⊗w(a)being added to the cycle weight, where [a, b] is a weightpair (this edge is travelled in both directions). In order forcondition 3 not to hold, all edges E of the cycles have tohave a weight belonging to the same L[u] set. For this tohappen in a linear circuit the weight pair [a, b] would have tobelong to a set L[u], which is in contradiction with condition4.

In closed circuits of length at least three (according todefinition 5) the argument of this proof is no longer true

8

since a cycle can be formed by only one of the elementsof each weight pair. So, in this case we must still verify ifcondition 3 holds. The verification complexity can be greatlyreduced if for all L[u] sets we have L[u] = {u} (all sets haveonly one element) since in this case we only need to verifythat all edges in all the possible cycles occurring in a closedcircuit are equal.

In summary, finite time convergence is assured in oneof three possible ways: 1) by stopping the path calculationalgorithm when a node appears repeated in the path; 2) byrestricting the applicable policies according to condition 4 oftheorem 4; or 3) verifying that all cycles in closed circuits areaccording to condition 3. Multipath loop-free forwarding inlinear circuits is only assured in the second and third cases.

We now analyse how loop-free multipath forwardingcan be achieved in closed circuits by checking under whichalgebraic properties condition 2 is verified.

Vi

Vi+1L

Vi+1R

Vi+2R

Vi+2L

Vd

Fig. 4. Generalized graph circuit

5.2 Verifying multipath loop-free forwarding in closedcircuitsConsider a destination d reached from a vertex vd of aclosed circuit C through a path Pvd,d with weight wl(Pvd,d)that does not traverse all edges of the circuit. Accordingto theorem 2 only one node (vertex vi) in C can forwardpackets simultaneously on equally preferred paths to d.This allows an origin node to split traffic among multiplepaths and then according to the same condition all othernodes in the path will forward it in the consistent direction.Consider figure 4 for illustration purposes. Consider that avertex vi has two paths to destination d in its minimal set, i.ewl(C

Rvd,vi

) ' wl(CLvd,vi

). If the weight of the link between viand another vertex( e.g., vRi+1 when added to wl(C

Rvd,vR

i+1)

and wl(CLvd,vi

) maintains the preference of both directedcycle segments, a forwarding loop can occur between thesetwo vertices since they have equal preference paths to dthrough each other. This suggests that forwarding loops inclosed circuits occur if and only if two nodes va and vbbelonging to a closed circuit Ci have a path weight betweenthem that maintains the preference of both the paths from vato d and vb to d. We thus formulate the following theorem:

Theorem 5. In a preordered magma with a non-decreasing ⊗operation, multipath forwarding over the result of σ is loop-freein closed circuits if and only if: for all closed circuits Ci in anetwork graph there are no two nodes va and vb with b 6= a andva, vb ∈ Ci such that :

wl(CRvd,vb

) ' wl(CRvb,va

)⊗ wl(CRvd,vb

)

andwl(C

Lvd,va

) ' wl(CLva,vb

)⊗ wl(CLvd,va

)

for all vd ∈ Ci, for all reachable destinations d.

Theorem 5 states that a closed circuit in the networkgraph causes a forwarding loop if at least two vertices in thecircuit are connected by a path that maintains the preferenceof the directed cycle segments between these two verticesand at least another vertex in the circuit. The detailed proofof theorem 5 can be found in the appendix.

This result applies to multipath forwarding in a non-decreasing ⊗ operation model. Previous works [4], [5], [9]show that in the increasing ⊗ case, forward loops neveroccur. They also show that if a single path is used forforwarding, non decreasing algebras cause no forwardingloops as long as the path chosen for forwarding betweenall paths in the minimal set is the shortest path to leave thecircuit.

We can use theorem 5, the definition of theL[u] sets givenin condition 3, and the definition 10 of weight pairs to obtaina sufficient condition for loop-free multipath forwarding inclosed circuits which can be used to check if the attributionof weights to the links of a circuit is correct.

Let us start by noting that two nodes va, vb connectedin a closed circuit by n edges (denoted by Ei withi ∈ 〈1, · · · , n〉) have the weights of the cycle segmentswl(C

Rvb,va

) and wl(CLva,vb

) given by (w(ERn ) means the

weight of the edge n travelled in the right direction):

wl(CRvb,va

) = w(ER1 )⊗ (· · · ⊗ (w(ER

n−1)⊗ w(ERn )))

and

wl(CLva,vb

) = w(ELn )⊗ (· · · ⊗ (w(EL

2 )⊗ w(EL1 )))

where the n, [w(ERn ), w(EL

n )] pairs are weight pairs of thealgebraic model.

Condition 5. Consider all possible vertices vd in a closed circuitC and for each one two other vertices: va and vb connected to vdvia cycle segments with weights wl(C

Rvb,va

) and wl(CLva,vb

).For cycle segments CR

vd,vband CL

vd,vawith weights belonging

to the same equivalence class [u] the weight pairs [a, b] applied inall edges between va and vb are formed by weights a, b ∈ S suchthat a 6∈ L[u] ∧ b 6∈ L[u].

Theorem 6. If condition 5 holds then multipath forwarding isloop-free.

Proof. In order for theorem 5 not to hold, both wl(CRvb,va

)and wl(C

Lva,vb

) must maintain preference when ⊗ added tothe weights of the CR

vd,vband CL

vd,vacycle segments (with

weight in the same equivalence class [u]). This implies, bydefinition of the L[u] set, that wl(C

Rvb,va

) and wl(CLva,vb

)belong to L[u]. It also implies that the weights applied inall the edges between va and vb in both directions (w(ER

n )and w(EL

n )) also belong to L[u], otherwise the total weightwould not remain in the equivalence class. Finally, for thisto occur at least one weight pair [w(ER

n ), w(ELn )] must exist

in the model where both weights of the pair belong to L[u]

and this is in contradiction with condition 5.

9

Lets examine the time complexity of an algorithm forverifying condition 5. Such an algorithm would receive asinputs:

• The set S of possible policy values.• The definition of the ⊗ operation.• The preference order � established over the values

in S.

and would return:

• The L[u] sets for each equivalence class u.• The set of incorrect circuit link labels.

Finding the L[u] sets implies performing the ⊗ additionbetween each element i in the equivalent class u will alln elements of S, since u contains at most the n elementsof S the upper bound for the time complexity is O(n2).Obtaining the incorrect circuit labellings implies a linearsearch in all elements of all L[u] sets. That search has to beperformed (n−1)! ∗n times for a circuit with n vertices. The(n − 1)! is due to checking if the path weights between allpossible pairs of vertices of the circuit, except the Vd vertex(the destination reaching vertex), belong to the same L[u]

set. That search is then performed n times, each time con-sidering one of the vertices of the circuit as the destinationvertex Vd. Finally this has to be done for all circuits in thetopology. Since the time complexity for such an algorithmdoes not scale well, we now derive a sufficient condition fora preordered magma with a non decreasing⊗ operation, suchthat condition 5 is always met for all possible circuit edgeweight attributions E → S.

It follows directly from condition 5 that if weight pairsare defined such that their elements do not belong to thesame L[u] set then condition 5 and therefore the conditionsin theorems 5and 2 are always verified for all possibleedge weight attributions. This is stated in theorem 4, whichmeans that the same condition also assures multipath loopfree forwarding for all possible edge weight attributions inclosed circuits. We re-state the condition 4 and include loopfree multipath forwarding on closed circuits:

Condition 6. Multipath forwarding is loop free if For all definedweight pairs [a, b] with a, b ∈ S and all L[u] sets defined ∀u ∈ Swe have: a /∈ L[u] or b /∈ L[u].

An algorithm based on condition 6 to verify if a givenset of possible policy pairings is correct would receive asinputs:

• The set S of possible policy values.• The definition of the ⊗ operation.• The preference order � established over the values

in S.• The definition of the valid [a, b] label pairings.

and return:

• The L[u] sets for each equivalence class u.• The set of incorrect [a, b] label pairings.

The complexity of finding the L[u] remains, while find-ing the incorrect weight pairs implies checking if bothelements of a pair belong to the sameL[u] set, this is a simplesearch that has a linear time complexity of O(n) where n istotal number of elements in all L[u] sets.

Condition 6 is less complex to verify in the design of aprotocol model, but it also limits policy flexibility becauseit forbids policies that are symmetric in both directions of alink.

6 APPLICATIONS

In this section we show how the contributions of this papercan be used in the design/verification of multipath policybased protocols with simple destination based hop-by-hopforwarding. The example we present can correspond to afew different real world scenarios.

The first one is to consider that it corresponds to aconcrete policy routing model being implemented withmultipath-BGP. Although multipath is not part of the BGPspecification, there are a number of implementations thatallow equally preferred BGP routes to be installed in theforwarding table. Multipath BGP is a multipath policy basedprotocol and the model in this section can be tough oflike a set of inter-domain Internet routing policies reflectinghierarchical business relationships between AutonomousSystems (ASes) that has to be formally checked to ensurerobust behaviour.

The other real world scenario is the implementation of arouting protocol using SDN technologies. SDN, opens wayfor complex routing schemes and provides the possibility tomake use of centralized network information, like topology,available paths, or current traffic distribution. It also allowsthe forwarding of packets at different granularities, provid-ing the possibility of flow based forwarding, and the abilityto define the entire path of all packets of a flow. Severaltechnologies exist (e.g. OpenFlow [10], BGP FlowSpec [11],Path Computation Element (PCE) Communication Protocol(PCEP) [12]) and a lot of successful SDN deployments espe-cially in the data center and SDN-WAN scenarios. However,it has found resistance to be adopted in other scenariosdue to scaling issues, like the limited number of forwardingrule space in hardware and the cost of the communicationwith the controller. Using flow based forwarding can beprohibitive in larger networks that deal with a high numberof individual flows, and the use of destination based for-warding rules can increase the scenarios where new routingprotocols can be implemented using SDN.

In this view the model described in this section corre-sponds to a set of routing policies that are to be applied intree like topologies or power law topologies by building acustom routing protocol implemented using SDN.

The policy model is based in the hierarchy of the nodesin a network.

We start by defining a simple S set with three elements:

1) DW - links and paths in the downwards directionof the hierarchy.

2) UW - links and paths in the upwards direction.3) SL - links and paths between routers at the same

level of the hierarchy.

Two other elements in S represent trivial paths 1 andinvalid paths 0.

The next step is to define the weight pairs (how labelscan be applied in links):

10

TABLE 1⊗ operation

⊗ 1 DW SL UW

1 1 DW SL UW

DW DW DW 0 0

SL SL SL SL 0

UW UW UW UW UW

1) [UW , DW ] for edges between two nodes at differenthierarchy levels.

2) [SL, SL] for edges at the same hierarchy level.

Finally, we establish the preference preorder ≺:

1 ≺ DW ≺ SL ≺ UW ≺ 0

meaning that DW paths have the highest preference fol-lowed by SL paths and finally UW paths. So, going down-wards in the hierarchy is more preferred than going up-wards (local routing solutions are more preferred than goingto the core of the network).

The ⊗ operation is defined in Table 1.

1

2 3 4

5 6

Dw Dw

DwDw

DwUw

Uw Uw

UwUwUwDw

SL

SL

SL

SL

SL

SL

Fig. 5. Graph of network weighted with S values of the hierarchicalmodel

Figure 5 shows a graph where edges have weights in Swhich define a hierarchy: node 1 is higher in the hierarchythan nodes 2,3,4 that are all at the same level and are higherthan nodes 5 and 6 (the lowest level). Some path examplesare: 〈2, 1, 3〉 with weight UW = UW ⊗ DW ; 〈1, 3, 5〉 withweight DW = DW ⊗ DW ; and 〈1, 2, 3〉 that is invalid 0 =DW ⊗ SL.

The first step of an algorithm to check either condition 5or condition 6 is to calculate the L[u] sets resulting in:

• L[DW ] = {DW }.• L[SL] = {SL}.• L[UW ] = {UW }.

The next step to verify condition 6 is to obtain the setof incorrect label pairs, by checking if any pair has its twovalues in the same L[u] set:

• [SL, SL] is incorrect since both elements belong toL[SL].

This means that without changes the model does notverify condition 6. Correct behaviour in this case impliesverifying condition 5 in the topology to verify if all closedcircuits have their links correctly labelled to assure correctforwarding. And verifying condition 3 to assure path calcu-lation convergence.

Condition3 implies that a circuit cannot have all linkslabelled with the same value.

Regarding condition 3, the complexity of its verification(discussed in section 5) is, in this case, reduced becausethe L[u] sets only have one element. This means that onlysymmetric label parings can cause incorrect circuit labellingsand the only symmetric pair is [SL, SL]. An incorrect la-belling occurs if two vertices va, vb exist in a closed circuitsuch that wl(C

Rvb,va

) = SL and wl(CLva,vb

) = SL; and cyclessegments CR

vd,vband CL

vd,vawith weight SL for at least one

destination reaching vertex vd. Figure 6 is an example ofa circuit with an incorrect labelling (causing forwardingloops). If we consider va = 3 and vb = 2 and for examplevd = 5 the we have:

wl(CR2,3) = wl(C

L3,2) = SL

and alsowl(C

R5,2) = wl(C

L5,3) = SL

. This means that both vertices 2 and 3 have SL pathstowards 5 in both directions of the circuit and a forwardingloop can occur between them. The topology in figure 5 forexample has all its closed circuits correctly labelled in termsof condition 5 and multipath forwarding operates correctly.

1 2 3

5

DwDw

Uw Uw

SL

SL

SL

SL

SL

SL

4

Fig. 6. Circuit with incorrect weight attributions in S

The model, as it is implies the correct labelling of circuits.If we consider the two real-world scenarios that we dis-cussed, this can have distinct implications, but it is alwaysundesirable. If the model were to be used for implementinga routing protocol using SDN, then the circuit verificationalgorithm could be implemented in the software, with theadded time complexity (although circuit labellings are not acommon event) that can become high for large topologies.In the case of a multipath BGP implementation circuitverification is almost impossible since there is no centralauthority applying policies.

It is therefore relevant that the used model complies withcondition 4 and condition 6 so that these verifications are notneeded.

There is only one incorrect label pair [SL, SL]. To solvethis we can replace SL with two new elements in S: SLR

and SLL and remove the [SL, SL] label pair, replacing it bya [SLR, SLL] pair. The idea is that links with the SL weightare now weighted with SLR in one direction and SLL in theother.

11

TABLE 2⊗ operation

⊗ 1 DW SLR SLL UW

1 1 DW SLR SLL UW

DW DW DW 0 0 0

SLR SLR SLR SLR 0 0

SLL SLL SLL SLL SLL 0

UW UW UW UW UW UW

The ⊗ operation is now the one in Table 2and the preference relation is:

1 ≺ DW ≺ SLR ≺ SLL ≺ UW ≺ 0

With this new model we now have the following weightpairs [DW , UW ] and [SLR, SLL].

The same steps are used to verify the new model startingwith the L[u] sets:

• L[DW ] = {DW }.• L[SLR] = {SLR}.• L[SLL] = {SLL}.• L[UW ] = {UW }.

Followed by the verification of the incorrect label pairs.In this case all pairs are correct, since there are none withboth weights belonging to the same L[u] set.

This verifies conditions 6 and therefore means that for-warding is loop free. The finite time convergence condition3 is verified if the following four possible cycle edge weightattributions do not exist: all edges with Dw ; all edges withUw ; all edges with SLL ; all edges with SLR, this labellingsare against the semantics of the policies and should notoccur. Besides in the considered real-world scenarios it iseasy to implement path calculation algorithms that onlyadmit simple paths in the solution. This model is inherentlysafe without the added complexity of verifying condition 5.Figure 7 revisits the graph of figure 6 now weighted withthe new model. In the same example (considering va = 3,vb = 2 and vd = 5) we now have:

w(CR2,3) = SLL and w(CL

3,2) = SLR

with different weights, and

wl(CR5,2) = SLL and wl(C

L5,3) = SLR

also with different weights. Forwarding loops no longeroccur since vertex 2 prefers the path 2,3,4,5 with weightSLR and 3 prefers the path 3,4,5 also with weight SLR.It is interesting to see that in this particular case there isno vertex with more than one forwarding path. This is aconsequence of having a very ”linear” network. However,if for instance we wanted two different forwarding pathsto be available in vertex 2, we could attribute to the edges(1, 2); (2, 3); (3, 4) the weight pair [DW , UW ] and in thatcase for vd = 5 vertex 2 can forward through vertex 3 orvertex 1 since both paths have weight Dw and both vertex3 and 1 prefer the DW towards vd instead of the UW pathsback to 2 that would cause a forwarding loop.

1 2 3

5

Uw

(SLR)

4

Uw

DwDw

(SLR)(SLR)

(SLL) (SLL) (SLL)

Fig. 7. Circuit labelled with SLL and SLR

7 RELATED WORK

Some work has been done in developing frameworks basedon formal models to implement routing protocols or verifynetwork configurations. Metarouting [18] is a first effortthat defines a high-level declarative language to describe arouting protocol and uses routing algebra to verify correct-ness and derive implementations. A more recent proposal,FSR [19] presents a tool-kit for analysing and implement-ing routing policies in the inter-domain routing scenario.Both of these approaches rely on previous routing algebraresults and namely rely on the existence of an increasing⊗ operation as the minimal condition for correct operationif multipath forwarding is considered. Our results can beeasily incorporated in these works bringing more generalityin their application.

Classical routing theory has related traditional path cal-culation algorithms like Dijkstra and Bellman-Ford to thealgebraic framework and to matrix iteration. It has alsoshown that non decreasing ⊗models like our own can be im-plemented by matrix iteration algorithms [2], Bellman-Fordor by n iterations of Dijkstra [3]. Our results are consistentwith these findings meaning that several implementationoptions are available for the class of protocols that are dealtwith by our results.

Another related subject is the the upper time bound forconvergence. For monotonic algebras it is known that it isO(n) for a network with n nodes [2] [1], but in routingmodels where ⊗ is non decreasing, like in our results, thisis poorly studied. In [17] there is mention to ongoing workthat proves a stronger upper bound of O(n2) for algebraswithout monotony, but the results are not published yet atthe time of this writing.

We propose a theoretical model for a different typeof protocol (policy-based, and with destination based dis-tributed forwarding) from most of the multipath routingprotocols proposals in the literature. A first class of multi-path protocols are Equal Cost Multipath Protocols (ECMP),there are several of such protocols like OSPF [20] or IS-IS OMP [21] that allow the use of all equal cost paths forforwarding. Some protocols like EIGRP [22] even allow theuse of paths with different costs provided that the nexthops of the secondary paths have a lower cost than theprimary path. The common point in all these protocols isthat their models hold the monotony property and thatassures correct behaviour. With our results however onlya non decreasing⊗ operation is needed, and so the additionof links can maintain preference opening the potential to amuch higher number of equally preferred paths.

12

In order to use non equal cost multiple paths (andtherefore not necessarily the more preferred ones) simpledistributed destination based forwarding must be dropped,route deflection is an example of such an approach, weretags are used to build a tunnel-like mechanism [23]. LISP[24] and MPLS [25] also use tunnels for multipath routing.There are also specific scenarios like data center networkswere SDN based approaches are used that achieve multi-path routing by using flow based forwarding mechanisms,[26] contain a discussion and several examples of suchmechanisms.

BGP in multipath enabled implementations is a protocolthat falls under the category of our model. Multipath BGPdoes not hold monotony and it does not assure correctbehaviour unless some policy restrictions are applied [27].Our results can be used to formally verify a given policyconfiguration or to design policy restrictions that assurecorrect behaviour in multipath BGP scenarios.

In [14] we used our results to design a routing algorithmsuitable for the inter-domain scenario and in [15] we de-scribed an architecture for a Layer 2 fabric that uses a pathfinding algorithm with policies and multipath forwardingthat was designed using this work and implemented usingOpenflow and an SDN controller. Both works constituteexamples of the flexibility that our findings can bring interms of the number of usable paths and illustrate possibleapplications of protocols designed with this new correctnessconditions.

8 CONCLUSIONS

Our work extends the classical routing theory with resultson the correct operation of routing protocols that exhibit asimple non decreasing ⊗ operation and work with multipathrouting solutions and simple destination-based hop-by-hopforwarding. It increases the generality of the existing theoryand allows the modelling and verification of a wider classof policy based protocols. We showed that in non decreasingmodels incorrect behaviour is always related to circuits inthe network graph. We then obtained a set of necessaryconditions that have to be met in all circuits, and that weproved assure correct protocol behaviour. Verifying thoseconditions can have a high time and space complexitydepending on on the particular model and topology size.We also derived a sufficient condition for correct behaviourthat is more easily verifiable and provides a rule for thepairs of labels that can be applied to a link. If a modelverifies this condition it operates correctly and there is noneed for the expensive circuit verification. These resultscan be integrated in the known formal models and can beused in the still poorly explored policy routing algorithmspace. They open way for the formal verification of a classof protocols with different characteristics from most of theexisting multipath routing approaches.

Future work includes the study of the use of these flexi-ble routing protocols in several scenarios. Our work allowsthe use of several paths with great flexibility and no needfor tunnelling forwarding schemes. We believe that thismodel is suited to obtain SDN based network optimizationstrategies that have low computational complexity and scalebetter in terms of data plane state and control signalling.

I

REFERENCES

[1] M. Gondran and M. Minoux, Graphs, Dioids and Semirings: Newmodels and algorithms, 2008.

[2] J. Baras and G. Theodorakopoulos, “Path problems in networks,”Synthesis Lectures on Communication Networks, vol. 3, no. 1, pp. 1–77, 2010.

[3] J. Sobrinho and T. Griffin, “Routing in equilibrium,” MathematicalTheory of Networks and Systems MTNS, Jan 2010.

[4] T. Griffin and A. Gurney, “Increasing bisemigroups and algebraicrouting,” Relations and Kleene Algebra in Computer Science, vol.4988/2008, pp. 123–137, 2008.

[5] A. Gurney, “Construction and verification of routing algebras,”Ph.D. dissertation, University of Cambridge, 2009.

[6] A. J. T. Gurney and T. G. Griffin, “Neighbor-specific BGP: Analgebraic exploration,” in IEEEInternational Conference on NetworkProtocols (ICNP), 2010, 2010, pp. 103–112.

[7] C. Chau, “Policy-based routing with non-strict preferences,” Pro-ceedings of the 2006 conference on Applications, technologies, architec-tures, and protocols for computer communications, pp. 387–398, 2006.

[8] L. Gao and J. Rexford, “Stable internet routing without globalcoordination,” IEEE/ACM Transactions on Networking (TON), vol. 9,no. 6, pp. 681–692, 2001.

[9] J. L. Sobrinho, “An algebraic theory of dynamic network routing,”IEEE/ACM Transactions on Networking, vol. 13, no. 5, pp. 1160–1173,2005.

[10] N. McKeown, et.al, “OpenFlow: enabling innovation in cam-pus networks,” ACM SIGCOMM Computer Communication Review,vol. 38, no. 2, pp. 69–74, 2008.

[11] P. Marques, et. al, “Rfc 5575: Dissemination of flow specificationrules,” August, vol. 2, pp. 6–3, 2009.

[12] J. P. Vasseur and J. L. Le Roux, “Rfc 5440: Path computationelement (pce) communication protocol (pcep),” Tech. Rep., 2009.

[13] P. Amaral, P. F. Pinto, L. Bernardo, and F. Silva, “SDN based trafficengineering without optimization: A centrality based approach,”in Communications (ICC), 2017 IEEE International Conference on.IEEE, 2017, pp. 1–7.

[14] P. Amaral, L. Bernardo, and P. Pinto, “Multipath policy routing forthe inter-domain scenario,” in IEEE ICC ’14., 2014.

[15] P. Amaral, L. Bernardo, and P. Pinto, “An l2 policy based multipathfabric,” in IEEE ICC ’14., 2014.

[16] P. Amaral, L. Bernardo, and P. Pinto, “Multipath policy routingusing destination based hop-by-hop forwarding,” in Network Pro-tocols (ICNP), 2013 21st IEEE International Conference on, Oct 2013,pp. 1–6.

[17] M. L. Daggitt, A. J. T. Gurney, and T. G. Griffin, “Asynchronousconvergence of policy-rich distributed bellman-ford routingprotocols,” in Proceedings of the 2018 Conference of the ACM SpecialInterest Group on Data Communication, ser. SIGCOMM ’18.

[18] T. G. Griffin and J. L. Sobrinho, “Metarouting,” in SIGCOMM ’05.New York, NY, USA: ACM, 2005, pp. 1–12.

[19] Y. Ren, et.al, “FSR: Formal analysis and implementation toolkit forsafe inter-domain routing,” in Proceedings of the ACM SIGCOMM2011 Conference, New York, NY, USA, 2011, pp. 440–441.

[20] J. Moy, “Ospf standardization report,” Tech. Rep., 1998.[21] C. Villamizar and T. Li, “Is-is optimized multipath (isis-omp),”

draft-ietf-isis-omp-02, Internet draft, 1998.[22] R. Albrightson, J. Garcia-Luna-Aceves, and J. Boyle, “Eigrp–a fast

routing protocol based on distance vectors,” 1994.[23] X. Yang and D. Wetherall, “Source selectable path diversity via

routing deflections,” in ACM SIGCOMM Computer CommunicationReview, vol. 36, no. 4., ACM, 2006, pp. 159–170.

[24] D. Saucez, L. Iannone, O. Bonaventure, and D. Farinacci, “De-signing a deployable internet: The locator/identifier separationprotocol,” IEEE Internet Computing, vol. 16, no. 6, pp. 14–21, 2012.

[25] X. Xiao, A. Hannan, B. Bailey, and L. M. Ni, “Traffic engineeringwith mpls in the internet,” IEEE network, vol. 14, no. 2, pp. 28–33,2000.

[26] I. F. Akyildiz, A. Lee, P. Wang, M. Luo, and W. Chou, “A roadmapfor traffic engineering in sdn-openflow networks,” Computer Net-works, vol. 71, pp. 1–30, 2014.

[27] I. Van Beijnum, J. Crowcroft, F. Valera, and M. Bagnulo, “Loop-freeness in multipath bgp through propagating the longest path,”in 2009 IEEE International Conference on Communications Workshops.IEEE, 2009, pp. 1–6.

13

Pedro Amaral Received the Phd. in Electricand Computer Engineering in 2013 the M.Sc.in Computer Engineering in 2006 from Univer-sidade Nova de Lisboa.He Is an Assistant Pro-fessor at Faculdade de Ciencias e Tecnologia,Universidade Nova de Lisboa and a researcherat Instituto de Telecomunicacoes, Lisboa anda IEEE member. Current research interests in-clude routing modelling, very low delay networkqueuing and intelligent network management inSDN and NFV enviroments.

Luis Bernardo Received the Ph.D. degree inElectrical Engineering and Computers from In-stituto Superior Tecnico (IST), Technical Univer-sity of Lisbon, Portugal, in 2002 and the Ha-bilitation in Telecommunications from Faculdadede Ciencias e Tecnologia (FCT), UniversidadeNova de Lisboa (UNL), in 2013. He is a Pro-fessor at FCT- UNL (Faculdade de Ciencias eTecnologia da Uni- versidade Nova de Lisboa)since 1999. He is a researcher at Instituto deTelecomunicacoes, Lisboa. His main research

interests include MAC protocols for wireless communications, routingprotocols and network modelling.

Paulo Pinto Received the Ph.D. degree in Com-puter Science from the University of Kent, atCanterbury, and diploma in Electrical Engineer-ing from Instituto Superior Tecnico, Lisbon, Por-tugal. He is a full professor at UniversidadeNova de Lisboa, Portugal. His current researchinterests include interconnection of wireless net-works, MAC protocols for wireless systems, rout-ing protocols and very low delay network queu-ing and congestion control.