E cient veri cation of network fault tolerance via ...

Consist

ent *Complete *

Well D

ocumented*Easyt

oR

euse* *

Evaluated

*CAV*Ar

tifact *

AECEfficient verification of network fault tolerance

via counterexample-guided refinement?

Nick Giannarakis1, Ryan Beckett2, Ratul Mahajan3,4, and David Walker1

1 Princeton University, Princeton NJ 08544, USA{ng8,dpw}@cs.princeton.edu

2 Microsoft Research, Redmond WA 98052, [email protected]

3 University of Washington, Seattle WA 98195, USA4 Intentionet, Seattle WA, USA

[email protected]

Abstract. We show how to verify that large data center networks sat-isfy key properties such as all-pairs reachability under a bounded num-ber of faults. To scale the analysis, we develop algorithms that identifynetwork symmetries and compute small abstract networks from largeconcrete ones. Using counter-example guided abstraction refinement, wesuccessively refine the computed abstractions until the given propertymay be verified. The soundness of our approach relies on a novel notionof network approximation: routing paths in the concrete network are notprecisely simulated by those in the abstract network but are guaranteedto be “at least as good.” We implement our algorithms in a tool calledOrigami and use them to verify reachability under faults for standarddata center topologies. We find that Origami computes abstract net-works with 1–3 orders of magnitude fewer edges, which makes it possibleto verify large networks that are out of reach of existing techniques.

1 Introduction

Most networks decide how to route packets from point A to B by executingone or more distributed routing protocols such as the Border Gateway Protocol(BGP) and Open Shortest Path First (OSPF). To achieve end-to-end policyobjectives related to cost, load balancing, security, etc., network operators authorconfigurations for each router. These configurations control various aspects of theroute computation such as filtering and ranking route information received fromneighbors, information injection from one protocol to another, and so on.

This flexibility, however, comes at a cost: Configuring individual routers toenforce the desired policies of the distributed system is complex and error-prone [16,22]. The problem of configuration is further compounded by threechallenges. The first is network scale. Large networks such as those of cloud

? This work was supported in part by NSF Grants 1703493 and 1837030, and gifts fromCisco and Facebook. Any opinions, findings, and conclusions expressed are those ofthe authors and do not necessarily reflect those of the NSF, Cisco or Facebook.

2 Nick Giannarakis, Ryan Beckett, Ratul Mahajan, and David Walker

providers can consist of millions of lines of configuration spread across thou-sands of devices. The second is that operators must account for the interactionwith external neighbors who may sent arbitrary routing messages. Finally onehas to deal with failures. Hardware failures are common [15] and lead to a com-binatorial explosion of different possible network behaviors.

To combat the complexity of distributed routing configurations, researchershave suggested a wide range of network verification [2,14,26] and simulation[11,12,24] techniques. These techniques are effective on small and medium-sizednetworks, but they cannot analyze data centers with 1000s of routers and alltheir possible failures. To enable scalable analyses, it seems necessary to exploitthe symmetries that exist in most large real networks. Indeed, other researchershave exploited symmetries to scale verification in the past [3,23]. However, it hasnever been possible to account for failures, as they introduce asymmetries thatchange routing behaviors in unpredictable ways.

To address this challenge, we develop a new algorithm for verifying reacha-bility in networks in the presence of faults, based on the idea of counterexample-guided abstraction refinement (CEGAR) [5]. The algorithm starts by factoringout symmetries using techniques developed in prior work [3] and then attemptsverification of the abstract network using an SMT solver. If verification succeeds,we are done. However, if verification fails, we examine the counter-example todecide whether we have a true failure or we must refine the network further andattempt verification anew. By focusing on reachability, the refinement procedurecan be accelerated by using efficient graph algorithms, such as min cut, to ruleout invalid abstractions in the middle of the CEGAR loop.

We prove the correctness of our algorithm using a new theory of faulty net-works that accounts for the impact of all combinations of k failures. Our keyinsight is that, while routes computed in the abstract network may not simulatethose of the concrete network exactly, under the right conditions they are guar-anteed to approximate them. The approximation relation between concrete andabstract networks suffices to verify key properties such as reachability.

We implemented our algorithms in a tool called Origami and measured theirperformance on common data center network topologies. We find that Origamicomputes abstract networks with 1-3 orders of magnitude fewer edges. Thisreduction speeds verification dramatically and enables verification of networksthat are out of reach of current state-of-the-art tools [2].

2 Key Ideas

The goal of Origami is to speed up network verification in the presence of faults,and it does so by computing small, abstract networks with similar behavior toa given concrete network.

As a first approximation, one can view a network as a directed graph cap-turing the physical topology, and its routing solution as a subgraph where theremaining edges denote the forwarding decision at each node for some fixed des-tination. In the absence of faults, given a concrete and abstract network, one

Efficient verification of network fault tolerance 3

d

b2b1 b3

a

(a)

d

b

a

(b)

d

b2b1 b3

a

(c)

d

b12 b3

a

(d)

Fig. 1: All graph edges shown correspond to edges in the network topology, and wedraw edges as directed to denote the direction of forwarding eventually determined foreach node by the distributed routing protocols for a fixed destination d. In (a) nodesuse shortest path routing to route to the destination d. (b) shows a compressed networkthat precisely captures the forwarding behavior of (a). Figure (c) shows how forwardingis impacted by a link failure, shown as a red line. Figure (d) shows a compressed networkthat is sound approximation of the original network for any single link failure.

can define a natural notion of similarity as a graph homomorphism: assigningeach concrete node a corresponding abstract node such that, for any solution tothe routing problem, the concrete node forwards “in the same direction” as thecorresponding abstract node. For example, the concrete network in Figure 1a isrelated to its abstract counterpart in Figure 1b according to the node colors.

To illustrate, consider the network of Figure 1a, which computes routes to-wards the destination d via a distributed shortest path routing protocol. Nodesb1, b2, and b3 all play the same “role” and we can exploit the symmetries intheir behavior to construct the compressed network presented in Figure 1b. Inthe latter diagram, each of b1, b2, b3 may be assigned to b (and d to d), as allthese nodes “route in the same direction:”. b1, b2, b3 route to concrete node dand b routes to the corresponding node d. Moreover, the compressed networkpreserves many interesting properties of the concrete one, such as the existenceof a path from any node to the destination, the length of such a path, and theset of nodes traversed (modulo the abstraction function). In fact, compressingthe network in this way is the approach taken by Bonsai [3], which works wellin cases where many symmetries exist. The advantage of using compression isthat a compressed network is often dramatically faster to verify than the originalnetwork.

Unfortunately, we run into two significant problems when defining abstrac-tions in this manner in the presence of faults. First, the concrete nodes of Fig-ure 1a have at least 2 disjoint paths to the destination whereas abstract nodesof Figure 1b have just one path to the destination, so the abstract network doesnot preserve the desired fault tolerance properties. Second, consider Figure 1c,which illustrates how the routing decisions change when a failure occurs. Here,the nodes (b1 in particular) no longer route “in the same direction” as the orig-inal network or its abstraction. Hence the invariant connecting concrete andabstract networks is violated.

Returning to the previous example, consider Figure 1c, which presents therouting behavior of the network when the link between b1 and d has failed.The nodes {b1, b2, b3} no longer have identical routing behaviors, and instead


exhibit two different behaviors. In this scenario, the behavior of b1 differs fromthe behavior of b2, b3 but in a setting where the link between b2 and d is theone that fails it will be b2 that has a different behavior. Hence it is not possibleto statically (i.e., when compressing the network) determine which bi nodesexhibit the same routing behavior. Even worse, in general statically determininghow many routing behaviors nodes in each abstract “role” may exhibit underarbitrary (but bounded in number) link failures seems to require solving the veryrouting problem we are trying to simplify. The bottom line is that given ourcurrent definition of similarity (routes “in the same direction”) abstraction ofany pair of concrete nodes as a single abstract node would appear impossible.There is always the potential for an ill-placed failure to generate a difference. Todevelop a sound compression algorithm in the presence of failures, we must goback to the drawing board.

Lossy compression.To achieve compression given a bounded number of link failures, we relax

the notion of similarity between concrete and abstract nodes: A node in theabstract network may merely approximate the behavior of concrete nodes. Thismakes it possible to compress nodes that, in the presence of failures, may routedifferently. In general, when we fail a single link in the abstract network, weare over-approximating the failures in the concrete network by failing multipleconcrete links, possibly more than desired. Nevertheless, the paths taken in theconcrete network can only deviate so much from the paths found in the abstractnetwork:

Property 1. If a node has a route to the destination in the presence of k linkfailures then it has a route that is “at least as good” (as prescribed by the routingprotocol) in the presence of k′ link failures for k′ < k.

In general, the preference of routes (“at least as good”) is protocol depen-dent. For instance, in the case of shortest path routing, “at least as good” isdetermined by the path length. In the case of shortest path routing, one path is“at least as good” as another when the length of the first path is less than orequal to the length of the second path. This relation suffices to verify importantnetwork reliability properties, such as reachability, in the presence of faults. Justas importantly, it allows us to achieve effective network compression to scaleverification.

Revisiting our example, consider the new abstract network of Figure 1d.When the link between b12 and d has failed, b12 still captures the behavior of b1precisely. However, b2 has a better (in this case better means shorter) path tod. Despite this difference, if the operator’s goal was to prove reachability to thedestination under any single fault, then this abstract network suffices.

To summarize so far, we can observe that the abstract network in ?? hasseveral nice properties:

1. Using a verification tool (e.g., Minesweeper [2]) we can verify that all nodesin this abstract network can reach the destination despite any single linkfailure.


2. The abstract network is related to the concrete network by virtue of the factthat concrete nodes have paths “at least as good” as their correspondingabstract nodes.

By 1 and 2, we can conclude that all concrete nodes have a path to the destina-tion, despite the presence of any single fault, and without having had to incurthe expense of verifying this property of the large (in practice, much larger)concrete network.

From specification to algorithm. It is not too difficult to find abstract net-works that approximate a concrete network; the challenge is finding a validabstract network that is small enough to make verification feasible and yet largeenough to include sufficiently many paths to verify the given fault toleranceproperty. Rather than attempting to compute a single abstract network withthe right properties all in one shot, we search the space of abstract networksusing an algorithm based on counter-example guided abstraction refinement [5].

The CEGAR algorithm begins by computing the smallest possible valid ab-stract network. In the example above, this corresponds to the original compressednetwork in Figure 1b, which faithfully approximates the original network whenthere are no link failures. However, if we try to verify reachability in the pres-ence of a single fault, we will conclude that nodes b and a have no route to thedestination when the link between b and d fails. The counterexample due to thisfailure could of course be spurious (and indeed it is). Fortunately, we can easilydistinguish whether such a failure is due to lack of connectivity or an artifact ofover-abstracting, by calculating the number of corresponding concrete failures.In this example a failure on the link 〈b, d〉 corresponds to 3 concrete failures.Since we are interested in verifying reachability for a single failure this cannotconstitute an actual counterexample.

The next step is to refine our abstraction by splitting some of the abstractnodes. The idea is to use the counterexample from the previous iteration to splitthe abstract network in a way that avoids giving rise to the same spurious coun-terexample in the next iteration (Section 5). Doing so results in the somewhatlarger network of Figure 1d. A second verification pass over this larger networktakes longer, but succeeds.

3 The Network Model

Though there are a wide variety of routing protocols in use today, they sharea lot in common. Griffin et al. [17] showed that protocols like BGP and otherssolve instances of the stable paths problem, a generalization of the shortest pathsproblem, and Sobrinho [25] demonstrated their semantics and properties can bemodelled using routing algebras. We extend these foundations by defining stablepaths problems with faults (SPPFs), an extension of the classic Stable PathsProblem that admits the possibility of a bounded number of link failures. Inlater sections, we use this network model to develop generic network compressionalgorithms and reason about their correctness.


Stable path problems with faults (SPPFs): An SPPF is an instanceof the stable paths problem with faults. Informally, each instance defines therouting behavior of an operational network. The definition includes both thenetwork topology as well as the routing policy. The policy specifies the wayrouting messages are transformed as they travel along links and through theuser-configured import and export filters/transformers of the devices, and alsohow the preferred routes are chosen at a given device. In our formulation, eachproblem instance also incorporates a specification of the possible failures andtheir impact on the routing solutions.

Formally, an SPPF is a tuple with six components:

1. A graph G = 〈V,E〉 denoting the network topology.2. A set of “attributes” (i.e., routing messages) A∞ = A ∪ {∞} that may be

exchanged between devices. The symbol∞ represents the absence of a route.3. A destination d ∈ V and its initial route announcement ad ∈ A. For simplic-

ity, each SPPF has exactly one destination (d). (To model a network withmany destinations, one would use a set of SPPFs.)

4. A partial order ≺ ⊆ A∞×A∞ ranks attributes. If a ≺ b then we say route ais preferred over route b. Any route a ∈ A is preferred to no route (a ≺ ∞).This relation determines the way devices select routes along which to forwardtraffic—every device will always select its most preferred route.5

5. A function trans : E → A∞ → A∞ that denotes how messages are processedacross edges. This function models the route maps and filters that transformroute announcements as they enter or leave routers.

6. A bound k on the maximum number of link failures that may occur.

Examples: By choosing an appropriate set of routing attributes, a prefer-ence relation and a transfer function, one can model the semantics of commonlyused routing protocols. For instance, the Routing Information Protocol (RIP)is a simple shortest paths protocol. It can be modelled by an SPPF where (1)the set of attributes A is the set of integers between 0 and 15 (i.e., the setof permitted path lengths), (2) the preference relation is integer inequality soshorter paths are preferred, and (3) the transfer function increments the receivedattribute by 1 or drops the route if it exceeds the maximum hop count of 15:

trans(e, a) =

{∞ if a ≥ 15

a + 1 otherwise

Going beyond simple shortest paths, BGP is a complex, policy-driven proto-col that drives the Internet, and increasingly, data centers [19]. Operators oftenchoose BGP due to its high expressiveness. We can model a version of BGP (sim-plified for presentation) using messages consisting of triples (LP,Comm,Path)where LP is an integer-valued local preference, Comm is a set of community val-ues (which are essentially string tags) and Path is a list of nodes, representing

5 Though there is just one preference relation, when modelling a network, one caninclude the name of the current router (or the entire path) in each attribute. Asa consequence, different routers can have different preferences and make differentrouting decisions.


the path a routing message has traversed. The transfer function always adds thecurrent device to the Path (or drops the message if a loop is detected) and willmodify the LP and Comm components of the attribute according to the deviceconfiguration. For instance, one device may attach a community tag to a routeand another device may filter or modify routes that have the tag attached.

trans((u, v), a) =

{∞ if u ∈ path

config(u,v)(a) otherwisewhere a = (lp, comms,path)

The protocol semantics dictates the preference relation (preferring routeswith higher local preference first, and shorter paths second). A more completeBGP model is not fundamentally harder to model—it simply has additionalattribute fields and more complex transfer and preference relations [21].

SPPF Solutions: In a network, routers will repeatedly exchange messages,applying their transfer functions to neighbor routes and selecting a current bestroute based on the preference relation, until the network reaches a fixpoint (sta-ble state). Interestingly, Griffin et al. [17] showed that all routing solutions canbe described via a set of local stability constraints. We exploit this insight todefine a series of logical constraints that capture all possible routing behaviorsin a setting that includes link failures. More specifically, we define a solution(aka, stable state) S of an SPPF to be a pair 〈L,F〉 of a labelling L and a failurescenario F . The labelling L is an assignment of the final attributes to nodes inthe network. If an attribute a is assigned to node v, we say that node has selected(or prefers) that attribute over other attributes available to it. The chosen routealso determines packet forwarding. If a node X selects a route from neighbor Y ,then X will forward packets to Y . The failure scenario F is an assignment of 0(has not failed) or 1 (has failed) to each edge in the network.

A solution S = 〈L, F 〉 to an SPPF = (G,A, ad,≺, trans, k) is a stable statesatisfying the following conditions:

L(u) =

ad u = d

∞ choicesS(u) = ∅min≺({a | (e, a) ∈ choicesS(u)}) choicesS(u) 6= ∅

subject to∑e∈EF(e) ≤ k

where the choices from the neighbors of node u are defined as:

choicesS(u) = {(e, a) | e = 〈u, v〉, a = trans(e,L(v)), a 6=∞, F(e) = 0}The constraints require that every node has selected the best attribute (accord-ing to its preference relation) amongst those available from its neighbors. Thedestination’s label must always be the initial attribute ad. For verification, thisattribute (or parts of it) may be symbolic, which helps model potentially un-known routing announcements from peers outside our network. For other nodesu, the selected attribute a is the minimal attribute from the choices availableto u. Intuitively, to find the choices available to u, we consider the attributesb chosen by neighbors v of u. Then, if the edge between v and u is not failed,


we push b along that edge, modifying it according to the trans function. Finally,failure scenarios are constrained so that the sum of the failures is at most k.

4 Network approximation theory

Given a concrete SPPF and an abstract SPPF, a network abstraction is a pair offunctions (f, h) that relate the two. The topology abstraction f : V → V mapseach node in the concrete network to a node in the abstract network, while theattribute abstraction h : A∞ → A∞ maps a concrete attribute to an abstractattribute. The latter allows us to relate networks running protocols where nodesmay appear in the attributes (e.g. as in the Path component of BGP).

The goal of Origami is to compute compact SPPFs that may be used for

verification. These compact SPPFs must be closely related to their concrete

counterparts. Otherwise, properties verified on the compact SPPF will not betrue of their concrete counterpart. Section 4.1 defines label approximation, whichprovides an intuitive, high-level, semantic relationship between abstract and con-crete networks. We also explain some of the consequences of this definition andits limitations. Unfortunately, while this broad definition serves as an impor-tant theoretical objective, it is difficult to use directly in an efficient algorithm.Section 4.2 continues our development by explaining two well-formedness re-quirements of network policies that play a key role in establishing label ap-proximation indirectly. Finally, Section 4.3 defines effective SPPF approximationfor well-formed SPPFs. This definition is more conservative than label approx-imation, but has the advantage that it is easier to work with algorithmicallyand, moreover, it implies label approximation. We prove the soundness of ef-fective SPPF approximation relative to SPPF approximation proper. See theappendix [?] for proofs.

4.1 Label approximation

Intuitively, we say the abstract SPPF label-approximates the concrete SPPF

when SPPF has at least as good a route at every node as SPPF does.

Definition 1 (Label Approximation). Consider any solutions S to SPPF

and S to SPPF and their respective labelling components L and L. We say SPPFlabel-approximates SPPF when ∀u ∈ V. h(L(u)) � L(f(u))

If we can establish a label approximation relation between a concrete and anabstract network, we can typically verify a number of properties of the abstractnetwork and be sure they hold of the concrete network. However, the details ofexactly which properties we can verify depend on the specifics of the preferencerelation (≺). For example, in an OSPF network, preference is determined byweighted path length. Therefore, if we know an abstract node has a path ofweighted length n, we know that its concrete counterparts have paths of weightedlength of at most n. More importantly, since “no route” is the worst route, we


d

a b1 b2

c1 c2

g

d

a b12

c12

g

Fig. 2: Concrete network (left) and its corresponding abstraction (right). Nodesc1, c2 prefer to route through b1 (resp. b2), or g over a. Node b1 (resp. b2) dropsrouting messages that have traversed b2 (resp. b1). Red lines indicate a failedlink. Dotted lines show a topologically available but unused link. A purple arrowshow a route unuseable by traffic from b1.

know that if a node has any route to the destination in the abstract network, sodo its concrete counterparts.

As another example, one may wish to check that a BGP network implementsthe Gao-Rexford conditions [13], a set of guidelines to prevent inter-domainoscillation. Hence the network should prefer customer over peer over provider(which are three different types of business relationships). Here, we might learnthat a particular node routes to a peer in the abstract network and hence wewould be guaranteed they route either to a peer or to customer but not to aprovider. Notice, however, that in this case, the preference relation (customer ≺peer ≺ provider) is determined by the configuration and the configuration couldbe incorrect — indeed, our goal is to check such configurations. So it would notbe wise to count on the correctness of the definition of preference everywherein the network. In such a situation, one could check the preference relation atevery node (a simple, inexpensive local check).

Limitations. Some properties are beyond the scope of our tool (indepen-dent of the preference relation). For example, our model cannot reason aboutquantitative properties such as bandwidth, probability of congestion, or latency.

4.2 Well-formed SPPFs

Not all SPPFs are well-behaved. For example, some never converge and oth-ers do not provide sensible models of any real network. To avoid dealing withsuch poorly-behaved models, we demand henceforth that all SPPFs are well-formed. Well-formedness entails that an SPPF is strictly monotonic and isotonic:∀a, e. a 6=∞⇒ a ≺ trans(e, a) strict monotonicity∀a, b, e. a � b ⇒ trans(e, a) � trans(e, b) isotonicity

Monotonicity and isotonicity properties are often cited [7,8] as desirable prop-erties of routing policies because they guarantee network convergence and pre-vent persistent oscillation. In practice too, prior studies have revealed that almostall real network configurations have these properties [14,20].

In our case, these properties help establish additional invariants that tiethe routing behavior of concrete and abstract networks together. To gain some


d

a

b

c x1 x2

d

a

b

c

x12

Fig. 3: Concrete network (left) and its corresponding ∀∃−abstraction (right).Node c prefers to route through x1, x2 and nodes x1, x2 prefer to route throughc.

intuition as to why, consider the networks of Figure 2. The concrete networkon the left runs BGP with the routing policy that node c1 (and c2) prefers toroute through node g instead of a, and that b1 drops announcements comingfrom b2. In this scenario, the similarly configured abstract node b12 can reachthe destination—it simply takes a route that happens to be less preferred by c12than it would if there had been no failure. However, in the concrete analogue,b1, is unable to reach the destination because c1 only sends it the route throughb2, which it cannot use. In this case, the concrete network has more topologicalpaths than the abstract network, but, counterintuitively, due to the network’srouting policy, this turns out to be a disadvantage. Hence having more pathsdoes not necessarily make nodes more accessible. As a consequence, in general,abstract networks cannot soundly overapproximate the number of failures in aconcrete network—an important property for the soundness of our theory.

The underlying issue here is that the networks of Figure 2 are not iso-tonic: suppose L′(c1) is the route from c1 to the destination through nodea, we have that L(c1) ≺ L′(c1) but since the transfer function over 〈b1, c1〉drops routes that have traversed node b2, we have that trans(〈b1, c1〉,L(c1)) 6≺trans(〈b1, c1〉,L′(c1)). Notice that L′(c1) is essentially the route that the ab-

stract network uses i.e. h(L′(c1)) = L(c12), hence the formula above implies

that h(L(b1)) 6≺ L(b12) which violates the notion of label approximation.

A similar problem arises when the routing policy is not monotonic. Considerthe topology of Figure 3 running eBGP with a routing policy that dictates thatnode c prefers to route through nodes x1, x2 over b and nodes x1, x2 prefer toroute through c. The abstract node x12 has a (unique) choice to route throughc. On the other hand, depending on how the concrete network convergences, x2

may route directly to d and c may route through x2. In this scenario x2 endsup having a route to the destination that is worse than the one of its abstractcounterpart x12.

Fortunately, if a network is strictly monotonic and isotonic, such situationsnever arise. Moreover, we check these properties via an SMT solver using a localand efficient test.


4.3 Effective SPPF approximation

We seek abstract networks that label-approximate given concrete networks. Un-fortunately, to directly check that a particular abstract network label approx-imates a concrete network one must effectively compute their solutions. Doingso would defeat the entire purpose of abstraction, which seeks to analyze largeconcrete networks without the expense of computing their solutions directly.

In order to turn approximation into a useful computational tool, we defineeffective approximation, a set of simple conditions on the abstraction functions fand h that are local and can be checked efficiently. When true those conditionsimply label approximation. Intuitively effective approximations impose threemain restrictions on the abstraction functions 6:

1. The topology abstraction conforms to the ∀∃−abstraction condition; thisrequires that there is an abstract edge (u, v) iff for every concrete node usuch that f(u) = u there is some node v such that f(v) = v and (u, v) ∈ E.

2. The abstraction preserves the rank of attributes (rank-equivalence):

∀a, b. a ≺ b ⇐⇒ h(a) ≺ h(b)3. The transfer function and the abstraction functions commute (trans-equivalence):

∀e, a. h(trans(e, a)) = trans(f(e), h(a))

Relating abstract and concrete choices: To relate the solutions of thetwo networks, we must relate the routing choices between abstract and concretenodes. The examples of section 4.2 demonstrate that, in general, the solutionsof the concrete network and the solutions of its effective abstractions are notnecessarily label-approximate. At the same time, they provided us with usefulinsights about the class of networks for which effective abstractions imply label-approximate.

Definition 2. Given an SPPF and its abstraction SPPF defined by (f, h) wesay that the two networks are choice approximate if their solutions satisfy thefollowing property:

∀〈u, v〉 ∈ E, a ∈ A. (f(〈u, v〉), a) ∈ min(choicesS(f(u))) =⇒(∃a. h(a) � a ∧ (〈u, v〉, a) ∈ choicesS(u)) ∨(u ∈ L(v).path)

Our initial intuition is that if an abstract node f(u) has an optimal routingchoice through the link 〈f(u), f(v)〉 then the concrete node u has a routing choiceover 〈u, v〉 that is at least as good. Formally this means that trans(〈u, v〉,L(v)) �trans(〈f(u), f(v)〉, L(f(v)). Thinking inductively, one may wonder: given that vand f(v) are label-approximate, can we prove the same of u and f(u)? As we sawin section 4.2 the transfer function must be isotonic for this to hold.

The example of Figure 3 illustrated the case where using protocols that havea loop detection mechanism, such as BGP, a concrete node may not have a

6 See the technical appendix for a full definition


choice that its abstract counterpart had. This is captured in Definition 4 bythe case where u ∈ L(v).path. To establish label-approximate in this case weneed to resort to monotonicity. To understand how monotonicity is used in thiscase, imagine there was no loop-detection (i.e. the route causing a loop is notdropped), then according to monotonicity, the routing choice v would offer to uwould be strictly worse, as v’s route uses u and every application of the transferfunction produces a worse attribute.

We prove that when these conditions hold, we can approximate any solutionof the concrete network with a solution of the abstract network.

Theorem 1. Given a well-formed SPPF and its effective approximation SPPF,

for any solution S ∈ SPPF there exists a solution S ∈ SPPF, such that theirlabelling functions are label approximate.

5 The verification procedure

The first step of verification is to compute a small abstract network that satis-fies our SPPF effective approximation conditions. We do so by grouping networknodes and edges with equivalent policy and checking the forall-exists topologicalcondition, using an algorithm reminiscent of earlier work [3]. Typically, how-ever, this minimal abstraction will not contain enough paths to prove any fault-tolerance property. To identify a finer abstraction for which we can prove afault-tolerance property we repeatedly:

1. Search the set of candidate refinements for the smallest plausible abstraction.2. If the candidate abstraction satisfies the desired property, terminate the

procedure. (We have successfully verified our concrete network.)3. If not, examine whether the returned counterexample is an actual counterex-

ample. We do so, by computing the number of concrete failures and checkthat it does not exceed the desired bound of link failures. (If so, we havefound a property violation.)

4. If not, use the counterexample to learn how to expand the abstract networkinto a larger abstraction and repeat.

Both the search for plausible candidates and the way we learn a new abstrac-tion to continue the counterexample-guided loop are explained below.

5.1 Searching for plausible candidates

Though we might know an abstraction is not sufficient to verify a given faulttolerance property, there are many possible refinements. Consider, for example,Figure 4(a) presents a simple concrete network that will tolerate a single linkfailure, and Figure 4(b) presents an initial abstraction. The initial abstractionwill not tolerate any link failure, so we must refine the network. To do so, wechoose an abstract node to divide into two abstract nodes for the next iteration.We must also decide which concrete nodes correspond to each abstract node. For


d

a2a1

b1 b2

a3 a4

b3 b4

(a)

d

a

b

(b)

d

a13

b

a24

(c)

d

a12

b12

a34

b34

(d)

Fig. 4: Eight nodes in (a) are represented using two nodes in the abstract network (b).Pictures (c) and (d) show two possible ways to refine the abstract network (b).

example, in Figure 4(c), node a has been split into a13 and a24. The subscriptsindicate the assignment of concrete nodes to abstract ones.

A significant complication is that once we have generated a new abstraction,we must check that it continues to satisfy the effective approximation conditions,and if not, we must do more work. Figure 4 (c) satisfies those conditions, butif we were to split a into a12 and a34 rather than a13 and a24, the forall-existscondition would be violated—some of the concrete nodes associated with b areconnected to the concrete nodes in a12 but not to the ones in a34 and vice versa.To repair the violation of the forall-exists condition, we need to split additionalnodes. In this case, the b node, giving rise to diagram (4d).

Overall, the process of splitting nodes and then recursively splitting furthernodes to repair the forall-exists condition generates many possible candidateabstractions to consider. A key question is which candidate should we select toproceed with the abstraction refinement algorithm?

One consideration is size: A smaller abstraction avoids taxing the verifier,which is the ultimate goal. However, there are many small abstractions that wecan quickly dismiss. Technically, we say an abstraction is plausible if all nodesof interest have at least k + 1 paths to the destination. Implausible abstractionscause nodes to become unreachable with k failures. To check whether an ab-straction is plausible, we compute the min-cut of the graph. Figure 4(d) is anexample of an implausible abstraction that arose after a poorly-chosen split ofnode a. In this case, no node has 2 or more paths to the destination and hencethey might not be able to reach the destination when there is a failure.

Clearly verification using an implausible abstraction will fail. Instead of con-sidering such abstractions as candidates for running verification on, the refine-ment algorithm tries refining them further. A key decision the algorithm needsto make when refining an abstraction is which abstract node to split. For in-stance, the optimal refinement of Figure 4(b) is Figure 4(c). If we were to split

node b instead of a we would end up with a sub-optimal (in terms of size) ab-straction. Intuitively, splitting a node that lies on the min-cut and can reach thedestination (e.g. a) will increase the number of paths that its neighbors on the

unreachable part of the min-cut (e.g. b) can use to reach the destination.

To summarize, the search for new candidate abstractions involves (1) splittingnodes in the initial abstraction, (2) repairing the abstraction to ensure the forall-


exists condition holds, (3) checking that the generated abstraction is plausible,and if not, (4) splitting additional nodes on the min cut. This iterative processwill often generate many candidates. The breadth parameter of the search boundsthe total number of plausible candidates we will generate in between verificationefforts. Of all the plausible candidates generated, we choose the smallest one toverify using the SMT solver.

5.2 Learning from counterexamples

Any nodes of an abstraction that have a min cut of less than k+1 definitely can-not tolerate k faults. If an abstraction is plausible, it satisfies a necessary condi-tion for source-destination connectivity, but not a sufficient one—misconfiguredrouting policy can still cause nodes to be unreachable by modifying and/orsubsequently dropping routing messages. For instance, the abstract network ofFigure 4c is plausible for one failure, but if b’s routing policy blocks routes ofeither a13 or a24 then the abstract network will not be 1-fault tolerant. Indeed, itis the complexity of routing policy that necessitates a heavy-weight verificationprocedure in the first place, rather than a simpler graph algorithm alone.

In a plausible abstraction, if the verifier computes a solution to the networkthat violates the desired fault-tolerance property, some node could not reachthe destination because one or more of their paths to the destination could notbe used to route traffic. We use the generated counterexample to learn edgesthat could not be used to route traffic due to the policy on them. To do so,we inspect the computed solution to find nodes u that (1) lack a route to the

destination (i.e. L(u) = ∞), (2) have a neighbor v that has a valid route tothe destination, and (3) the link between u and v is not failed. These conditionsimply the absence of a valid route to the destination not because link failuresdisabled all paths to the destination, but because the network policy droppedsome routes. For example, in picture Figure 4c, consider the case where b doesnot advertise routes from a13 and a24; if the link between a13 and d fails, thena13 has no route the destination and we learn that the edge 〈b, a13〉 cannot beused. In fact, since a13 and a12 belonged to the same abstract group a beforewe split them, their routing policies are equal modulo the abstraction functionby trans-equivalence. Hence, we can infer that in a symmetric scenario, the link〈b, a24〉 will also be unusable.

Refining using learned paths:

Given a set of unuseable edges, learned from a counterexample, we restrict themin cut problems that define the plausible abstractions, by disallowing the use ofthose edges. Essentially, we enrich the refinement algorithm’s topological basedanalysis (based on min-cut) with knowledge about the policy; the algorithm willhave to generate abstractions that are plausible without using those edges. Withthose edges disabled, the refinement process continues as before.


6 Implementation

Origami uses the Batfish network analysis framework [12] to parse network con-figurations, and then translate them into a pure functional intermediate repre-sentation (IR) designed for network verification. This IR represents the structureof routing messages and the semantics of transfer and preference relations usingstandard functional data structures.

The translation generates a separate functional program for each destina-tion subnet. In other words, if a network has 100 top-of-rack switches and eachsuch switch announces the subnets for 30 adjacent hosts, then Origami gener-ates 100 functional programs (i.e. problem instances). We separately apply ouralgorithms to each problem instance, converting the functional program to anSMT formula when necessary according to the algorithm described earlier. Sincevendor routing configuration languages have limited expressive power (e.g., noloops or recursion) the translation requires no user-provided invariants. We useZ3 [10] to determine satisfiability of the SMT problems. Solving the problemsseparately (and in parallel) provides a speedup over solving the routing problemfor all destinations simultaneously: The individual problems are specialized to aparticular destination. By doing so, opportunities for optimizations that reducethe problem size, such as dead code elimination, arise.

Optimizing refinement: During the course of implementing Origami, wediscovered a number of optimizations to the refinement phase.

– If the min-cut between the destination and a vertex u is less than or equalto the desired number of disjoint paths, then we do not need to computeanother min-cut for the nodes in the unreachable portion of vertices T ; weknow nodes in T can be disconnected from the destination. This significantlyreduces the number of min-cut computations.

– We stop exploring abstractions that are larger in size than the smallestplausible abstraction computed since the last invocation of the SMT solver.

– We bias our refinement process to explore the smallest abstractions first.When combined the previous optimization, this prunes our search spacefrom some abstractions that were unnecessary large.

Minimizing counterexamples: When the SMT solver returns a coun-terexample, it often uses the maximum number of failures. This is not surprisingas maximizing failures simplifies the SMT problem. Unfortunately, it also con-founds our analysis to determine whether a counterexample is real or spurious.For instance, if the failure bound is 2, failing 2 abstract links could result in 4concrete failures, which exceeds the failure bound, and leads the simple analysisto believe the counterexample is spurious, causing another round of refinement.However, it could also be the case that there exists another failure scenario thatresults in just 2 concrete failures. This latter scenario might even use just a sub-set of the links from the initial scenario. If we do not detect such issues, we willslow analysis by performing unnecessary iterations of the refinement loop.

To mitigate the effect of this problem, we could ask the solver to minimizethe returned counterexample, returning a counterexample that corresponds to


Topo Con V/E Fail Abs V/E Ratio Abs Time SMT Calls SMT Time

FT20 500/8000

1 9/20 55.5/400 0.1 1 0.13 40/192 12.5/41.67 1.0 2 7.65 96/720 5.20/11.1 2.5 2 24810 59/440 8.48/18.18 0.9 - -

FT401 12/28 166.7/2285.7 0.1 1 0.1

2000/64000 3 45/220 44.4/290.9 33 2 12.35 109/880 18.34/72.72 762.3 2 184.1

SP401 13/32 153.8/2000 0.2 1 0.1

2000/64000 3 39/176 51.3/363.6 30.3 1 25 79/522 25.3/122.6 372.2 1 22

FbFT1 20/66 37.2/164.8 0.1 3 1

744/10880 3 57/360 13.05/30.22 1 4 18.35 93/684 8/15.9 408.9 - -

Fig. 5: Compression results. Topo: the network topology. Con V/E: Number ofnodes/edges of concrete network. Fail: Number of failures. Abs V/E: Numberof nodes/edges of the best abstraction. Ratio: Compression ratio (nodes/edges).Abs Time: Time taken to find abstractions (sec.). SMT Calls: Number of callsto the SMT solver. SMT Time: Time taken by the SMT solver (sec.).

the fewest concrete link failures. We could do so by providing the solver withadditional constraints specifying the number of concrete links that correspondto each abstract link and then asking the solver to return a counterexample thatminimizes this sum of concrete failures. Of course, doing so requires we solve amore expensive optimization problem. Instead, given an initial (possibly spuri-ous counter-example), we simple ask the solver to find a new counterexamplethat (additionally) satisfies this constraint. If it succeeds, we have found a realcounterexample. If it fails, we use it to refine our abstraction.

7 Evaluation

We evaluate Origami on a collection of synthetic data center networks that areusing BGP to implement shortest-paths routing policies over common industrialdatacenter topologies. Data centers are good fit for our algorithms as they canbe very large but are highly symmetrical and designed for fault tolerance. Datacenter topologies (often called fattree topologies) are typically organized in lay-ers, with each layer containing many routers. Each router in a layer is connectedto a number of routers in the layer above (and below) it. The precise number ofneighbors to which a router is connected, and the pattern of said connections,is part of the topology definition. We focus on two common topologies: fattreetopologies used at Google (labelled FT20, FT40 and SP40 below) and a differentfattree used at Facebook (labelled FB12). These are relatively large data centertopologies ranging from 500 to 2000 nodes and 8000 to 64000 edges.

SP40 uses a pure shortest paths routing policy. For other experiments (FT20,FT40, FB12”, we augment shortest paths with additional policy that selectivelydrops routing announcements, for example disabling “valley routing” in various


places which allows up-down-up-down routes through the data centers insteadof just up-down routes. The pure shortest paths policy represents a best-casescenario for our technology as it gives rise to perfect symmetry and makes ourheuristics especially effective. By adding variations in routing policy, we providea greater challenge for our tool.

Experiments were done on a Mac with a 4GHz i7 CPU and 16GB memory.

7.1 Compression results

Figure 5 shows the level of compression achieved, along with the required timefor compression and verification. In most cases, we achieve a high compressionratio especially in terms of links. This drastically reduces the possible failurecombinations for the underlying verification process. The cases of 10 link fail-ures on FT20 and 5 link failures on FbFT demonstrate another aspect of ouralgorithm. Both topologies cannot sustain that many link failures, i.e. some con-crete nodes have less than 10 (resp. 5) neighbors. We can determine this as werefine the abstraction; there are (abstract) nodes that do not satisfy the mincut requirement and we cannot refine them further. This constitutes an actualcounterexample and explains why the abstraction of FT20 for 10 link failures issmaller than the one for 5 link failures. Importantly, we did not use the SMTsolver to find this counterexample. Likewise, we did not need to run a min cut onthe much larger concrete topology. Intuitively, the rest of the network remainedabstract, while the part that led to the counterexample became fully concrete.

7.2 Verification performance

The verification time of Origami is dominated by abstraction time and SMTtime, which can be seen in Figure 5. In practice, there is also some time takento parse and pre-process the configurations but it is negligible. The abstractiontime is highly dependent on the size of the network and the abstraction searchbreadth used. In this case, the breadth was set to 25, a relatively high value.

While the verification time for a high number of link failures is not negligible,we found that verification without abstraction is essentially impossible. We usedMinesweeper[2], the state-of-the-art SMT-based network verifier, to verify thesame fault tolerance properties and it was unable to solve any of our queries.This is not surprising, as SMT-based verifiers do not scale to networks beyondthe size of FT20 even without any link failures.

7.3 Refinement effectiveness

We now evaluate the effectiveness of our search and refinement techniques.

Effectiveness of search. To assess the effectiveness of the search procedure, wecompute an initial abstraction of the FT20 network suitable for 5 link failures,using different values of the search breadth. On top of this, we additionallyconsider the impact of some of the heuristics described in Section 5. Figure 6


1 5 15 25

75

250272

2624 26242176 21762080 2080 1920 19202080 1920

458 458

2080 1920

458 444

Search Breadth

Abst

ract

ion

size

(#nodes

)

FT20 Abstractions

Heuristics off Reachable off Common off All Heuristics

Fig. 6: The initial abstraction of FT20 for 5 link failures using different heuristicsand search breadth. On top of the bars is the number of edges of each abstraction.

presents the size (the number of nodes are on the y axis and the number of edgeson top of the bars) of the computed abstractions with respect to various valuesfor the breadth of search and sets of heuristics:

– Heuristics off means that (almost) all heuristics are turned off. We still tryto split nodes that are on the cut-set.

– Reachable off means that we do not bias towards splitting of nodes in thereachable portion of the cut-set.

– Common off means that we do not bias towards splitting reachable nodesthat have the most connections to unreachable nodes.

The results of this experiment show that in order to achieve effective compres-sion ratios we need to employ both smart heuristics and a wide search throughthe space of abstractions. It is possible that increasing the search breadth wouldmake the heuristics redundant, however, in most cases this would make the re-finement process exceed acceptable time limits.

Use of counterexamples. We now assess how important it is to 1) use symme-tries in policy to infer more information from counterexamples, and 2) minimizethe counterexample provided by the solver.

We see in Figure 7 that disabling them increases number of refinement iter-ations. While each of these refinements is performed quickly, the same cannotbe guaranteed of the verification process that runs between them. Hence, it isimportant to keep refinement iterations as low as possible.

8 Related work

Our approach to network fault-tolerance verification draws heavily from ideas inprior work exploiting symmetry and abstraction in model checking [18,4,6] andautomatic abstraction refinement via CEGAR [1,5,9]. However, we apply theseideas to network routing, which introduces different challenges and opportunities.For example, our notion of abstraction (∀∃−abstraction) differs from the typical


existential abstraction used in model checking [6]. In addition, we have to dealwith network topological structure and asymmetries introduced by failures.

3 5

822

248253

4015

5

6

2

2

2

2

Link Failures

SM

Tti

me

FT20 Counterexample Optimizations

Symmetric policies offMinimize counterexamples off

All Optimizations

Fig. 7: Effectiveness of minimizingcounterexamples and of learning un-used edges. On top of the bars isthe number of SMT calls. The re-finement time is insignificant so weomit it.

Bonsai [3] and Surgeries [23] bothleverage abstraction to accelerate verifica-tion for routing protocols and packet for-warding respectively. Both tools computea single abstract network that is bisimi-lar to the original concrete network. Alas,neither approach can be used to reasonabout properties when faults may occur.

Minesweeper [2] is a general approachto control plane verification based on astable state encoding, which leverages anSMT solver in the back-end. It supports awide range of routing protocols and prop-erties, including fault tolerance proper-ties. Our compression is complementaryto such tools; it is used to alleviate thescaling problem that Minesweeper faceswith large networks.

With respect to verification of faulttolerance, ARC [14] translates a limitedclass of routing policies to a weightedgraph where fault-tolerance propertiescan be checked using graph algorithms.However, ARC only handles shortest path routing and cannot support statefulfeatures such as BGP communities, or local preference, etc. While ARC appliesgraph algorithms on a statically-computed graph, we use graph algorithms aspart of a refinement loop in conjunction with a general purpose solver.

9 Conclusions

We present a new theory of distributed routing protocols in the presence ofbounded link failures, and we use the theory to develop algorithms for networkcompression and counterexample-guided verification of fault tolerance proper-ties. In doing so, we observe that (1) even though abstract networks route differ-ently from concrete ones in the presence of failures, the concrete routes wind upbeing “at least as good” as the abstract ones when networks satisfy reasonablewell-formedness constraints, and (2) using efficient graph algorithms (min cut)in the middle of the CEGAR loop speeds the search for refinements.

We implemented our algorithms in a network verification tool called Origami.Evaluation of the tool on synthetic networks shows that our algorithms accelerateverification of fault tolerance properties significantly, making it possible to verifynetworks out of reach of other state-of-the-art tools.


Future work. Relaxing the strict monotonicity condition is an obvious ex-tension to our theory. In [3], they handle potentially non-monotonic featuresof BGP (local preference) using a conservative approach that statically boundsthe number of abstract nodes required to represent a group of nodes running anon-monotic policy. Whether this approach applies in conjunction with failuresneeds to be studied.

Concerning the algorithm, it would be interesting to devise refinement al-gorithms that can efficiently handle properties beyond reachability, for instance(weighted) path length.

References

1. Ball, T., Majumdar, R., Millstein, T.D., Rajamani, S.K.: Automatic predicateabstraction of C programs. In: Proceedings of the 2001 ACM SIGPLAN Confer-ence on Programming Language Design and Implementation (PLDI). pp. 203–213(2001)

2. Beckett, R., Gupta, A., Mahajan, R., Walker, D.: A general approach to networkconfiguration verification. In: SIGCOMM (August 2017)

3. Beckett, R., Gupta, A., Mahajan, R., Walker, D.: Control plane compression. In:Proceedings of the 2018 Conference of the ACM Special Interest Group on DataCommunication. pp. 476–489. ACM (2018)

4. Clarke, E.M., Filkorn, T., Jha, S.: Exploiting symmetry in temporal logic modelchecking. In: Computer Aided Verification, 5th International Conference, CAV,Proceedings. pp. 450–462 (1993)

5. Clarke, E.M., Grumberg, O., Jha, S., Lu, Y., Veith, H.: Counterexample-guidedabstraction refinement. In: Computer Aided Verification, 12th International Con-ference, CAV, Proceedings. pp. 154–169 (2000)

6. Clarke, E.M., Grumberg, O., Long, D.E.: Model checking and abstraction. ACMTrans. Program. Lang. Syst. 16(5), 1512–1542 (September 1994)

7. Daggitt, M.L., Gurney, A.J.T., Griffin, T.G.: Asynchronous convergence of policy-rich distributed bellman-ford routing protocols. pp. 103–116. SIGCOMM (2018)

8. Daggitt, M.L., Gurney, A.J., Griffin, T.G.: Asynchronous convergence of policy-richdistributed bellman-ford routing protocols. In: Proceedings of the 2018 Conferenceof the ACM Special Interest Group on Data Communication. pp. 103–116. ACM(2018)

9. Das, S., Dill, D.L.: Successive approximation of abstract transition relations. In:Proceedings of the 16th Annual IEEE Symposium on Logic in Computer Science.pp. 51–. LICS ’01 (2001)

10. De Moura, L., Bjørner, N.: Z3: An efficient SMT solver. In: TACAS (2008)11. Feamster, N., Rexford, J.: Network-wide prediction of BGP routes. IEEE/ACM

Trans. Networking 15(2) (2007)12. Fogel, A., Fung, S., Pedrosa, L., Walraed-Sullivan, M., Govindan, R., Mahajan,

R., Millstein, T.: A general approach to network configuration analysis. In: NSDI(2015)

13. Gao, L., Rexford, J.: Stable internet routing without global coordination. In: SIG-METRICS (2000)

14. Gember-Jacobson, A., Viswanathan, R., Akella, A., Mahajan, R.: Fast controlplane analysis using an abstract representation. In: SIGCOMM (2016)


15. Gill, P., Jain, N., Nagappan, N.: Understanding network failures in data centers:Measurement, analysis, and implications. In: SIGCOMM (2011)

16. Godfrey, J.: The summer of network miscon-figurations. https://blog.algosec.com/2016/08/

business-outages-caused-misconfigurations-headline-news-summer.html

(2016)17. Griffin, T.G., Shepherd, F.B., Wilfong, G.: The stable paths problem and interdo-

main routing. IEEE/ACM Trans. Networking 10(2) (2002)18. Kesten, Y., Pnueli, A.: Control and data abstraction: The cornerstones of practical

formal verification. Software Tools for Technology Transfer 4, 2000 (2000)19. Lapukhov, P., Premji, A., Mitchell, J.: Use of BGP for routing in large-scale data

centers. Internet draft (2015)20. Lopes, N.P., Rybalchenko, A.: Fast bgp simulation of large datacenters. In: Inter-

national Conference on Verification, Model Checking, and Abstract Interpretation.pp. 386–408. Springer (2019)

21. Lougheed, K.: A border gateway protocol (bgp). RFC 1163, RFC Editor (1989),http://www.rfc-editor.org/rfc/rfc1163.txt, http://www.rfc-editor.org/

rfc/rfc1163.txt

22. Mahajan, R., Wetherall, D., Anderson, T.: Understanding BGP misconfiguration.In: SIGCOMM (2002)

23. Plotkin, G.D., Bjørner, N., Lopes, N.P., Rybalchenko, A., Varghese, G.: Scalingnetwork verification using symmetry and surgery. In: POPL (2016)

24. Quoitin, B., Uhlig, S.: Modeling the routing of an autonomous system with c-bgp.Netwrk. Mag. of Global Internetwkg. 19(6), 12–19 (November 2005)

25. Sobrinho, J.a.L.: An algebraic theory of dynamic network routing. IEEE/ACMTrans. Netw. 13(5), 1160–1173 (October 2005)

26. Weitz, K., Woos, D., Torlak, E., Ernst, M.D., Krishnamurthy, A., Tatlock, Z.:Formal semantics and automated verification for the border gateway protocol. In:NetPL (2016)

https://blog.algosec.com/2016/08/business-outages-caused-misconfigurations-headline-news-summer.html

https://blog.algosec.com/2016/08/business-outages-caused-misconfigurations-headline-news-summer.html

http://www.rfc-editor.org/rfc/rfc1163.txt




Appendix

A1: Summary of SPPF Definitions

SPPF instance SPPF = (G,A, ad,≺, trans, k)

G = (V,E, d) network topologyV topology verticesE : V × V topology edgesd : V destination vertexA∞ = A ∪ {∞} routing attributesad : A∞ initial route≺ ⊆ A∞ ×A∞ comparison relationtrans : E ×A∞ → A∞ transfer functionk number of link failures

Well-formed SPPF

∀v. (v, v) /∈ E self-loop freedom∀e. trans(e,∞) =∞ non-spontaneity∀a, e. a 6=∞⇒ a ≺ trans(e, a) strict monotonicity∀a, b, e. a � b⇒ trans(e, a) � trans(e, b) isotonicity∀a, b, u, v. a � b ∧ u 6∈ a. path⇒ trans(〈u, v〉, a) � trans(〈u, v〉, b) bgp-isotonicity

SPPF solution S , 〈L,F〉 : (V → A)× (E → {0, 1})

∑e∈E

F(e) ≤ k

L(u) =

ad u = d

∞ choicesS(u) = ∅min≺({a | (e, a) ∈ choicesS(u)}) choicesS(u) 6= ∅

wherechoicesS(u) = {(e, a) | e = 〈u, v〉, a = trans(e,L(v)), a 6=∞,F(e) = 0}

Effective approximations (f, h) : (V → V )× (A→ A)

(f(d) = d) ∧ (∀d′. d 6= d′ =⇒ f(d′) 6= d) dest-equivalenceh(∞) =∞ drop-equivalenceh(ad) = ad orig-equivalence∀a, b. a ≺ b ⇐⇒ h(a) ≺ h(b) rank-equivalence

∀e, a. h(trans(e, a)) = trans(f(e), h(a)) trans-equivalence∀e, a. e = (u, v) ∧ f(u) /∈ h(a).path =⇒ transfer-equivalence-bgp

h(trans(e, a)) = trans(f(e), h(a))

(u, v) ∈ E ⇐⇒(∀u. f(u) = u =⇒ ∃v. f(v) = v ∧ (u, v) ∈ E

)∀∃−abstraction


Definition 3 (Forwarding Relation). Given an SPPF and the labelling com-ponent of its solution we define the edges that are used to forward traffic asfollows:

fwdS(u) = {e | (e, a) ∈ choicesS(u), a ≈ L(u)}where a1 ≈ a2 , a1 6≺ a2 ∧ a2 6≺ a1

A2: Proofs of Theorems

Our proofs proceed by induction on the forwarding relation of the abstract net-work; hence we first establish that forwarding is a well-founded relation.

Lemma 1. Let S be a finite set and @ a relation on S. If its transitive closureis irreflexive then @ is well-founded.

Proof. Suppose there is an infinite descending chain. As the set S is finite, thisimplies that the chain has repeating elements, i.e.

. . . aj @ . . . @ aj . . .

But this implies that aj @+ aj , a contradiction. Hence there are no infinitedescending chains and @ is well-founded.

Lemma 2. The relation b @ a = f(a, b) ∈ fwdS(f(a)) over the set of concretevertices V is well-founded.

Proof. First of all, notice that all routing protocols under consideration are loop-free. Hence the transitive closure of fwdS is irreflexive. Moreover, the set ofconcrete vertices V is finite, hence by Lemma 1 @ is well-founded.

Notation 1 (Min) We write min(A) to denote an element a ∈ A s.t ∀b ∈A. a � b.

A note about BGP. BGP has an implicit loop detection mechanism whichdrops a route when a loop is formed. Intuitively, we can model this mechanismusing a transfer function of the following form:

trans(〈u, v〉, a) =

{∞ if u ∈ path

config(u,v)(a) otherwise

where a = (lp, comms,path)

where config depends on the specific configuration of the device. To supportthis idiom we have to adopt many of our definitions to BGP friendly variants.

In our proofs, we do not explicitly distinguish the cases between BGP andnon-BGP protocols, but we always prove the extra preconditions to use anyBGP variants of our definitions. For protocols which do not have utilize thisexplicit loop-detection mechanism, we can simple omit/ignore the proof of thesepreconditions.

In order to relate the labelling functions of two networks we must relate their(≺-minimal) choices.


Definition 4. Given an SPPF and its abstraction SPPF defined by (f, h) wesay that the two networks are choice approximate if their solutions satisfy thefollowing property:

∀〈u, v〉 ∈ E, a ∈ A. (f(〈u, v〉), a) ∈ min(choicesS(f(u))) =⇒(∃a. h(a) � a ∧ (〈u, v〉, a) ∈ choicesS(u)) ∨(u ∈ L(v).path)

Intuitively, every node in the concrete network will have a choice over someedge that is at least as good as the choice of its abstraction over the correspondingabstract edge. A corner case is when this choice creates a loop in the concretenetwork; but in this case it must be that the concrete network already has abetter choice (by monotonicity).

Next, we prove that choice approximate implies label approximate.

Lemma 3. Given an SPPF and its effective approximation SPPF defined by(f, h), if the networks are choice approximate then they are label approximate.

Proof. Consider a node u ∈ V . We proceed by well-founded induction on @. Wedistinguish two cases for u:

– (u = d) By definition L(d) = ad and L(d) = ad, and using orig-equivalence

we get that L(d) = L(d).– (u 6= d) For vertices other than the destination there are two more cases:

• (L(f(u)) =∞) then the goal holds trivially as ∀a. a � ∞.

• (L(f(u)) 6=∞) then by definition there exists some vertex v s.t. 〈f(u), v〉 ∈fwdS(f(u)), i.e. f(u) has a ≺-minimal choice which it forwards to:

(〈f(u), v〉, a) ∈ min(choicesS(f(u)))

for a = trans(〈f(u), v〉, L(v)). Moreover notice that 〈f(u), v〉 ∈ E andhence by ∀∃−abstraction we have that ∃v ∈ V. f(v) = v. Using u, v, a weinstantiate the choice approximate assumption which leads to two cases:1. Either, ∃a ∈ A. h(a) � a and (〈u, v〉, a) ∈ choicesS(u), i.e. vertex

u has at least one choice that is as good as the best choice of theabstract vertex f(u), hence either L(u) = a or there is some better

choice and L(u) ≺ a. In any case, as h(a) � a and a ≈ L(f(u)) we

have that h(L(u)) � L(f(u)).2. Or u ∈ L(v). path, then notice that by monotonicity:

L(u) ≺ L(v) (1)

and moreover, by L(f(u)) = trans(〈f(u), f(v)〉, L(f(v))) 6= ∞ andmonotonicity we deduce that:

L(f(v)) ≺ L(f(u)) (2)


Equation (1) and rank-equivalence imply that

h(L(u)) ≺ h(L(v)) (3)

Finally, v @ u as 〈f(u), f(v)〉 ∈ fwdS(f(u)) and hence we can utilizethe induction hypothesis to deduce that

h(L(v)) � L(f(v)) (4)

By Equation (2), Equation (4) and transitivity we get that h(L(v)) ≺ L(f(u))

and using Equation (3) and transitivity we conclude that h(L(u)) ≺ L(f(u)).

Lemma 4 (Surjective attribute abstraction). For every vertex u there ex-

ists some attribute a ∈ A such that h(a) = L(f(u)).

Proof. The proof proceeds by well-founded induction on @.

– (u = d) then the equation is satisfied by ad.– (u 6= d) then we distinguish two cases:

• (L(f(u)) =∞) then the equation is satisfied by a =∞.

• (L(f(u)) 6=∞) then by definition, there exists some v ∈ V s.t.:

(〈f(u), v〉, trans(〈f(u), v〉, L(v))) ∈ min(choicesS(f(u))) (5a)

〈f(u), v〉 ∈ fwdS(f(u)) (5b)

By ∀∃−abstraction, ∃v. f(v) = v and 〈u, v〉 ∈ E and Equation (5b)

implies that v @ u hence by induction hypothesis ∃av. h(av) = L(f(v)).

Further notice, that 〈f(u), f(v)〉 6∈ h(av).path since L(f(u)) 6=∞. Hencewe can apply transfer-equivalence:

L(f(u)) = trans(〈f(u), f(v)〉, h(av)) by transfer-equivalence

L(f(u)) = h(trans(〈u, v〉, av))

Thus we pick a = trans(〈u, v〉, av) as the witness to the existential as itsatisfies the equation.

Next we prove that the solutions of the concrete and abstract network willbe choice approximate.

Theorem 2. Given a well-formed SPPF = (G,A, ad,≺, trans, k) and its effec-

tive approximation SPPF = (G, A, ad, ≺, trans, k) defined by (f, h), for any solu-

tion (L,F) ∈ SPPF and (L,F) ∈ SPPF, L and L are choice approximate.

Proof. Consider a vertex u ∈ V . We strengthen our induction hypothesis byproving choice and label approximate at the same time, and proceed by well-founded induction on the @ relation.

We consider two separate cases for u:


– (u = d) By definition L(d) = ad and L(d) = ad. Moreover L(f(d)) = L(d)

by dest-equivalence and hence h(L(d)) � L(f(d)) follows trivially by orig-equivalence.

– (u 6= d) We prove choice approximate first. By hypothesis we have that(〈f(u), f(v)〉, a) ∈ min(choicesS(f(u))), which implies:

〈f(u), f(v)〉 ∈ E (6a)

a = trans(〈f(u), f(v)〉, L(f(v))) (6b)

a 6=∞ (6c)

F(〈f(u), f(v)〉) = 0 (6d)

〈f(u), f(v)〉 ∈ fwdS(f(u)) (6e)

We distinguish two cases:• (u 6∈ L(v).path)

Notice that 〈u, v〉 ∈ E by ∀∃−abstraction and F(〈u, v〉) = 0 by definition

of F . It suffices to prove that a = trans(〈u, v〉,L(v)) 6=∞ and h(a) � a.Since 〈f(u), f(v)〉 ∈ fwdS(f(u)) we have that v @ u and we can apply

the induction hypothesis to deduce that h(L(v)) � L(f(v)). By Lemma 4we have that:

∃av. h(av) = L(f(v))

h(L(v)) � h(av) by induction hypothesis

L(v) � av by rank-equivalence

trans(〈u, v〉,L(v)) � trans(〈u, v〉, av) by isotonicity as u 6∈ L(v).path

h(trans(〈u, v〉,L(v) � trans(〈f(u), f(v)〉, h(av)) by transfer-equivalence-bgp asf(u) 6∈ h(av).path since

otherwise a =∞ contradicting Equation (6d)

h(a) � a• (u ∈ L(v).path) The goal is trivial.

Having established that the two SPPFs are choice-approximate we leverageLemma 3 to obtain label approximate and conclude the proof.

Definition 5 (Failures approximation). Given an SPPF and its abstraction,

for a selection of concrete link failures F : V → {0, 1} we write F : V → {0, 1}to denote a selection of abstract link failures such that:

F(e) = 1 ⇐⇒ ∃e.e 7→ e ∧ F(e) = 1

Theorem 3. Given a well-formed SPPF = (G,A, ad,≺, trans, k) and its effec-

tive approximation SPPF = (G, A, ad, ≺, trans, k), for any solution S ∈ SPPF

there exists a solution S ∈ SPPF, such that their labelling functions are labelapproximate.


Proof. Suppose S = 〈L,F〉. We pick F as described in Definition 5. It is easy tosee that ∑

e∈EF(e) ≤ K =⇒

∑e∈E

F(e) ≤ K

Since the transfer function of SPPF is strictly monotonic and isotonic, weknow by [25] that it converges to a unique labelling solution L. By Theorem 2

and Lemma 3 L and L are label approximate.

E cient veri cation of network fault tolerance via ...

Documents