P4-IPsec: Implementation of IPsec Gateways in P4 with SDN ... · Frederik Hauser, Marco Häberle, Mark Schmidt, Michael Menth Abstract—In this paper we propose P4-IPsec which follows

P4-IPsec: Implementation of IPsec Gateways in P4with SDN Control for Host-to-Site Scenarios

Frederik Hauser, Marco Häberle, Mark Schmidt, Michael Menth

Abstract—In this paper we propose P4-IPsec which follows thesoftware-defined networking (SDN) paradigm. It comprises a P4-based implementation of an IPsec gateway, a client agent, and acontroller-based, IKE-less signalling between them. P4-IPsec fea-tures the Encapsulation Security Payload (ESP) protocol, tunnelmode, and various cipher suites for host-to-site virtual privatenetworks (VPNs). We consider the use case of a roadwarrior andmultiple IPsec gateways steered by the same controller. P4-IPsecsupports on-demand VPN which sets up tunnels to appropriateresources within these sites when requested by applications.To validate the P4-based approach for IPsec gateways, weprovide three prototypes leveraging the software switch BMv2,the NetFPGA SUME card, and the Edgecore Wedge 100BF-32Xswitch as P4 targets. For the latter, we perform a performanceevaluation giving experimental results on throughput and delay.

Index Terms—IPsec, P4, Software-Defined Networking, VPN

I. INTRODUCTION

Internet Protocol Security (IPsec) is a widespread IETFstandard for virtual private networks (VPNs). It protects Layer3 data that is transmitted over insecure networks such as theInternet by message authentication and optional encryption.IPsec supports VPNs between two hosts, hosts and networks,and networks. IPsec clients are part of all modern operatingsystems, a large selection of open-source and proprietary IPsecgateways are available. IPsec is criticized for its complexityin architecture, protocols, implementation, and configuration.However, it is still one of the most widely-used VPN tech-nologies nowadays.

Software-defined networking (SDN) splits the strong bind-ing between data and control plane and facilitates simplifieddeployment of distributed technologies such as IPsec. How-ever, the currently wide-spread SDN switches, e.g., OpenFlowswitches, have a fixed-function data plane with a limited setof functions. This can be extended by outsourcing functionsto the control plane, but performance overhead is critical.Programmable data planes, e.g., as offered by P4, are a game-changer as data plane behaviour can be described in a high-level programming language. Thereby, new functionality canbe implemented and deployed to a large number of softwareand hardware switches. In previous work [1], we implementedMACsec and secure link discovery in P4 and demonstrated,how this new flexibility can help to re-implement common

Frederik Hauser, Marco Häberle, Mark Schmidt, and Michael Menthare with Chair of Communication Networks, University of Tuebin-gen, Tuebingen, Germany. E-mail: {frederik.hauser,marco.haeberle,mark-thomas.schmidt,menth}@uni-tuebingen.de

This work was supported by the bwNET100G+ project which is funded bythe Ministry of Science, Research and the Arts Baden-Württemberg (MWK).The authors alone are responsible for the content of this paper.

concepts and protocols in a novel fashion including newfeatures. Instead of complex configuration on a large numberof network devices, MACsec is automatically deployed on alllinks between P4 switches.

In this paper, we propose a novel concept and imple-mentation of IPsec in P4 and call it P4-IPsec. It featuresthe Encapsulation Security Payload (ESP) protocol in tunnelmode with different cipher suites. It comprises a Linux-basedclient module and a P4-based IPsec gateway implementationto support host-to-site scenarios. As these components aresteered by a centralized control plane through an authenticatedand encrypted control connection, complex IKE-based keyexchange protocols are substituted by simple setup, re-keying,and teardown procedures for IPsec tunnels. IPsec’s secu-rity policy database (SPD) and security association database(SAD) are converted to appropriate match-and-action tablesof the P4-based IPsec gateway. Converting P4 switches intoIPsec gateways provides the opportunity to operate multiple ofthem close to protected resources that are otherwise shieldedfrom external access. This limits the size of the perimeterand improves security through better isolation compared totoday’s solutions with a single VPN concentrator for a largerperimeter. Our second contribution, on-demand VPN, is tai-lored to a use case with multiple IPsec gateways steered bya single controller. It may be cumbersome for users to setup the appropriate VPN connection before utilizing a desiredresource. With on-demand VPN, the client agent detectswhether a protected resource is requested and automaticallyopens a VPN connection to the appropriate IPsec gateway.A third contribution of this work is the validation of P4-IPsec through prototypical implementations on the bmv2 P4software switch and two P4 hardware platforms, the NetFPGASUME board and the Edgecore Wedge 100BF-32X switch thatfeatures a Tofino ASIC. Furthermore, we perform an extensiveevaluation of the implementation for the Edgecore Wedgeswitch with two alternative implementation variants regardingthroughput and delay. The rest of the paper is structured asfollows. We first review technical background on IPsec inSection II and data plane programming with P4 (Section III).We then we give an overview and categorization of existingcontroller-based operation and data plane implementations ofIPsec in Section IV and Section V. We describe the data planeimplementation of our proposed P4-based IPsec gateway inSection VI. The control plane concept comprising controllerand client module for on-demand VPN are presented inSection VII. For validation purposes, we realize these conceptsfor P4-IPsec in a Mininet environment with the BMv2 P4software switch Section VIII. In addition, we provide dataplane implementations of P4-IPsec on the NetFPGA SUME

arX

iv:1

907.

0359

3v1

[cs

.NI]

8 J

ul 2

019

platform in Section IX and on the Edgecore Wedge 100BF-32X switch that features a Tofino ASIC in Section X. For thelatter we evaluate throughput and delay for two alternativeimplementation variants. Finally, Section XI concludes thiswork.

II. VPN WITH IPSEC: FOUNDATIONS

We give an introduction on virtual private networks (VPN)and provide an overview on IPsec. We discuss advantages anddisadvantages of IPsec and briefly introduce alternative VPNtechnologies.

A. Virtual Private Networks (VPNs)

VPNs extend private networks across a public network suchas the Internet. To protect the private network access andtransmitted data, VPNs leverage methods for authentication,authorization, encryption, and integrity validation. Figure 1depicts three usage scenarios for VPNs. In the host-to-host (1)scenario, two remote peers are connected by a VPN that is setup directly between both hosts. In the host-to-site (2) scenario,a remote peer connects to an internal network. The VPN is setup between the remote host and a VPN gateway that connectsthe internal network to the Internet. The remote host receivesan IP address from the internal network so that it can sendand receive network packets as if it is part of the internalnetwork. In the site-to-site (3) scenario, two internal networksare connected by a VPN that is set up between two VPNgateways that connect both internal networks to the Internet.

(1)

GW(2)

(3)

H

H…

HH

H

GWH

H…GW

H

H…

Fig. 1: Three VPN usage scenarios: host-to-host (1), host-to-site (2), and site-to-site (3).

B. Overview of IPsec

Internet Protocol Security (IPsec) introduces authenticationand encryption on Layer 3. It features host-to-host, host-to-site,and site-to-site scenarios and is standardized by the IETF. Itsarchitecture and functionality is described in RFC 4301 [2].

1) Protocols: IPsec includes two protocols. The Authen-tication Header protocol (AH, specified in RFC 4302 [3])ensures sender authenticity and packet integrity. Hash func-tions and a shared key are used to calculate an integritycheck value (ICV) through keyed-hash message authenticationcodes (HMACs) such as HMAC-SHA256. AH also leveragespacket sequence numbers to provide protection against replayattacks. The Encapsulating Security Payload (ESP, specified inRFC 4302 [4]) protocol ensures packet confidentiality through

symmetric encryption. Like AH, ESP also ensures sender au-thenticity, integrity, and protection against replay attacks. ESPsupports various symmetric ciphers such as 3DES, Blowfish,and AES. If pure encryption ciphers such as AES in cipherblock chaining mode or counter mode are used, additionalintegrity algorithms such as SHA are also applied. As analternative, authenticated encryption (AE) ciphers may be usedwhich provide both confidentiality and authenticity protection.AES in Galois/Counter Mode (GCM) is the most commonAES cipher and its application in IPsec is described in RFC4106 [5].

2) Operation Modes: Figure 2 depicts the two operationmodes of IPsec. In transport mode (1), the IPsec header isinserted between the IP header and IP payload. Transport modeprovides end-to-end protection and introduces little overheadbut can be applied only in host-to-host scenarios. In tunnelmode (2), IP packets are encapsulated in IP packets with anIPsec header. The outer IP header holds the IP addresses ofthe hosts or gateways that perform IPsec, the inner IP packetmay be addressed to a host behind the IPsec gateway. Tunnelmode is required for host-to-site and site-to-site scenarios.

Outer IP header

(1) IP header IP payloadIPsec header

(2) IP payloadIP headerIPsec header

Fig. 2: IPsec operation modes: Transport mode (1) for host-to-host VPNs andtunnel mode (2) for host-to-site and site-to-site VPNs.

3) Packet Processing: Figure 3 depicts the base compo-nents of IPsec on a host or VPN gateway. IPsec clearly sep-arates connection management, key management, and packetprocessing. First, IP packets are processed by a Security PolicyDatabase (SPD). It is configured with a list of security policiesthat define whether a IP packet needs to be protected byIPsec, processed by IP forwarding without IPsec, or dropped.The SPD is configured by the control plane, e.g., by anadministrator or network management tool. SPD entries forIPsec connections point to the protocol (AH or ESP), theoperation mode (transport or tunnel), and the cipher suite to beused. They refer to a Security Association (SA) that containsall required data for IPsec processing, e.g., cipher keys, validsequence numbers, and a SA lifetime. SAs are part of the SADatabase (SAD) that is either configured manually or managedby an Internet Key Exchange (IKE) daemon that runs on thecontrol plane. The IKE protocol (specified in RFC 2409 [6])was introduced with IPsec. It authenticates both peers, sets upa secure channel for key exchange, and negotiates SAs. Today,its successor IKEv2 (specified in RFC 7296 [7]) is typicallyused and solves complexity and incompatibility issues of IKE.

C. Evaluation

IPsec is probably the most widespread VPN technologynowadays. IPsec clients are part of all current operating sys-tems for computers, and mobile devices. IPsec gateway func-tionality is part of server operating systems, network hardwareappliances such as firewalls or routers, or dedicated boxes.

AH/ESP

Key exchange

Packet

IKE

IPsec

Configuration

SPD SAD AH/ESP …

IKE

IPsec peer IPsec peer

Fig. 3: IPsec packet processing between two IPsec peers. Each peer featuresa SPD, SAD, and AH/ESP mechanisms on the data plane. The SPD isconfigured by the control plane, the SAD is either configured manually ormanaged by an IKE daemon.

However, IPsec is highly criticized for its complexity sincemany years. The most encompassing analysis was performedby Ferguson and Schneier [8] in 2003. The authors especiallycriticize the redundancy of functionality caused by AH, ESP,and the two operation modes, the complex key exchange withIKE, and the complex configuration caused by the SPD andSAD. However, those issues can be easily solved by omittingtransport mode and AH, using AE ciphers in ESP with tunnelmode, and applying a less complex protocol for key exchange.

D. Alternative VPN Technologies

Application-layer VPNs that run on top of transport pro-tocols such as TCP or UDP are promoted as less-complexalternatives to IPsec. OpenVPN [9] is a popular open-sourceVPN that features a custom security protocol transmitted viaTCP or UDP. It leverages RSA for authentication and a customprotocol to secure the VPN connection. Session keys forencryption are either defined statically or generated in a Diffie-Hellman (DH) key exchange. OpenVPN supports VPNs onLayer 2 and Layer 3 networks. Just like IPsec, it supportshost-to-host, host-to-site, and site-to-site scenarios. However,the throughput performance is far beyond IPsec. The SecureSocket Tunneling Protocol (SSTP) is a host-to-host VPNintroduced by Microsoft. It leverages TLS to secure the VPNconnection and provides support for several authenticationmethods, e.g., X.509 certificates and EAP methods. Due toits restriction to host-to-host scenarios, applicability is highlylimited. WireGuard [10] is the most recent open-source VPNalternative to IPsec. It features host-to-host, host-to-site, andsite-to-site VPNs with a custom security protocol transmittedvia UDP. WireGuard has been designed to be less complexthan IPsec. It implements only a fixed set of cryptographymechanisms for authentication, key exchange, encryption, andintegrity checks. Its throughput performance and latency issimilar to IPsec, but it lacks any management functionsto build host-to-site or site-to-site setups. However, we seesimilar advantages and a similar level of complexity whenusing IPsec with a reduced set of functionality as proposed byFerguson and Schneier.

III. P4: A PROGRAMMABLE DATA PLANE

We give a short introduction to programmable data planesin general. Then, we give an overview of P4 programmingand deployment, major P4 language components, and the P4runtime control.

A. Programmable Network Data Planes

Programmable network data planes facilitate reconfigurationof the packet processing functionality of a network device. Bi-fulco et al. [11] provide an encompassing overview. Hardwareplatforms that support programmability are typically field-programmable gate arrays (FPGAs), i.e., integrated circuitsthat can be programmed after manufacturing, or networkprocessing units (NPUs) with physical network ports.

B. P4

P4 is a domain-specific language that allows the specifica-tion of packet forwarding behaviour of programmable networkdata planes. P4 was first published as research paper in 2014[12]. Today, P4 is a project of the Open Network Foundation(ONF). Its latest specification is P416 [13]. We describe P4’score concepts and components as visualized in Figure 4.

1) P4 Programming & Deployment: P4 programs includethe forwarding behaviour of a switch described with P4language components. P4 programs refer to a particular P4architecture that represents the programming model of aswitch. P4 architectures are implemented by software orhardware switches called P4 targets. Target-specific compilersthen translate P4 programs into code that can be executed onthe P4 target.

2) P4 Language Components: P416 features six core lan-guage components for describing packet forwarding behaviour.Header types describe packet header formats by an orderedcollection of base types. For instance, an IPv4 header isdescribed by bit vectors, e.g., for the source and destinationaddress, and protocol field. The parser identifies and extractspacket data by applying predefined sequences based on headertypes. For instance, the value of the protocol field in anIP packet determines the parsing sequence for the followingpacket header format, e.g., IPsec, TCP, or UDP. Match-and-action tables hold a list of user-defined keys that refer toparticular actions that modify packets. Externs are functionswith a clearly-defined interface provided by a P4 target thatcan be used within P4 programs. An example is a function thatencrypts given data using AES-GCM. The deparser assemblesthe headers back into a well-formed network packet that canbe sent out via an egress port of the switch.

P4 target

Dep

arse

r

Extern

Control plane

Key 2 Action

Match-and-action tables

Key 1

P4 program

Userspecified

Runtime APICompiler

Pars

er

P4architecture

Ingr

ess

Egre

ssHea

der

type

s

Fig. 4: Concept and core components of P416 (similar to [1]).

3) P4 Runtime Control: The forwarding behaviour of P4programs can be controlled in runtime by modifying thematch-and-action tables via a control plane interface. Basicexamples are command-line interfaces (CLIs) or custom APIsas part of a P4 target. As an alternative, the P4 Runtime APIintroduces a standardized control-plane interface for table ma-nipulations and packet exchange between the data and controlplane. It relies on gRPC [14], protocol buffer data structures[15], and connection security with TLS using certificate-basedmutual authentication.

IV. CONTROLLER-BASED OPERATION OF IPSEC:FOUNDATIONS & RELATED WORK

We first describe foundations and use cases on controller-based operation of IPsec. Afterwards, we review and cate-gorize related work by its operation modes and southboundprotocols for data plane management.

A. Overview

Controller-based operation of IPsec applies the principle ofSDN to IPsec deployment. Functions for connection and keymanagement, e.g., SPD and SAD maintenance, are outsourcedto a control plane. This introduces several advantages overtraditional deployment. First, the control plane has an encom-passing view on the network topology with all devices. It canmonitor utilization and detect outages for reliable operation.Second, the centralized control plane features northboundinterfaces for management applications and southbound inter-faces for controlling data plane devices. Instead of manual per-device configuration, VPN management can be performed ona high abstraction layer with policy languages that allow rulevalidation. Last, the centralized control plane offers flexibilityso that VPN operation can be extended by other mechanisms,e.g., user authentication with 802.1X [16].

B. Use Cases

In the following, we describe three use cases for controller-based operation of IPsec.

1) SD-WAN: Large organizations with distributed locationsrequire network connectivity between the different sites. Asdedicated links are expensive, site-to-site IPsec-VPNs overprovider networks are increasingly used. However, manuallysetting up VPN connections between all branches is time-intensive and complex. SD-WAN [16]–[18] proposes IPsecdata plane functionality as part of hardware appliances orsoftware modules at the perimeter of the different sites of theorganization. Then, a centralized controller automatically setsup and maintains IPsec-VPN connections.

2) Cloud Provider Networks: In many cases, internal ser-vices offered by a public or private cloud provider need tobe accessed from within networks of an organization. Again,site-to-site IPsec-VPN tunnels are a cost-efficient alternativeto dedicated links. Administrators define IPsec-VPN gatewaysvia a cloud management interface. Then, the cloud orchestratordeploys IPsec-VPN gateways as virtual network function onthe cloud provider’s infrastructure. Its run-time operation is

managed by a controller. In addition, controller-based opera-tion of IPsec can be also used to dynamically connect differentcloud networks by a multi-cloud orchestrator [19].

3) Dynamic VPN Setup: Managing many IPsec-VPN con-nections to different hosts or services on a client host can becumbersome. Dynamic VPN setup performed by a controllertakes over the tasks of tunnel setup and management. In [20],users request VPN access to a particular network device fromthe controller. It then automatically sets up a VPN tunnel to theremote domain. The authors of [21] combine dynamic VPNsetup with authentication and authorization to automaticallydeploy IPsec-VPN tunnels between IoT network devices.

C. Operation Modes for Data Plane Management

We describe three operation modes for controller-based dataplane management of IPsec as visualized in Figure 5.

1) IKE on the Data Plane: IPsec data plane nodes stillfeature an IKE daemon. To reduce the message exchanges inan IKE process, it is preconfigured by the controller. In [21],[22], the controller pre-configures authentication keys, in [23],the controller distributes Diffie-Hellman public values to all as-sociated IPsec data plane nodes. An IETF draft [24] describesthe same mechanism as one of two alternative mechanismsfor controller-based operation of IPsec. The authors of [18]propose a similar approach that is compatible to older IKEdaemons that only support IKEv1.

2) IKE on the Control Plane: The authors of [25] relocatethe IKE daemon to the control plane. It performs key exchangewith peers and manages the SAD of the IPsec data planenodes. This approach even supports migration schemes so thatthe SA can be transferred to other IPsec data plane nodes, e.g.,in case of fail-over or load-balancing operations.

3) IKE-less Operation: The authors of [21], [24], [26], and[27] propose SA management without IKE. The controllergenerates keying material and sets up SAs in the SAD ofassociated IPsec data plane nodes.

IKE

ESP + AH

IPsec control

Management

1) IKE on thedata plane

Dat

a pl

ane

Cont

rol

plan

e

IKE

ESP + AH

IPsec control

Management

ESP + AH

IPsec control

Management

2) IKE on the control plane

3) IKE-less operation

SPDSAD SPDSAD SPDSAD

Fig. 5: Operation Modes for Data Plane Management of IPsec.

D. Southbound Protocols for Data Plane Management

On legacy IPsec devices, e.g. [28], SNMP is used for basicconfiguration and monitoring. The authors of [18] extend thisusage in making an IKE daemon manageable by SNMP aswell. In [17], SSH is used as southbound interface to manage

and monitor IPsec data plane nodes. The work in [24] usesNETCONF with YANG configuration models. In additionto the southbound protocol, they consider east-/westboundinterfaces for controller-to-controller communication via dif-ferent domains. Aragon et al. in [21] used OAuth 2.0 todeliver configuration data within authorization messages. In[26], OpenFlow is extended using Experimenter messages. Thework in [27] leverages BGP. Li and Mao [16] use a customsouthbound protocol to interface an IPsec module on an OpenvSwitch. The authors of [29] propose a custom southboundprotocol with notification, configuration, and query messagesthat are transmitted via TCP or TLS.

E. Positioning of Own Work

P4-IPsec features controller-based operation without IKE.The SPD and SAD as part of the data plane are managedby a controller via the P4 Runtime API. We introduce a newapproach for dynamic VPN setup that establishes on-demandIPsec tunnels for requested network resources without userinteraction.

V. DATA PLANE IMPLEMENTATION OF IPSEC: RELATEDWORK

In the following, we review related work on data planeimplementations for IPsec gateways. We describe IPsec soft-ware implementations, hardware acceleration techniques, andhardware implementations In the end, we position our work.

A. IPsec Software Implementations

IPsec is part of the system kernel of Linux and BSD sincemany years. Windows server supports IPsec since Version2000 [30]. The performance of pure software implementationsdepends on the hardware, the used cryptographic algorithms,and the average packet size.

Many optimization techniques aim at improving the packetprocessing of operating systems. The Data Plane DevelopmentKit (DPDK) [31], Netmap [32], and PF_RING [33] optimizethe Linux network stack to improve packet I/O rates. Theauthors of [34] and [35] describe mechanisms to distributeIPsec processing on multiple CPU cores while PacketShader[36] proposes to improve IPsec throughput by using the GPU.Gallenmüller et al. [37] compare packet I/O improvementmechanisms in an extensive study. Most of the described op-timization mechanisms are only applicable to Linux operatingsystems.

B. Hardware Acceleration for Software Implementations

The performance of IPsec software implementations can beimproved by offloading functionality to hardware components.Figure 6 depicts cryptography hardware acceleration (1) andIPsec hardware acceleration (2) in comparison to pure softwareimplementations.

HW

Crypto

IKE IPsec control

SPDSAD

ESP + AH

SW

Software implementation

Crypto

IKE IPsec control

SPDSAD

ESP + AH

(1) Cryptography hardware acceleration

Crypto

IKE IPsec control

SPDSAD

ESP + AH

(2) IPsec hardware acceleration

SWH

W

Fig. 6: Hardware acceleration techniques for IPsec software implementations.

1) Cryptography Hardware Acceleration: IPsec throughputcan be improved by offloading complex encryption and de-cryption mechanisms to cryptographic hardware accelerationunits. Current CPU architectures from Intel, AMD, and ARMinclude functions for encryption and decryption. AES-NI [38]and the ARMv8 Cryptographic Extension [39] are examplesfor AES instruction sets that can be used instead of AESsoftware implementations. Besides, system-on-a-chip (SoC)platforms and circuit boards may contain chips for offloadingcryptographic processing. The Marvell Cryptographic Enginesand Security Accelerator (CESA) is an example for a cryp-tographic processor that is part of SoC platforms such asthe Marvell ARMADA 38x family [40]. Intel QuickAssist[41] is a cryptographic acceleration processor that can beembedded into a mainboard’s chipset. Processors for offload-ing cryptographic processing can be also part of an extensioncircuit board that is connected to the mainboard, e.g., via PCI.For example, the Intel QuickAssist Server Acceleration Card[41] embeds the QuickAssist processor on a PCIe extensionboard. Several vendors [42], [43] supply implementations forcryptographic algorithms as intellectual property (IP) coresthat can be executed on FPGAs. For cryptography hardwareacceleration, an FPGA running the IP core is part of a PCIeextension board in a computer.

2) IPsec Hardware Acceleration: IPsec throughput can bealso improved by offloading IPsec processing or parts ofit to IPsec hardware acceleration units. Application-SpecificIntegrated Circuits (ASICs) are integrated circuits that aredesigned and manufactured to execute specific functions. Forexample, [44] and [45] describe IPsec hardware acceleratorsthat are built as ASICs to implement particular IPsec cryptosuites. Besides, NPUs are programmable integrated circuitsthat are optimized for networking functions. The authors of[46] and [47] describe implementations of IPsec functionson multi-core NPUs. Accellerated Processing Units (APUs)combine a CPU and GPU with physically shared memory.PIPSEA [48] is an IPsec implementation for an APU thatleverages OpenGL for IPsec processing. In addition, sev-eral works [49], [50] implemented parts of the IPsec dataplane processing on FPGAs. The functions are then usedby a software implementation that interfaces the FPGAs viaPCIe. However, development for FPGA requires expensivedevelopment software environments besides knowledge andexperiences in hardware programming.

C. IPsec Hardware Implementations

In contrast to the previous mechanisms, IPsec processingmay be completely implemented on hardware platforms. IPsec

processing of proprietary VPN gateways, e.g., as sold by Ciscoor Juniper, mostly feature high-performance implementationsof IPsec. Due to its not public architectural details, we cannotget insight into technical details. The authors of [51] and[52] describe IPsec data plane implementations for FPGAs.IPsec processing is performed on the FPGA, its SPD andSAD are maintained by an external control plane. In 2016,a Xilinx employee reported on the P4-Development mailinglist [53] that IPsec was successfully implemented in PX[54], a high-level domain-specific programming language forprogrammable data planes.

D. Positioning of Own Work

We propose an implementation of IPsec in P4 that featuresESP in tunnel mode with support for different cipher suites.For the BMv2 and NetFPGA SUME board, we implementcipher suites as P4 externs. For the Edgecore Wedge 100BF-32X with a Tofino ASIC, we implement the cipher suites insoftware on the main CPU module and provide an interface tothe processing pipeline. The implementation is managed by acontrol plane that maintains the SPD and SAD.

VI. P4-IPSEC: DATA PLANE IMPLEMENTATION

In this section, we describe our data plane implementationof IPsec in P4. We provide an overview of the proposed P4processing pipeline and describe its components.

A. Overview

The proposed data plane implementation of P4-IPsec fea-tures ESP in tunnel mode with support for different ciphersuites. This simplification of the original specification corre-sponds to today’s applications of IPsec and was also proposedby [8] (see Section II-C). We adopt IPsec components suchas the SPD and SAD and implement IPv4 packet forwardingwith longest-prefix matching.

B. P4 Processing Pipeline

Figure 7 depicts packet processing in the proposed process-ing pipeline of a P4-IPsec switch. For ease of understanding,we have grouped together functionalities in function blocks(FBs). Runtime behaviour of the data plane can be managed bymanipulating the four match-and-action tables (MATs), e.g.,with a CLI on the P4 switch or an interface to a control plane.

We now overview packet processing of the proposed P4processing pipeline. When a packet arrives via the ingress,the P4 parser first extracts the packet headers (1). If the packethas an IPv4 header but no ESP header, it is forwarded to thesecurity policy (SP) matching (2) function block. It determineswhether a particular IPv4 packet should be dropped (2a),further processed by IP forwarding (2b), or sent to the IPsecencryption (3) function block. In the IP forwarding FB, thepacket is either dropped due to a missing entry in the routingtable (2c) or sent out (2d). The deparser (4) reassembles allheaders and re-calculates the IPv4 checksum as some fields,e.g., the TTL, is changed. If the packet has an ESP header,it is forwarded to the IPsec decryption (3) function block. It

validates the packet’s authenticity, decrypts the ESP message,and extracts the original IPv4 packet. As before, the IPv4packet is either dropped (2c) or sent out (2d) by the IPforwarding function block.

Pars

er

Dep

arse

r

Ingr

ess

Egre

ss

P4 processing pipeline

IPsecencryption

SPmatching

Drop

IPsecdecryption

(2)

FB

IP forwarding

IPv4, no ESP

ESP

(3)

(2b)

(2a)

(2d)(2c)

(3)(3a)(1

)

(4)

FB

FB

FB

Fig. 7: Data plane processing pipeline of P4-IPsec. For ease of understanding,related functionalities are grouped together as function block (FB).

C. Function Blocks: SP Matching and IP Forwarding

Figure 8 visualizes the operation of the function blocks ofSP matching and IP forwarding in detail.

1) SP Matching: We integrate the security policy database(SPD) introduced in Section II with a SP MAT. Its entriesdetermine if a packet that belongs to a particular IP flow shouldbe either discarded (1a), sent to the functional part of IPsecencryption (1b), or processed by IP forwarding (1c). Therefore,packets are matched using their IPv4 source address, IPv4destination address, and optionally IP protocol, and TCP/UDPsource and destination ports and assigned to one of the threeactions. If the SP MAT yields no match, the packet is discarded(1c).

IP hdr+ data

SPMAT

IP packet

Drop

IP srcIP dstTCP/UDPPort srcPort dst

IPsecencrypt

(1a)

SP matching IP forwarding

(1b)

(1c) (3b)

(2)

(3a)

LPMMAT

FB FB

Fig. 8: Function blocks (FBs) of SP matching and IP forwarding with LPM.Each function block features a MAT.

2) IP Forwarding: Packets received from the SP matching(1c) or IPsec encryption (2) FB are further processed by alongest-prefix MAT. It implements IP packet forwarding to thenext hop via a particular output port of the switch. The longest-prefix MAT maps IPv4 destination addresses to a particularoutput port of the switch and the MAC address of the next hop.If it yields no match for the given IPv4 destination address, thepacket is dropped (3a). If it yields a match for the given IPv4destination address (3b), the following steps are applied. First,the packet’s source MAC address is set to the MAC address ofthe P4 switch. Second, the packet’s destination MAC addressis set to the MAC address of the next hop. Third, the IPv4TTL of the packet is decreased by 1. Last, the packet is passedto the deparser.

D. Function Blocks: IPsec Encryption and Decryption

The FBs of IPsec encryption and IPsec decryption consistsof cipher suite externs and security association (SA) MATs.

1) Cipher Suite Externs: P4 does not feature cryptographicfunctions for encryption, decryption, or message authentica-tion that might be used to implement IPsec. Therefore, weleverage P4 externs (see Section III) to implement IPseccipher suites that consist of a particular set of cryptographicalgorithms. Each cipher suite is implemented by two P4externs, one for encryption and one for decryption. Figure 9depicts IPsec encryption and decryption using a cipher suiteof AES-CTR for encryption and HMAC-MD5 for messageauthentication. In the IPsec encryption (1) FB, the cipher suiteextern receives the original IP packet, the outer IP source anddestination addresses, the SPI, and the keys for AES-CTRand HMAC-MD5. The encryption cipher suite extern encryptsthe original IP packet, calculates the message authenticationcode, and creates an IPsec packet. In the IPsec decryption (2)FB, the cipher suite extern receives the IPsec packet and thekeys for AES-CTR and HMAC-MD5. The decryption ciphersuite extern decrypts the IPsec packet, validates the messageauthentication code, and returns the original IP packet.

dec(

AES

-CT

R,H

MA

C-M

D5)

Outer IP hdrESP header IP hdr + dataESP trailerESP auth.

IP hdr+ data

IP packet

IPsec packet

enc(

AES

-CT

R,H

MA

C-M

D5)

…

Key

(aut

h.)

Key

(enc

rypt

.)Outer IP src Outer IP dstSPIKey (auth.)Key (crypt.)

…

IP hdr+ data

(1) IPsec encryption (2) IPsec decryption

FBFB

Fig. 9: Cipher suite externs that implement IPsec encryption and decryption.

2) SA Match-and-Action Tables: Cipher suite externs re-ceive the required parameters and keying material from SAMATs. That functionality corresponds to the SAD as in-troduced in Section II-B3. As SAs are unidirectional, weintroduce an encryption SA MAT and a decryption SA MAT.Figure 10 depicts IPsec encryption and decryption using thepreviously described cipher suite and the SA MAT. In the IPsecencryption (1) FB, packets are matched using the IP source anddestination address. The SA entry then invokes the particularcipher suite extern with the required data and keying material.If a packet does not match an SA entry, it is directly discarded.In the IPsec decryption (2) FB, packets are matched using theSPI. Again, the SA entry invokes the particular cipher suiteextern with the required data and keying material. If it yieldsno match, the packet is discarded due to a missing SA.

VII. P4-IPSEC: CONTROL PLANE IMPLEMENTATION

In this section, we describe the control plane of P4-IPsec.We describe host-to-site standard operation and our novelapproach of on-demand VPN setup.

A. Regular Host-to-Site Operation

Our control plane implementation focuses on the host-to-site usage scenario as described in Section II-A. Figure 11

IPse

c pa

cket

Ciph

er s

uite

ex

tern

(de

c)

IP p

acke

t

Dec

rypt

ion

SA M

AT

…

Encr

yptio

n SA

MA

T

SPIDro

p

IP p

acke

t

(2) IPsec decryption

Ciph

er s

uite

ex

tern

(en

c)

…

(1) IPsec encryption

Dro

p

IP sr

c +

dst

FB FB

Fig. 10: Security association (SA) MATs for IPsec encryption and decryption.

depicts an overview. The P4 switch implements IPsec andconnects an internal network to the Internet. It is managed by acontroller via the P4 Runtime API and holds gateway profilesfor each P4 switch that acts as IPsec gateway. Gateway (GW)profiles contain the following information. IPsec propertiesinclude the public IP address of the P4 switch as IPsec gatewayaddress to the shared network resource and the private IPv4network. In addition, it holds a list of user identities thatare permitted to access the VPN. More sophisticated ways ofmanaging access permissions could be realized by using LDAPor RADIUS. On the public IP address of the P4 switch, theinternal network, and a list of user identities that are permittedto access the network. The roadwarrior host runs an IPsecagent for interacting with the controller.

Controller

HostHost

Internet

Internal LAN

IPsec agent

Roadwarrior

IPsec properties

Gateway: 134.2.11.123Network: 10.0.20.1/24

johndoe…

AccessGate

way

prof

iles

Fig. 11: Overview on regular host-to-site operation between a roadwarriorhost and a P4 switch. IPsec operation is managed by a controller.

Following Figure 12, we describe the steps performed bythe IPsec agent to setup, renew, and teardown a VPN tunnelwith the help of the controller.

1) Controller Connection Setup: After its start, the IPsecagent establishes a connection to the controller via its IPv4address or FQDN (1). To prevent unauthorized access, theIPsec agent and controller perform mutual authentication.Therefore, the IPsec agent validates the server certificate ofthe controller and provides its own certificate that includes itsidentity. The controller checks if the identity is part of a listof authorized users in the requested GW profile and grants theconnection attempt. To protect against eavesdropping, packetmanipulation, and replay attacks, the communication betweenIPsec agent and controller is protected with encryption andmessage authentication.

2) Tunnel Setup: To set up an IPsec tunnel to a particularP4 switch, the IPsec agent sends a tunnel setup request withthe public IPv4 address of the target P4 switch to the controller(2a). If the user is permitted to set up an IPsec tunnel tothe particular P4 switch, the controller creates and installs

the IPsec tunnel. As a first step, it creates keying materialfor the chosen cipher suite and generates the SAs. To setup the new IPsec tunnel on the P4 switch, the controllerupdates the encryption SA MAT, decryption SA MAT, andSP MAT by entries for the new tunnel (2b). Once the tablemanipulations are confirmed by the P4 switch, the controllersends the configuration parameters to the IPsec agent (2c).

3) Tunnel Renewal: IPsec SAs have a limited lifetime, i.e.,keying material needs to be renewed on a regular basis. There-fore, the IPsec agent continuously monitors packet number andtimeout limits. Once a threshold has been exceeded, the IPsecagent sends a tunnel renewal request to the controller (3a). Thecontroller creates new SAs with a new SPI and installs themon the P4 switch. First, it updates the decryption SA MAT(3b). As entries in the decryption SA MAT are identified bythe SPI, both old and new SA entries can coexist. Second, thecontroller replaces the current SA in the encryption SA MATby the new SA (3c). A direct replacement is necessary as aMAT cannot contain two entries with the same match key. Ifthe MAT modifications are confirmed by the P4 switch, thecontroller delivers the new SA for encryption and decryptionto the IPsec agent as well (3d). Last, the controller deletes theold SA in the decryption SA MAT once it is ensured that nomore packets encrypted with the old SA can reach the P4-switch (3e).

4) Tunnel Deletion: If an IPsec tunnels should be deleted,the controller simply removes the associated entries in the SP,encryption SA, and decryption SA MAT. Tunnel deletion istriggered by the user or by timeout limits. The IPsec agentsends a tunnel deletion request (4a) to the controller. It deletesall related entries in the MATs of the P4 switch (4b). If theMAT modifications are confirmed, it sends a client tunneldeletion command to the IPsec agent (4c).

(1) Controller connection setup

(2a) Tunnel setup request (2b) Setup SA, update SPD

(3a) Tunnel renewal request (3b) Setup new decryption SA

(4a) Tunnel deletion request (4b) Switch tunnel deletion

(3c) Replace encryption SA

(2c) Client tunnel setup

(3d) Renew client SAs

(4c) Client tunnel deletion

IPsec agent Controller P4 switch

(3e) Remove old decryption SA

Fig. 12: Steps performed by the IPsec agent to setup, renew, and teardown aIPsec tunnel with the help of the controller.

B. On-Demand Host-to-Site Operation

P4-IPsec introduces IPsec functionality on P4 switches.Therefore, IPsec connections may terminate close to protectedresources that should be accessible via VPN tunnels. Thisdistributes high processing loads of current deployments with

VPN concentrators at the perimeter to multiple P4 switchesthat are steered by a controller. In addition, it improves securitythrough better isolation.

1) Use Case Description: As configuring many VPN tun-nels for different protected resources may be cumbersomefor users, we introduce on-demand VPN. Figure 13 depictsan overview. An internal server is connected to a P4 switchthat has a public IPv4 address reachable from the Internet.The switches and routers in between forward IPsec trafficaddressed to that public IPv4 address to the P4 switch. Weextend the IPsec agent on the roadwarrior host by a DNS-based tunnel setup detection mechanism. We extend the GWprofiles from default host-to-site operation by a FQDN of theprotected resource and its private IPv4 address in the internalnetwork to resource profiles. If the IPsec agent detectes that aVPN tunnel needs to be established, it sends an IPsec tunnelsetup request to the controller that performs IPsec tunnel setupas described before.

IPsec

Controller

S

Internet

Host

IPsec agent

Roadwarrior

Tunnel setup

Tunnel setup

IPsec properties

Gateway: 134.2.11.123Private IP: 10.0.20.12

johndoe…

AccessOn-

Dem

and

Gate

way

prof

iles

srv-01.uni-tuebingen.deFQDN

DNS server

Tunn

el se

tup

dete

ctio

n

Fig. 13: Overview of the novel on-demand host-to-site operation between anroadwarrior host and a P4 switch. IPsec operation is managed by a controller.

2) DNS-Based Lookup of VPN Profiles: We propose thefollowing mechanism for DNS-based tunnel setup detectionon the IPsec client. We install text (TXT) records on a DNSserver that indicate whether a FQDN is only reachable via aIPsec tunnel. This principle of storing meta data relating to aFQDN in TXT records is used by several applications, e.g.,DNS-based Service Discovery [55], DNS-AS [56], SPF [57].Figure 14 depicts the operation principle. Therefore, the con-troller continuously updates the DNS records for all resourceprofiles on the DNS server (1). It sets up the private IPv4address as address (A) record and installs a TXT record setto ipsec-tunneling:yes to indicate that the particularnetwork resource is only reachable via an VPN tunnel. Whena user tries to access the shared network resource, e.g., byentering the FQDN in the browser, a DNS A record request isdispatched. We extend this by requesting the TXT record inparallel (2a) so that the DNS server responds with the DNS Aand optionally the TXT record (2b). In case the IPsec agentreceived a TXT record, it requests IPsec tunnel setup for theparticular FQDN via the controller (3).

3) P4-IPsec Support for On-Demand VPN: Figure 15 visu-alizes the steps performed by the IPsec agent that implementsthis operation principle. Whenever a program on the user host

Resource profile

Cont

rolle

r

DNS server

IPse

c ag

ent(1)

(3)

(2a)

(2b)

Fig. 14: Operation of DNS-based Tunnel Setup Detection as used by P4-IPsec.

(e.g., a web browser) requests a particular network resourceby its FQDN, the program request DNS resolving from theoperating system’s DNS resolver (1). It is responsible forrequesting the IP addresses for the inquired FQDN. The IPsecagent monitors all DNS queries on the system and checksif a requested FQDN matches a domain of a pre-configuredwildcard list (2). The preconfigured list contains domains thatsupport on-demand IPsec setup. It can be part of the IPsecagent application or dynamically loaded from the controller.If a requested FQDN is part of the wildcard list, the IPsecagent issues a TXT request for the particular FQDN viathe DNS resolver (3a) in addition to the original A recordrequest initiated by the application (3b). If an IPsec tunnelis required and if there is not already a tunnel active to thisresource, the IPsec agent sends a tunnel setup request to thecontroller (4a). If the client is authorized to setup a tunnelto the requested resource, the controller responds with IPsectunnel configuration data (4b). The IPsec agent then sets upthe IPsec tunnel by configuring the networking stack of theoperating system with the received configuration data (5). Atunnel enabling secure connection is now active.

Roadwarrior host

DNS resolver

Application

IPsecagent

(1)

OS IPsec module

*.uni-tuebingen.de

(3b)

(3a)

(4a)

(4b) (5)

(2)

Cont

rolle

r

DN

S se

rver

Fig. 15: DNS-based IPsec tunnel setup performed by the IPsec agent that runson the roadwarrior host. It involves interactions with a DNS server and thecontroller.

VIII. P4-IPSEC CONTROL AND DATA PLANEIMPLEMENTATION FOR BMV2 AND MININET

We describe the prototypical implementation of P4-IPsec ina Mininet testbed environment. We provide an overview onthe platform, describe the P4-IPsec data plane implementationfor the Behavioral Model version 2 (BMv2) P4 softwareswitch [58], the implementation of the IPsec agent, and theimplementation of the controller.

A. Platform Overview and Development Process

We use Mininet [59] to build an emulated network environ-ment that consists of a Linux host that runs the IPsec agent,

a P4 switch, and a Linux server. The testbed runs inside of aKVM/QEMU virtual machine with Ubuntu 16.04.

B. P4-IPsec Data Plane: BMv2 P4 Software Switch

We implement the data plane functions of P4-IPsec on aBMv2 P4 software switch. It consists of multiple P4 targetsthat represent different types of P4 switches. We use the BMv2P4 software switch in the version from February 2018 [60]. Ituses the PI library for implementing a P4 Runtime server ina version from April 2018 [61]. For implementing P4-IPsec,we extend the most commonly used P4 target simple_switchas follows. First, we implemented the cipher suite externsfor AES-CTR with HMAC-MD5 for IPsec encryption anddecryption as part of the simple_switch P4 target. We programthe extensions in C++ and leverage OpenSSL to apply AES-CTR for encryption and decryption, and HMAC-MD5 forpacket authentication. Both functions can be used as P4 externswithin the P4 processing pipeline. The functions of the P4processing pipeline are implemented as a P4_16 program usingknown P4 constructs. We run the P4 program on our modifiedsimple_switch P4 target within the the simple_switch_grpctarget. It encapsulates our modified simple_switch P4 targetand provides the P4 Runtime API that is used by the controlplane to modify match-and-action tables.

C. P4-IPsec Control Plane: IPsec Agent

We implement the IPsec agent for Linux hosts as acommand-line tool in Python 3.6 [62]. As interface to thecontroller, we implement a gRPC client using the gRPClibrary [14]. At startup, users are required to specify thenetwork interface and an FQDN or IP address of the controller.Afterwards, the connection to the controller is established. Formonitoring DNS queries triggered by applications, we use theScapy library [63]. It listens on the loopback network interfacewhere the libresolve library on Linux systems sends DNSqueries to caching DNS resolvers like dnsmasq or systemd-resolved. Requesting TXT records when a potential tunnelendpoint has been identified in a DNS query is done usingthe dnspython [64] library. For IPsec tunnel setup, the IPsecagent translates received configuration data into particular ipxfrm commands from the iproute2 tool to configure IPsec onthe host system. In addition, it sets up IP routes for routingIP traffic through the IPsec tunnel. The IPsec agent saves thereceived and applied configuration data because it needs to bereverted after the tunnel has been shutdown or when rekeyingis performed. In case of rekeying, the old tunnel is deleted anda tunnel with the new parameters is set up. We implementrekeying with the help of Netlink [65]. We receive Netlinkmessages by listening on the according Netlink socket andbinding to the XFRMNLGRP_EXPIRE address so that XFRMExpire messages can be received. When receiving an XFRMExpire message, it extracts parameters such as SPI and IPaddresses of the tunnel endpoints. In order to initiate rekeying,the tunnel source and destination address and SPI are put intoa queue for processing in the main class.

D. P4-IPsec Control Plane: Controller

We implement the controller as a Python 2.7 application.We leverage the P4 runtime library [66] to program theinterface to the P4 switches and the gRPC [14] library forcommunication with the IPsec agent. The controller featuresa simple command line interface (CLI) for development andtesting purposes that displays information about all activeIPsec tunnels.

IX. P4-IPSEC DATA PLANE IMPLEMENTATION FOR P4HARDWARE SWITCHES: NETFPGA SUME

In the following, we briefly describe experiences with theNetFPGA SUME platform. We briefly review the platformand development process and report on our experiences inimplementing P4-IPsec. Due to severe limitations, we did notconduct a performance evaluation.

A. Platform Overview and Development Process

The NetFPGA SUME board is a open-source platform forrapid prototyping of network applications with throughputrates up to 100 Gb/s. It is based on the Xilinx Virtex-7690T field-programmable gate array (FPGA), an integratedcircuit that can be programmed after its production. The boardfeatures four SFP+ interfaces along a PCIe interface to acomputer host system.

XilinxSwitch architecture

Vivado HLx suite

P4-SDNet compiler

FPG

A

SDNet-to-FPGA toolchain

FPGA simulationHDL simulation

HW synthesisHW implementation

exam

ple.

vHDL compilation

SDNet compiler

P4 compilation example.p4

exam

ple.

img

exam

ple.

sdne

t

Fig. 16: Tool chain for running P4 programs on a NetFPGA SUME board(based on [67]).

A user guide [68] provided by Xilinx describes all specificsfor writing P416 programs for the NetFPGA SUME board. Itsupports three P4 architectures, namely the XilinxEngineOnly,XilinxStreamSwitch, and XilinxSwitch architecture. The latteris the most common architecture for this P4 target that consistsof a parser, a single match-and-action pipeline, and a deparser.The user guide describes several specific examples, e.g., animplementation of IP checksum calculation as a P4 extern.

Figure 16 depicts the P4-NetFPGA toolchain [69] that wasdeveloped by Xilinx to run P4 programs on the NetFPGASUME board. First, the Xilinx P416 compiler translates P4programs that are written for a Xilinx P4 architecture into aprogram for the Software Defined Specification Environmentfor Networking (SDNet) [70] from Xilinx. SDNet is a high-level design environment that was created by Xilinx priorto P4. It aims to simplify the design of packet processingdata planes that target FPGA hardware. Although SDNet

and P4 share some design goals, SDNet lays a bigger fo-cus on custom architectures compared to the approach ofP4 to abstract the hardware on a high level. Second, theSDNet compiler generates hardware descriptions in Verilog.To validate the function of the hardware description language(HDL) descriptions, generic HDL simulations and platform-specific FPGA simulations are performed. Third, the VivadoHLx suite performs hardware synthesis and implementation.It transforms the HDL representation into a hardware designusing lookup tables (LUTs) and 1-bit-registers (flipflops) toprogram the FPGA. Last, the resulting bitfile is used toprogram the FPGA.

B. Prototypical Implementation

We managed to implement a very limited prototype. It onlyallows to apply the NULL cipher on fixed-length packets thatdo not exceed a total length of 140 bytes. In the following, wedescribe the encountered problems and limitations in detail.

First, P4-SDNet is limited to packet header manipulation.Therefore, we parsed payload fields of packets as additionalheader fields. Besides performance downsides of this ap-proach, P4-SDNet does not support parsing variable-lengthheader fields. This restricts the implementation to packets witha fixed length. Second, P4-SDNet has severe limitations onthe implementation of P4 externs. They are programmed ashardware definitions in Verilog that run separated from the restof the P4 program. In contrast to BMv2, a data stucture thatcontains the parsed header fields, the payload, and a field forthe returning value needs to be passed to the P4 extern. As weexperienced major performance drawbacks in implementingcryptographic functions within P4 externs in our previous work[1], we only implemented the IPsec NULL cipher. For IPsecprotection, it encapsulates the plain text IP header and packetwithin an ESP packet. For IPsec decryption, it extracts theoriginal IP header and payload from the ESP packet. Again,P4-SDNet turned out to be not designed for packet payloadmanipulations. Data transmission between the P4 extern andP4 pipeline is limited to 10 kbit for one function call whichlimits the maximum packet size to be processed through a P4extern to approximately 600 bytes. In addition, data transfer tothe P4 extern needs to be executed within one clock cycle ofthe FPGA. During the synthesis, the Vivado suite optimizes thehardware implementation through several algorithms. Variousexperiments have shown a practical upper bound of 140 bytesfor packets. Either the hardware implementation did use moreresources than offered by the FPGA, or data transfer andcalculation within the P4 extern exceeded one clock cycle.Last, we encountered several more general problems withP4-SDNet and the NetFPGA. Probably due to a bug, wewere not able to access the values of a LPM table for IProuting with our SDN controller. We solved that problemby using exact matching tables instead, an approach that isnot acceptable for a productional implementation. In addition,we experienced several stability problems. No matches inmatch-and-action tables were found when data was written tohardware registers. Finally, we missed many important detailsin the documentation.

X. P4-IPSEC DATA PLANE IMPLEMENTATION FOR P4HARDWARE SWITCHES: EDGECORE WEDGE WITH TOFINO

In the following, we describe experiences with the EdgecoreWedge 100BF-32X switch that features a Tofino P4 ASICfrom Barefoot networks. We provide an overview of theplatform, describe the development process, and report on theimplementation of P4-IPsec.

A. Platform Overview and Development Process

The Edgecore Wedge 100BF-32X [71] switch is a top-of-rack (ToR) switch for data center networks. It features 32QSFP network ports that can be configured to support through-put rates up to 100 Gbit/s. Packet switching is performed bya Tofino ASIC [72] from Barefoot Networks. It connects toa main CPU module via PCIe. The main CPU module is anoff-the-shelf CPU board that features an Intel Pentium D1517processor with 1.6 GHz on 4 cores, 8 GB RAM, and a 32GB SSD. Unfortunately, the main CPU module only supportsPCIe Gen 2 [73] while the Tofino ASIC can support Gen 3.

Figure 17 depicts the architecture and process of devel-opment for switches with a Tofino ASIC. The main CPUmodule runs the Barefoot P4 software development environ-ment (SDE) [74] on top of a Linux-based operating systemsuch as Open Network Linux (ONL) or SONiC [75]. It isresponsible for programming and managing the Tofino ASICand contains interfaces for loading, configuring, and managingP4 programs during execution. It also exposes managementoperations to APIs, e.g., the P4 Runtime API, that may beused by a control plane to modify match-and-action tables.The SDE also exposes a PCIe CPU port that enables networkpacket exchange between the P4 processing pipeline and themain CPU module. A Linux kernel module supplied with theSDE provides a virtual network interface for communicationwith the PCIe CPU port. P4 programs are written for theTofino native architecture (TNA), compiled by the BarefootP4 compiler, and loaded on the Tofino ASIC by the SDE.

Barefoot P4 Compiler

Tofino native architecture

Tofino ASIC

Tofino-based switch(e.g., Edgecore Wedge 100BF-32X)

Control plane

example.p4 Main CPU module

Port Port Port Port…

SDE CPU port APIs

Fig. 17: Development process for P4 programs running on a switch thatfeatures a Tofino ASIC (based on [74]).

B. Prototypical Implementation

The Tofino ASIC is optimized for high-speed packet pro-cessing and bandwidths up to 6.5 Tbit/s in data centeror core networks. The second version of the Tofino ASICeven supports bandwidths up to 12.8 Tbit/s. User-defined P4externs that may contain computation-intense functions are

not supported on high-speed switching silicon such as theTofino ASIC. To investigate the feasibility of implementingP4-IPsec on that hardware, we relocate the IPsec functionsthat were implemented as P4 externs in our BMv2 prototypeSection VIII to the main CPU module of the Wedge switch.

Figure 18 depicts this concept. We leverage the PCIeCPU port to exchange packets between the Tofino ASIC andthe main CPU module. Therefore, we replace all functioncalls of the P4 externs in the P4 processing pipeline by theCPU port. We use the IPsec kernel functions of the Linuxoperating system running on the main CPU module for IPsecprocessing. We implement an IPsec crypto manager programthat configures the IPsec kernel functions with data receivedfrom the controller. It is implemented in Python 3 [76] anduses iproute2 [77] commands for managing the SPD andSAD. After finishing IPsec processing, the main CPU moduletransfers the packets back to the Tofino ASIC via the CPUport. The Tofino ASIC then performs IP routing to forward thepackets received from the CPU port. Besides programming theSAD and SPD via the IPsec crypto manager, the control planeprograms IP forwarding and the SPD on the P4 processingpipeline.

Main CPU module

IPsec

Tofino switching ASIC

Front port

CPU

por

t

PCIe

Front port T

ofin

o A

SIC

IP

IP + IPsec

processing Kern

elsp

ace

User

spac

e

iproute2IPsec crypto manager program

Controller

SAD

SPD

Fig. 18: P4-IPsec on the Edgecore Wedge 100BF-32X. We relocate IPsecprocessing to the main CPU module that interfaces the Tofino switching ASICvia a PCIe CPU port.

C. Evaluation

We evaluate the prototypical implementation in experimentson latency and TCP throughput.

1) Experiment Setup: We attach two physical hosts runningUbuntu 16.04 LTS via 10 Gbit/s links to the front ports of theWedge switch and perform the following experiments. Thelink between the first client and the switch is secured usingIPsec while the link between the switch and the second clientis not secured. The main CPU module performs decryptionfor packets coming from the first client and encryption for thepackets coming from the second client.

2) Latency: We investigate the latency when using theCPU port and IPsec processing on the main CPU module.We send 100 ICMP echo requests from the first client tothe second client and measure an average round-trip time ofapproximately 1.5 ms.

3) TCP Throughput: We investigate the maximum TCPthroughput when using the CPU port with IPsec processing onthe main CPU module. Figure 19 depicts the results of threeexperiments, each performed with five runs and a duration of30 s. We generate TCP transmissions between both clients

with iperf3 [78] and measure the throughput. For a singleIPsec tunnel with AES-GCM-256, we measure an averageTCP throughput of approximately 1.4 Gbit/s. When usingthe NULL encryption and authentication cipher, i.e., whennot encrypting and authenticating packets, the average TCPthroughput rises to approximately 2 Gbit/s. Throughput is notonly determined by IPsec processing, also re-keying duringthe experiments affects the performance as some packets arelost in the time between the old SA is deleted and the newSA is set up. We also performed measurements for up to16 concurrent IPsec flows and calculate the average of 10runs with a duration of 300 s each. However, the maximumTCP throughput remains at 1.4 Gbit/s for IPsec with AES-GCM-256 and 2 Gbit/s for IPsec with the NULL cipher.For providing a reference measurement, we investigate themaximum TCP throughput of the main CPU module. Weassign an IP address to the virtual network interface of theCPU port on the main CPU module and perform iperf3 mea-surements between the first client and the main CPU module.We measure an average TCP throughput of approximately 3.3Gbit/s. That is the upper bound for TCP throughput that canbe handled by the main CPU module of the Wedge switch inour setup. We attribute the large differences in TCP throughputto the rather slow CPU that is used in the Edgecore Wedge100BF-32X. The Intel Pentium D1517 processor has a basefrequency of 1.6 GHz. We consider this is a very reasonableperformance that might be sufficient for scenarios where onlyfew shared network resources should be sporadically accessedby roadwarrior hosts.

0 500 1000 1500 2000 2500 3000 3500Throughput [Mbit/s]

IPsecaes-gcm 256

IPsecnull enc.

null auth.

no IPsec

Fig. 19: Maximum throughput for a single IPsec connection using the PCIeCPU port and different encryption algorithms. The uppermost bar shows themaximum throughput that the PCIe CPU port is capable of.

D. Improved Throughput through External Crypto Host

As the presented throughput was limited by the performanceof the switch CPU and the CPU port’s bandwidth, we suggestthe use of an external Linux-based crypto host as meansto improve throughput. Figure 20 depicts the concept ofoffloading IPsec processing. That means, IPsec processing isoffloaded from the switch to the external crypto host whichworks as network function. As a consequence, en-/decryptioncapacity could be scaled up by increasing the number of cryptohosts connected to the switch. The controller is extendedto instruct the switch that IPsec traffic is forwarded via aparticular front port instead of the CPU port. Furthermore, theIPsec crypto manager program from the previous approach

(see Section X-B) constitutes the heart of the crypto hostimplementation. IP and IPsec processing and communicationwith SAD and SPD are carried out in kernel space while otherfunctions are executed in user space. For the evaluation ofthis approach, we utilizes a crypto host with an 8-core and16-thread Intel Xeon Gold 6134 CPU, 128 GB RAM, and a240 GB SSD, and Ubuntu 18.04 LTS as operating system. Weperform the same experiments as in Section X-C. The round-trip time is approximately 2 ms which is slightly larger thanin the previous approach.

IP + IPsec processing Ke

rnel

spac

e

SPD

User

spac

e

iproute2IPsec crypto manager program

ControllerSAD

Internet …

Crypto host

Host

Fig. 20: Alternate approach for an implementation of P4-IPsec on a WedgeSwitch by replacing P4 externs with a dedicated Linux-based crypto host withkernel-space processing.

The TCP throughput for IPsec traffic is shown in Figure 21.For a single IPsec tunnel with AES-GCM-256, we measurean average TCP throughput of approximately 4 Gbit/s. Itcan be increased by running multiple connections over thesame crypto host. For 16 parallel IPsec flows, we measure anoverall average TCP throughput of approximately 24 Gbit/s.This effect can be attributed to receive-side scaling (RSS)of the network interface card, which can leverage multiplecores, but only one per flow. In case of multiple flow, theoverall throughput can be increased through RSS by leveragingthe processing power of more than a single core. The TCPthroughput can be further improved by optimization techniquesas presented in Section V.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16Number of flows

5

10

15

20

25

Thro

ughp

ut (G

bit/s

)

Crypto: internal CPUCrypto: external host

Fig. 21: Average throughput for a one to 16 IPsec connection using the PCIeCPU port of the Edgecore Wedge and using a dedicated compute node.

XI. CONCLUSION

In this work, we provided an extensive review and catego-rization of IPsec-related work. We proposed P4-IPsec, a P4-

based implementation of an IPsec gateway supporting ESPin tunnel mode and different cipher suites. To the best ofour knowledge, it is the first data plane implementation ofIPsec. This allows to turn any switch of a P4 network into aVPN concentrator so that access to critical resources can berestricted to very close to the target instead of having a singleVPN concentrator for a large network. The runtime operationof P4-IPsec is managed by a control plane avoiding complexkey exchange protocols such as IKE. It further supports thenew use case “on-demand VPN” where IPsec tunnels areautomatically set up to appropriate IPsec gateways when aspecific resource behind them is requested. We demonstratedthe feasibility of P4-IPsec in prototypical implementationsfor the BMv2 P4 software switch, the NetFPGA SUMEplatform, and the Edgecore Wedge 100BF-32X switch thatfeatures a Tofino ASIC. We performed performance evaluationexperiments on the Wedge switch with two different variantsfor encryption and decryption. They revealed an acceptablethroughput rate for a platform without specialized encryp-tion/decryption support, but clearly leave room for encryp-tion/decryption acceleration. The experiments show that P4is suitable for use cases that target security functions withmedium data rates, e.g., at the edge of enterprise or campusnetworks.

REFERENCES

[1] F. Hauser, M. Schmidt, M. Häberle, and M. Menth, “P4-MACsec:Dynamic Topology Monitoring and Data Layer Protection with MACsecin P4-SDN,” CoRR, vol. abs/1904.07088, 2019. [Online]. Available:http://arxiv.org/abs/1904.07088

[2] K. Seo and S. Kent, “Security Architecture for the Internet Protocol,”RFC 4301, Dec. 2005.

[3] S. Kent, “IP Authentication Header,” RFC 4302, Dec. 2005.[4] ——, “IP Encapsulating Security Payload (ESP),” RFC 4303, Dec. 2005.[5] J. Viega and D. McGrew, “The Use of Galois/Counter Mode (GCM) in

IPsec Encapsulating Security Payload (ESP),” RFC 4106, Jun. 2005.[6] D. Carrel and D. Harkins, “The Internet Key Exchange (IKE),” RFC

2409, Nov. 1998.[7] C. Kaufman, P. E. Hoffman, Y. Nir, P. Eronen, and T. Kivinen, “Internet

Key Exchange Protocol Version 2 (IKEv2),” RFC 7296, Oct. 2014.[8] N. Ferguson and B. Schneier, “A Cryptographic Evaluation of IPsec,”

Counterpane Internet Security, Inc, Tech. Rep., 2000.[9] “OpenVPN,” https://openvpn.net, accessed 06-06-2019.

[10] “WireGuard: Next Generation Kernel Network Tunnel,” https://www.wireguard.com/papers/wireguard.pdf, accessed 06-06-2019.

[11] R. Bifulco and G. Rétvári, “A Survey on the Programmable Data Plane:Abstractions Architectures and Open Problems,” in IEEE InternationalConference on High Performance Switching and Routing (HPSR), 2018.

[12] P. Bosshart et al., “P4: Programming Protocol-independent PacketProcessors,” SIGCOMM Comput. Commun. Rev., vol. 44, Jul. 2014.

[13] “P4_16 Language Specification, Version 1.1.0,” https://p4.org/p4-spec/docs/P4-16-v1.1.0-spec.html, accessed 10-09-2018.

[14] “gRPC,” https://grpc.io/, accessed 04-14-2019.[15] “Google Protocol Buffers,” https://developers.google.com/

protocol-buffers/, accessed 04-14-2019.[16] Y. Li and J. Mao, “SDN-based Access Authentication and Automatic

Configuration for IPsec,” in 2015 4th International Conference onComputer Science and Network Technology (ICCSNT), vol. 01, Dec.

[17] A. Alharbi, A. Bahnasse, M. Talea, H. Ait Oulahyane, and F. E.Louhab, “Smart SDN Policy Management Based VPN Multipoint,” inLecture Notes in Real-Time Intelligent Systems, J. Mizera-Pietraszko,P. Pichappan, and L. Mohamed, Eds. Cham: Springer InternationalPublishing, 2019, pp. 250–263.

[18] X. Guo, K. Yang, A. Galis, X. Cheng, B. Yang, and D. Liu, “A Policy-based Network Management System for IP VPN,” in InternationalConference on Communication Technology Proceedings, 2003. ICCT2003., vol. 2, April 2003, pp. 1630–1633 vol.2.

[19] M. Mechtri, I. Houidi, W. Louati, and D. Zeghlache, “SDN for InterCloud Networking,” in 2013 IEEE SDN for Future Networks andServices (SDN4FNS), Nov 2013, pp. 1–7.

[20] R. van der Pol, B. Gijsen, P. Zuraniewski, D. F. C. Romão, and M. Kaat,“Assessment of SDN Technology for an Easy-to-use VPN Service,”Future Gener. Comput. Syst., vol. 56, no. C, pp. 295–302, Mar. 2016.[Online]. Available: https://doi.org/10.1016/j.future.2015.09.010

[21] S. Aragon, M. Tiloca, M. Maass, M. Hollick, and S. Raza, “ACE ofSpades in the IoT Security Game: A Flexible IPsec Security Profilefor Access Control,” in 2018 IEEE Conference on Communications andNetwork Security (CNS), May 2018, pp. 1–9.

[22] S. Aragon, M. Tiloca, and S. Raza, “IPsec profile of ACE,”Internet Engineering Task Force, Internet-Draft draft-aragon-ace-ipsec-profile-01, Oct. 2017, work in Progress. [Online]. Available:https://datatracker.ietf.org/doc/html/draft-aragon-ace-ipsec-profile-01

[23] D. Carrel and B. Weis, “IPsec Key Exchange using a Controller,” InternetEngineering Task Force, Internet-Draft draft-carrel-ipsecme-controller-ike-01, Mar. 2019, work in Progress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-carrel-ipsecme-controller-ike-01

[24] R. Lopez, G. Lopez-Millan, and F. Pereniguez-Garcia,“Software-Defined Networking (SDN)-based IPsec Flow Pro-tection,” Internet Engineering Task Force, Internet-Draftdraft-ietf-i2nsf-sdn-ipsec-flow-protection-04, Mar. 2019, work inProgress. [Online]. Available: https://datatracker.ietf.org/doc/html/draft-ietf-i2nsf-sdn-ipsec-flow-protection-04

[25] J. Son, Y. Xiong, K. Tan, P. Wang, Z. Gan, and S. Moon, “Protego:Cloud-Scale Multitenant IPsec Gateway,” in 2017 USENIX AnnualTechnical Conference (USENIX ATC 17). Santa Clara, CA: USENIXAssociation, 2017, pp. 473–485.

[26] A. Braadland, “Key Management for Data Plane Encryption in SDN Us-ing WireGuard,” Master’s thesis, Norwegian University of Science andTechnology, Department of Information Security and CommunicationTechnology, 2017.

[27] A. Sajassi, A. Banerjee, S. Thoria, D. Carrel, and B. Weis, “SecureEVPN,” Internet Engineering Task Force, Internet-Draft draft-sajassi-bess-secure-evpn-01, Mar. 2019, work in Progress. [Online]. Available:https://datatracker.ietf.org/doc/html/draft-sajassi-bess-secure-evpn-01

[28] “Cisco: IPsec Management Configuration Guide,” https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/sec_conn_imgmt/configuration/xe-16/sec-ipsec-management-xe-16-book/sec-ipsec-snmp-supp.html,accessed 04-28-2019.

[29] W. Li, F. Lin, and G. Sun, “SDIG: Toward Software-Defined IPsecGateway,” in 2016 IEEE 24th International Conference on NetworkProtocols (ICNP), Nov 2016, pp. 1–8.

[30] R. Spenneberg, VPN mit Linux: Grundlagen und Anwendung virtuellerprivater Netzwerke mit Open-Source-Tools. Addison-Wesley, 2010.

[31] “DPDK,” https://www.dpdk.org/, accessed 04-05-2019.[32] L. Rizzo, “netmap: A Novel Framework for Fast Packet I/O,” in 2012

USENIX Annual Technical Conference (USENIX ATC 12). Boston, MA:USENIX Association, 2012, pp. 101–112. [Online]. Available: https://www.usenix.org/conference/atc12/technical-sessions/presentation/rizzo

[33] “ntop: pf_ring,” https://www.ntop.org/products/packet-capture/pf_ring/,accessed 05-04-2019.

[34] W. Li, S. Hu, G. Sun, and Y. Li, “Adaptive Load Balancing on Multi-core IPsec Gateway,” in ICA3PP, 2018.

[35] G. Xie, H. Jiang, and K. Salamatian, “Load Balancing by RulesetPartition for Parallel IDS on Multi-Core Processors,” in ICCCN 2013.

[36] S. Han, K. Jang, K. Park, and S. B. Moon, “Packetshader: a gpu-accelerated software router,” in SIGCOMM, 2010.

[37] S. Gallenmüller, P. Emmerich, F. Wohlfart, D. Raumer, and G. Carle,“Comparison of frameworks for high-performance packet IO,” in 2015ACM/IEEE Symposium on Architectures for Networking and Communi-cations Systems (ANCS), May 2015, pp. 29–38.

[38] “Intel AES-NI,” https://www.intel.de/content/www/de/de/architecture-and-technology/advanced-encryption-standard-aes/data-protection-aes-general-technology.html, accessed 05-04-2019.

[39] ARM, “Architecture Reference Manual (ARMv8) for ARMv8-A Archi-tecture Profile,” Arm Ltd., Tech. Rep., 2017.

[40] “Marvell ARMADA 38x Family,” https://www.marvell.com/embedded-processors/armada/armada-38x/, accessed 05-12-2019.

[41] J. DiGiglio and D. Ricci, “High Performance, Open Standard, Virtual-ization with NFV and SDN,” WindRiver, Tech. Rep., 2013.

[42] “Algotronix AES IP-Cores,” http://www.algotronix-store.com/AES_IP_Cores_s/20.htm, accessed 05-12-2019.

[43] “Cast AES IP-Cores,” http://www.cast-inc.com/ip-cores/encryption/aes-p/index.html, accessed 05-12-2019.

[44] C. wan Ha, J. H. Lee, D. S. Leem, M. soo Park, and B.-Y. Choi,“ASIC Design of IPsec Hardware Accelerator for Network Security,”Proceedings of 2004 IEEE Asia-Pacific Conference on Advanced SystemIntegrated Circuits, pp. 168–171, 2004.

[45] A. Hodjat, P. Schaumont, and I. Verbauwhede, “Architectural DesignFeatures of a Programmable High Throughput AES Coprocessor,”in International Conference on Information Technology: Coding andComputing, 2004. Proceedings. ITCC 2004., vol. 2, April 2004.

[46] Y. Liu, D. Xu, W. Song, and Z. Mu, “Design and Implementationof High Performance IPSec Applications with Multi-Core Processors,”in 2008 International Seminar on Future Information Technology andManagement Engineering, Nov 2008, pp. 595–598.

[47] J. Meng, X. Chen, Z. Chen, C. Lin, B. Mu, and L. Ruan, “TowardsHigh-Performance IPsec on Cavium OCTEON Platform,” in TrustedSystems, L. Chen and M. Yung, Eds. Berlin, Heidelberg: SpringerBerlin Heidelberg, 2011, pp. 37–46.

[48] J. Park, W. Jung, G. Jo, I. Lee, and J. Lee, “PIPSEA: A PracticalIPsec Gateway on Embedded APUs,” in Proceedings of the 2016ACM SIGSAC Conference on Computer and Communications Security,ser. CCS ’16. New York, NY, USA: ACM, 2016, pp. 1255–1267.[Online]. Available: http://doi.acm.org/10.1145/2976749.2978329

[49] M. Rao, J. Coleman, and T. Newe, “An FPGA-based ReconfigurableIPsec ESP Core Suitable for IoT Applications,” in 2016 10th Interna-tional Conference on Sensing Technology (ICST), Nov 2016, pp. 1–5.

[50] M. Vajaranta, V. Viitamäki, A. Oinonen, T. D. Hämäläinen, A. Kulmala,and J. Markunmäki, “Feasibility of FPGA Accelerated IPsec on Cloud,”in 2018 21st Euromicro Conference on Digital System Design (DSD),Aug 2018, pp. 569–572.

[51] M. Korona, K. Skowron, M. Trzepinski, and M. Rawski, “FPGA Imple-mentation of IPsec Protocol Suite for Multigigabit Networks,” in 2017International Conference on Systems, Signals and Image Processing(IWSSIP), May 2017, pp. 1–5.

[52] B. Driessen, T. Güneysu, E. B. Kavun, O. Mischke, C. Paar, andT. Pöppelmann, “IPSecco: A lightweight and reconfigurable IPSec core,”in 2012 International Conference on Reconfigurable Computing andFPGAs, Dec 2012, pp. 1–7.

[53] “[P4-dev] VPNs in P4,” http://lists.p4.org/pipermail/p4-dev_lists.p4.org/2016-June/002109.html, accessed 04-05-2019.

[54] “Xilinx SDNet PX Programming Language – User Guide,”https://www.xilinx.com/support/documentation/sw_manuals/xilinx2017_3/ug1016-px-programming.pdf, accessed 05-12-2019.

[55] S. Cheshire and M. Krochmal, “DNS-Based Service Discovery,” RFC6763, Feb. 2013.

[56] “Cisco Documentation: DNS-AS,” https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/qos_nbar/configuration/xe-16/qos-nbar-xe-16-book/nbar-dns-as.pdf, accessed 06-01-2019.

[57] S. Kitterman, “Sender Policy Framework (SPF) for Authorizing Useof Domains in Email, Version 1,” RFC 7208, Apr. 2014. [Online].Available: https://rfc-editor.org/rfc/rfc7208.txt

[58] “Github: p4lang/behavioral-model,” https://github.com/p4lang/behavioral-model, accessed 12-05-2018.

[59] “Mininet,” http://mininet.org, accessed 06-11-2019.[60] “BMv2: Version from 2018/02,” https:

//github.com/p4lang/behavioral-model/commit/39abe290b4143e829b8f983965fcdc711e3c450c, accessed 06-11-2019.

[61] “PI Library,” https://github.com/p4lang/PI/commit/1ca80066e065ae52a34416822c20b83173b2146f, accessed 06-11-2019.

[62] “Python 3.6.8,” https://www.python.org/downloads/release/python-368/,accessed 04-23-2019.

[63] “Scapy,” https://scapy.net/, accessed 04-23-2019.[64] “dnspython,” http://www.dnspython.org, accessed 06-06-2019.[65] “netlink,” http://man7.org/linux/man-pages/man7/netlink.7.html,

accessed 06-06-2019.[66] “Github: P4 language tutorials,” https://github.com/p4lang/tutorials/tree/

master/utils/p4runtime_lib, accessed 06-06-2019.[67] “P4-NetFPGA Tutorial,” https://p4.org/assets/p4_d2_2017_p4_netfpga_

tutorial.pdf, accessed 04-23-2019.[68] Xilinx, “P4-SDNet User Guide (v2018.01),” Xilinx, Tech. Rep., 2018.[69] “P4-NetFPGA Wiki,” https://github.com/NetFPGA/

P4-NetFPGA-public/, accessed 04-22-2019.[70] “Xilinx: Software Defined Specification Environment for Network-

ing (SDNet),” https://www.xilinx.com/publications/prod_mktg/sdnet/backgrounder.pdf, accessed 05-12-2019.

[71] “Datasheet: Wedge100BF-32X/65X Switch 100GbE,” https://www.edge-core.com/_upload/images/Wedge100BF-32X_65X_DS_R04_20180531.pdf, accessed 04-10-2019.

[72] “Barefoot Networks: Tofino,” https://barefootnetworks.com/products/brief-tofino/, accessed 12-05-2018.

[73] “Intel Intel R© Xeon R© D-1500 Processors Datasheet, Vol. 1,”https://www.intel.com/content/www/us/en/products/docs/processors/xeon/xeon-d-1500-datasheet-vol-1.html, accessed 06-06-2019.

[74] “Barefoot Networks: P4 Studio,” https://barefootnetworks.com/products/brief-p4-studio/, accessed 04-22-2019.

[75] “SONiC,” https://azure.github.io/SONiC/, accessed 05-09-2019.[76] “Python,” https://www.python.org/, accessed 05-09-2019.[77] “iproute2,” https://linux.die.net/man/8/ip, accessed 04-23-2019.[78] “iPerf,” https://iperf.fr/, accessed 04-05-2019.

Frederik Hauser studied computer science at theUniversity of Tuebingen, Germany, and received hisMaster degree. Since then, he has been a researcherat the Chair of Communication Networks at the Uni-versity of Tuebingen, pursuing his Ph.D. His mainresearch interests include software defined network-ing, network function virtualization, and networksecurity.

Marco Haeberle studied computer science at theUniversity of Tuebingen, Germany, and received hisMaster degree. Since then, he has been a researcherat the Chair of Communication Networks at theUniversity of Tuebingen, pursuing his Ph.D. Hismain research interests include software definednetworking, P4, network security and automatednetwork management.

Mark Schmidt [S’14] studied computer science atthe University of Tuebingen, Germany, and receivedhis diploma degree. Since then, he has been aresearcher at the Chair of Communication Networksat the University of Tuebingen, pursuing his Ph.D.His main research interests include software definednetworking, OpenFlow, high-speed networks, andvirtualization.

Michael Menth [M’05, SM’09] is a professor at theDepartment of Computer Science at the Universityof Tuebingen and Chair holder of CommunicationNetworks. His special interests are performanceanalysis and optimization of communication net-works, resilience and routing issues, resource andcongestion management, software-defined network-ing and Internet protocols, industrial networking, andthe Internet of Things. He has published more than150 papers in the field of computer networking.