Tracking Middleboxes with Tracebox IETF93: HOPS

u n i v e r s i t e d e l i e g e

Faculté des Sciences Appliquées

Tracking Middleboxes with TraceboxIETF93: HOPS

Korian Edeline, Benoit DonnetUniversity of Liège

July 22, 2015Slide 1/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Introduction

1 Middleboxes

2 How to detect them ?

3 Tracebox

4 Implementations

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 2/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Plan

1 Middleboxes

2 How to detect them ?

3 Tracebox

4 Implementations

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 3/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Deployment

1

• The market for security-oriented middleboxes isestimated to exceed $10B by 20162

1Justine Sherry et al. “Making middleboxes someone else’s problem: network processingas a cloud service”. In: ACM SIGCOMM Computer Communication Review 42.4 (2012),pp. 13–24.

2Rahul Potharaju and Navendu Jain. “Demystifying the dark side of the middle: A fieldstudy of middlebox failures in datacenters”. In: Proceedings of the 2013 conference onInternet measurement conference. ACM. 2013, pp. 9–22.Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 4/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Deployment

1

• The market for security-oriented middleboxes isestimated to exceed $10B by 20162

1Justine Sherry et al. “Making middleboxes someone else’s problem: network processingas a cloud service”. In: ACM SIGCOMM Computer Communication Review 42.4 (2012),pp. 13–24.

2Rahul Potharaju and Navendu Jain. “Demystifying the dark side of the middle: A fieldstudy of middlebox failures in datacenters”. In: Proceedings of the 2013 conference onInternet measurement conference. ACM. 2013, pp. 9–22.Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 4/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Router processing

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 5/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

NAT processing

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 6/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

ALG processing

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 7/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Potential processing over the wholeInternet

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 8/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Plan

1 Middleboxes

2 How to detect them ?

3 Tracebox

4 Implementations

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 9/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

TBIT• tbit3

• Basic idea• Send forged TCP packets over raw IP sockets• Host firewall prevents kernel from seeing response

packets• BPF delivers blocked packets to user process for

analysis• Effect

• a user-level, user-controllable TCP, without kernelchanges

• Purpose• detect whether ECN, IP options, and TCP options

can be safely used

3Alberto Medina, Mark Allman, and Sally Floyd. “Measuring interactions betweentransport protocols and middleboxes”. In: Proceedings of the 4th ACM SIGCOMMconference on Internet measurement. ACM. 2004, pp. 336–341.Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 10/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

TBIT• tbit3

• Basic idea• Send forged TCP packets over raw IP sockets

• Host firewall prevents kernel from seeing responsepackets

• BPF delivers blocked packets to user process foranalysis

• Effect• a user-level, user-controllable TCP, without kernel

changes• Purpose

• detect whether ECN, IP options, and TCP optionscan be safely used

3Alberto Medina, Mark Allman, and Sally Floyd. “Measuring interactions betweentransport protocols and middleboxes”. In: Proceedings of the 4th ACM SIGCOMMconference on Internet measurement. ACM. 2004, pp. 336–341.Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 10/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

TBIT• tbit3

• Basic idea• Send forged TCP packets over raw IP sockets• Host firewall prevents kernel from seeing response

packets

• BPF delivers blocked packets to user process foranalysis

• Effect• a user-level, user-controllable TCP, without kernel

changes• Purpose

• detect whether ECN, IP options, and TCP optionscan be safely used

3Alberto Medina, Mark Allman, and Sally Floyd. “Measuring interactions betweentransport protocols and middleboxes”. In: Proceedings of the 4th ACM SIGCOMMconference on Internet measurement. ACM. 2004, pp. 336–341.Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 10/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

TBIT• tbit3

• Basic idea• Send forged TCP packets over raw IP sockets• Host firewall prevents kernel from seeing response

packets• BPF delivers blocked packets to user process for

analysis

• Effect• a user-level, user-controllable TCP, without kernel

changes• Purpose

• detect whether ECN, IP options, and TCP optionscan be safely used

3Alberto Medina, Mark Allman, and Sally Floyd. “Measuring interactions betweentransport protocols and middleboxes”. In: Proceedings of the 4th ACM SIGCOMMconference on Internet measurement. ACM. 2004, pp. 336–341.Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 10/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

TBIT• tbit3

• Basic idea• Send forged TCP packets over raw IP sockets• Host firewall prevents kernel from seeing response

packets• BPF delivers blocked packets to user process for

analysis• Effect

• a user-level, user-controllable TCP, without kernelchanges

• Purpose• detect whether ECN, IP options, and TCP options

can be safely used

3Alberto Medina, Mark Allman, and Sally Floyd. “Measuring interactions betweentransport protocols and middleboxes”. In: Proceedings of the 4th ACM SIGCOMMconference on Internet measurement. ACM. 2004, pp. 336–341.Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 10/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

TBIT• tbit3

• Basic idea• Send forged TCP packets over raw IP sockets• Host firewall prevents kernel from seeing response

packets• BPF delivers blocked packets to user process for

analysis• Effect

• a user-level, user-controllable TCP, without kernelchanges

• Purpose• detect whether ECN, IP options, and TCP options

can be safely used3Alberto Medina, Mark Allman, and Sally Floyd. “Measuring interactions between

transport protocols and middleboxes”. In: Proceedings of the 4th ACM SIGCOMMconference on Internet measurement. ACM. 2004, pp. 336–341.Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 10/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

TCPExposure• TCPExposure4

• Basic idea• Client and Server Python scripts• Send forged TCP packets over raw IP sockets• Sent packets include payload commands bytes:

just ack, echo headers or don’t advance ack• Server sends back received&to-be-sent headers

as payload• Compare what was sent to what was received

• Effect• Detect last modification, errors• Differentiate inbound & outbound modifications

4Michio Honda et al. “Is it still possible to extend TCP?”. In: Proceedings of the 2011ACM SIGCOMM conference on Internet measurement conference. ACM. 2011,pp. 181–194.Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 11/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

TCPExposure• TCPExposure4

• Basic idea• Client and Server Python scripts

• Send forged TCP packets over raw IP sockets• Sent packets include payload commands bytes:

just ack, echo headers or don’t advance ack• Server sends back received&to-be-sent headers

as payload• Compare what was sent to what was received

• Effect• Detect last modification, errors• Differentiate inbound & outbound modifications

4Michio Honda et al. “Is it still possible to extend TCP?”. In: Proceedings of the 2011ACM SIGCOMM conference on Internet measurement conference. ACM. 2011,pp. 181–194.Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 11/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

TCPExposure• TCPExposure4

• Basic idea• Client and Server Python scripts• Send forged TCP packets over raw IP sockets

• Sent packets include payload commands bytes:just ack, echo headers or don’t advance ack

• Server sends back received&to-be-sent headersas payload

• Compare what was sent to what was received

• Effect• Detect last modification, errors• Differentiate inbound & outbound modifications

4Michio Honda et al. “Is it still possible to extend TCP?”. In: Proceedings of the 2011ACM SIGCOMM conference on Internet measurement conference. ACM. 2011,pp. 181–194.Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 11/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

TCPExposure• TCPExposure4

• Basic idea• Client and Server Python scripts• Send forged TCP packets over raw IP sockets• Sent packets include payload commands bytes:

just ack, echo headers or don’t advance ack

• Server sends back received&to-be-sent headersas payload

• Compare what was sent to what was received

• Effect• Detect last modification, errors• Differentiate inbound & outbound modifications

4Michio Honda et al. “Is it still possible to extend TCP?”. In: Proceedings of the 2011ACM SIGCOMM conference on Internet measurement conference. ACM. 2011,pp. 181–194.Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 11/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

TCPExposure• TCPExposure4

• Basic idea• Client and Server Python scripts• Send forged TCP packets over raw IP sockets• Sent packets include payload commands bytes:

just ack, echo headers or don’t advance ack• Server sends back received&to-be-sent headers

as payload

• Compare what was sent to what was received

• Effect• Detect last modification, errors• Differentiate inbound & outbound modifications

4Michio Honda et al. “Is it still possible to extend TCP?”. In: Proceedings of the 2011ACM SIGCOMM conference on Internet measurement conference. ACM. 2011,pp. 181–194.Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 11/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

TCPExposure• TCPExposure4

• Basic idea• Client and Server Python scripts• Send forged TCP packets over raw IP sockets• Sent packets include payload commands bytes:

just ack, echo headers or don’t advance ack• Server sends back received&to-be-sent headers

as payload• Compare what was sent to what was received

• Effect• Detect last modification, errors• Differentiate inbound & outbound modifications

4Michio Honda et al. “Is it still possible to extend TCP?”. In: Proceedings of the 2011ACM SIGCOMM conference on Internet measurement conference. ACM. 2011,pp. 181–194.Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 11/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

TCPExposure• TCPExposure4

• Basic idea• Client and Server Python scripts• Send forged TCP packets over raw IP sockets• Sent packets include payload commands bytes:

just ack, echo headers or don’t advance ack• Server sends back received&to-be-sent headers

as payload• Compare what was sent to what was received

• Effect• Detect last modification, errors• Differentiate inbound & outbound modifications

4Michio Honda et al. “Is it still possible to extend TCP?”. In: Proceedings of the 2011ACM SIGCOMM conference on Internet measurement conference. ACM. 2011,pp. 181–194.Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 11/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

TCP HICCUPS• TCP HICCUPS5

• Lightweight TCP extension that exposes in flightpacket header modification to end points

• Basic question behind TCP HICCUPS• did my packet arrive at the destination with the

same headers as sent?

5Ryan Craven, Robert Beverly, and Mark Allman. “A middlebox-cooperative TCP for anon end-to-end internet”. In: Proceedings of the 2014 ACM conference on SIGCOMM.ACM. 2014, pp. 151–162.Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 12/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

TCP HICCUPS• TCP HICCUPS5

• Lightweight TCP extension that exposes in flightpacket header modification to end points

• Basic question behind TCP HICCUPS• did my packet arrive at the destination with the

same headers as sent?

5Ryan Craven, Robert Beverly, and Mark Allman. “A middlebox-cooperative TCP for anon end-to-end internet”. In: Proceedings of the 2014 ACM conference on SIGCOMM.ACM. 2014, pp. 151–162.Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 12/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

TCP HICCUPS• TCP HICCUPS5

• Lightweight TCP extension that exposes in flightpacket header modification to end points

• Basic question behind TCP HICCUPS• did my packet arrive at the destination with the

same headers as sent?

5Ryan Craven, Robert Beverly, and Mark Allman. “A middlebox-cooperative TCP for anon end-to-end internet”. In: Proceedings of the 2014 ACM conference on SIGCOMM.ACM. 2014, pp. 151–162.Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 12/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

TCP HICCUPS• HICCUPS overloads 3 header fields in the TCP

3-way handshake• ISN, IPID, RWIN

• ... with a function of the packet header

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 13/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

TCP HICCUPS• HICCUPS overloads 3 header fields in the TCP

3-way handshake• ISN, IPID, RWIN

• ... with a function of the packet header

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 13/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

TCP HICCUPS• All in all, it creates an end-to-end tamper-evident

seal over the packet headers• Different than a checksum

• if some mods occur, the packet is still accepted

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 14/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Controlling both ends• Controlling both ends allows to detect middleboxes

on one path

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 15/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Controlling both ends• Controlling both ends allows to detect middleboxes

on one path

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 16/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Controlling both ends• What happens with uncontrolled server(s)?

• potentially miss a lot of middleboxes

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 17/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Controlling both ends• What happens with uncontrolled server(s)?

• potentially miss a lot of middleboxes

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 18/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Tracebox• Tracebox6

• Extension to traceroute• send TTL limited TCP probes• inspect incoming ICMP time-exceeded packets• compare the TCP probe quoted and the TCP

probe sent• in case of difference(s), a middlebox is found along

the path

• Server-independant, "One-sided"• Detect multiple modifications• Purpose

• Middlebox detection• Middlebox location

6Gregory Detal et al. “Revealing middlebox interference with tracebox”. In: Proceedingsof the 2013 conference on Internet measurement conference. ACM. 2013, pp. 1–8.Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 19/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Tracebox• Tracebox6

• Extension to traceroute• send TTL limited TCP probes• inspect incoming ICMP time-exceeded packets• compare the TCP probe quoted and the TCP

probe sent• in case of difference(s), a middlebox is found along

the path• Server-independant, "One-sided"• Detect multiple modifications

• Purpose• Middlebox detection• Middlebox location

6Gregory Detal et al. “Revealing middlebox interference with tracebox”. In: Proceedingsof the 2013 conference on Internet measurement conference. ACM. 2013, pp. 1–8.Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 19/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Tracebox• Tracebox6

• Extension to traceroute• send TTL limited TCP probes• inspect incoming ICMP time-exceeded packets• compare the TCP probe quoted and the TCP

probe sent• in case of difference(s), a middlebox is found along

the path• Server-independant, "One-sided"• Detect multiple modifications• Purpose

• Middlebox detection• Middlebox location

6Gregory Detal et al. “Revealing middlebox interference with tracebox”. In: Proceedingsof the 2013 conference on Internet measurement conference. ACM. 2013, pp. 1–8.Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 19/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Plan

1 Middleboxes

2 How to detect them ?

3 Tracebox

4 Implementations

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 20/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Tracebox

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 21/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Tracebox

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 22/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Tracebox

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 23/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Tracebox

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 24/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Tracebox

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 25/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Tracebox

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 26/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Tracebox

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 27/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Tracebox

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 28/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Tracebox

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 29/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Tracebox

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 30/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Tracebox

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 31/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Tracebox

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 32/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Tracebox

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 33/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Tracebox

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 34/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Tracebox

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 35/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Tracebox

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 36/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Tracebox

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 37/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Tracebox

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 38/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Tracebox

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 39/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Tracebox

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 40/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Tracebox

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 41/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Tracebox

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 42/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Tracebox

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 43/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Tracebox

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 44/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Tracebox

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 45/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Cannot detect all changes

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 46/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

ICMP Payload size• ICMP only includes the network header plus the

first 8 bytes of he transport header.• RFC792 (ICMPv4):

"Internet Header + 64 bits of Data Datagram"• RFC1812 (ICMPv4):

"the ICMP datagram SHOULD contain as much ofthe original datagram as possible without thelength of the ICMP datagram exceeding 576bytes."

• RFC4443 (ICMPv6):"As much of invoking packet as possible withoutthe ICMPv6 packet exceeding the minimum IPv6MTU"

• Maximal quoting by default on Linux, Cisco IOX,HP routers, Alcatel routers, PaloAlto Fiewall, etc.

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 47/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

ICMP Payload size• ICMP only includes the network header plus the

first 8 bytes of he transport header.• RFC792 (ICMPv4):

"Internet Header + 64 bits of Data Datagram"• RFC1812 (ICMPv4):

"the ICMP datagram SHOULD contain as much ofthe original datagram as possible without thelength of the ICMP datagram exceeding 576bytes."

• RFC4443 (ICMPv6):"As much of invoking packet as possible withoutthe ICMPv6 packet exceeding the minimum IPv6MTU"

• Maximal quoting by default on Linux, Cisco IOX,HP routers, Alcatel routers, PaloAlto Fiewall, etc.

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 47/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

ICMPv4 Payload size• RFC1812-compliant routers (2013, 72 PL VPs to

Alexa 5000)

0.0 0.2 0.4 0.6 0.8 1.0router proportion

0.2

0.4

0.6

0.8

1.0

cdf

• 80 % of Internet paths contains at least onRFC1812-capable router

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 48/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

ICMPv4 Payload size• RFC1812-compliant routers (2013, 72 PL VPs to

Alexa 5000)

0.0 0.2 0.4 0.6 0.8 1.0router proportion

0.2

0.4

0.6

0.8

1.0

cdf

• 80 % of Internet paths contains at least onRFC1812-capable router

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 48/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

ICMPv4 Payload size• RFC1812-compliant routers location (2013, 72 PL

VPs to Alexa 5000)

0 2 4 6 8 10

normalized distance

0.0

0.2

0.4

0.6

0.8

1.0

cd

fclose to VP core close to dst

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 49/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

ICMP detection limitation

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 50/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

ICMP detection limitation

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 51/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

ICMP detection limitation

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 52/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Use cases• Testing new protocols deployability

• MPTCP, TCP FO, TCP EDO, ...

• Testing new hardware/configurations• CGN deployment, ...

• Locating an issue• Network management/debugging

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 53/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Output Example

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 54/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Output Example

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 55/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Output Example

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 56/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Output Example

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 57/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Output Example

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 58/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Output Example

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 59/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

What about cellular networks ?• There are middleboxes too7:

7Zhaoguang Wang et al. “An untold story of middleboxes in cellular networks”. In: ACMSIGCOMM Computer Communication Review 41.4 (2011), pp. 374–385.Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 60/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

TraceboxAndroid8

• On-demand & Background probing

• A rooted version• Require to root the phone

• A non-rooted version• Non-rooted traceroutes to retreive path-level

information• Self-controlled server• Troubleshooting incentives

• Interested ?Send me an email at [email protected] tobe notified when the new version is released.

8Valentin Thirion, Korian Edeline, and Benoit Donnet. “Tracking Middleboxes in theMobile World with TraceboxAndroid”. In: Traffic Monitoring and Analysis. Springer, 2015,pp. 79–91.Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 61/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

TraceboxAndroid8

• On-demand & Background probing• A rooted version

• Require to root the phone

• A non-rooted version• Non-rooted traceroutes to retreive path-level

information• Self-controlled server• Troubleshooting incentives

• Interested ?Send me an email at [email protected] tobe notified when the new version is released.

8Valentin Thirion, Korian Edeline, and Benoit Donnet. “Tracking Middleboxes in theMobile World with TraceboxAndroid”. In: Traffic Monitoring and Analysis. Springer, 2015,pp. 79–91.Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 61/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

TraceboxAndroid8

• On-demand & Background probing• A rooted version

• Require to root the phone

• A non-rooted version• Non-rooted traceroutes to retreive path-level

information• Self-controlled server• Troubleshooting incentives

• Interested ?Send me an email at [email protected] tobe notified when the new version is released.

8Valentin Thirion, Korian Edeline, and Benoit Donnet. “Tracking Middleboxes in theMobile World with TraceboxAndroid”. In: Traffic Monitoring and Analysis. Springer, 2015,pp. 79–91.Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 61/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

TraceboxAndroid8

• On-demand & Background probing• A rooted version

• Require to root the phone

• A non-rooted version• Non-rooted traceroutes to retreive path-level

information• Self-controlled server• Troubleshooting incentives

• Interested ?Send me an email at [email protected] tobe notified when the new version is released.

8Valentin Thirion, Korian Edeline, and Benoit Donnet. “Tracking Middleboxes in theMobile World with TraceboxAndroid”. In: Traffic Monitoring and Analysis. Springer, 2015,pp. 79–91.Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 61/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Plan

1 Middleboxes

2 How to detect them ?

3 Tracebox

4 Implementations

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 62/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Tracebox implementations• Standalone Tracebox• Scamper

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 63/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Standalone Tracebox• Uses the previous mechanism to detect

middleboxes.• Implemented in C++ with Lua embedded.• Libcrafter allows for efficiently describe probes as

Scapy.• Open source• Supports Linux and Mac OSX.• http://github.com/tracebox/tracebox

• http://www.tracebox.org/

• More details:9

9Gregory Detal et al. “Revealing middlebox interference with tracebox”. In: Proceedingsof the 2013 conference on Internet measurement conference. ACM. 2013, pp. 1–8.Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 64/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Standalone Tracebox• Uses the previous mechanism to detect

middleboxes.• Implemented in C++ with Lua embedded.

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 65/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Standalone Tracebox• Uses the previous mechanism to detect

middleboxes.• Implemented in C++ with Lua embedded.• Libcrafter allows for efficiently describe probes as

Scapy.• Open source• Supports Linux and Mac OSX.• http://github.com/tracebox/tracebox

• http://www.tracebox.org/

• More details:10

10Gregory Detal et al. “Revealing middlebox interference with tracebox”. In: Proceedingsof the 2013 conference on Internet measurement conference. ACM. 2013, pp. 1–8.Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 66/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Scamper• All-around parallelized topology/performance

analyzing tool.• Implements various simple and complex

measurement methods (ping, traceroute, dealias,tbit, ...).

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 67/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Scamper

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 68/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Scamper• Native output format: warts.• IPv6 support• Open source• Supports FreeBSD, OpenBSD, NetBSD, Linux,

MacOS X, Solaris, Windows, and more.• http://www.caida.org/tools/measurement/scamper/

• Debian/Ubuntu packages, FreeBSD ports, ...• More details:11

11Matthew Luckie. “Scamper: a scalable and extensible packet prober for activemeasurement of the internet”. In: Proceedings of the 10th ACM SIGCOMM conference onInternet measurement. ACM. 2010, pp. 239–245.Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 69/70

u n i v e r s i t e d e l i e g e f a c u l t e d e s s c i e n c e s a p p l i q u e e s

Thank you !

Korian Edeline, Benoit Donnet — Tracking Middleboxes with Tracebox IETF93: HOPSSlide 70/70