Reliable Transport Protocols Should Forbid Reneging
Post on 25-Feb-2016
35 Views
Preview:
DESCRIPTION
Transcript
Reliable Transport Protocols Should Forbid Reneging
Nasif EkizPaul D. Amer, Professor
Preethi NatarajanErtugrul Yilmaz
Jon LeightonAbu Rahman
sponsored byU.S. Army Research Lab
2
Outline
• transport layer selective acknowledgments (SACKs)– what are SACKs?– how are SACKs renegable?
• what is the gain in making selective acks non-renegable?• what is the penalty in making selective acks non-renegable?
Conclusion: selective acks for reliable transport protocols (e.g., TCP, SCTP) should be non-renegable
Transport layer receive buffer
3 4 5 7 9 11 12 13
ordered data (ACKed)out-of-order data (SACKed)available space
receiving application
Internet
data sender
receive buffer
4
Types of acknowledgments
• For ordered data - cumulative ACK n– bytes [ ... to n-1 ] (TCP) [RFC 793]
– segments [ ... to n ] (SCTP) [RFC 2960]
• For out-of-order data - selective ACK (SACK) m-n– bytes [ m to n-1 ] (TCP) [RFC 2018]
– segments [ m to n ] (SCTP) [RFC 2960]
prevent unnecessary retransmissions during loss recovery improve throughput when multiple losses in same window
5
Types of ACKs
data senderreceive buffer
data receiver
3
11
ACK 1 2
ACK 2 2
4ACK 2, SACK 4-4 45ACK 2, SACK 4-5 4 56
4 5 6ACK 2, SACK 4-6
ACK 6 3
63 4 5
6
What is reneging?
[RFC 2018]: “The SACK option is advisory, in that, while it notifies the data sender that the data receiver has received the indicated segments, the data receiver is permitted to later discard data which have been reported in a SACK option.”
discarding SACKed data before delivery to the receiver application (or socket) is “reneging”
TCP and SCTP allow reneging - data sender retains copies of all SACKed data until ACKed
7
data senderreceiver buffer
1
1
2
3
4
5
6 4 5 6
data receiver
ACK 1
ACK 2, SACK 4-4
ACK 2, SACK 4-5
ACK 2 2
4
4 5
ACK 2, SACK 4-6
Reneging example
3ACK 3 3
ACK 3, SACK 7-7 77
OS needs memory and reneges
8
Summary of reneging
• Reneging : data receiver SACKs data, and later discards it (i.e., SACK information is “advisory”, not a delivery guarantee)
• Reneging is discouraged but permitted• Data sender keeps data in a send buffer until
cumulatively ACKed (i.e., cum ack is a guarantee)
Special Case for SCTP – out-of-order data already delivered to the application is non-renegable by definition
Special case for SCTP: unordered data
data senderreceive buffer
2
data receiver
11
ACK 1
3 3ACK 1, SACK 3-3
3 55
ACK 1, SACK 3-5
9
3 44
ACK 1, SACK 3-4 unordered
6
? ?unordered
Special case for SCTP: unordered data
data senderreceive buffer
1
3
3
3
4
5
1
2
3
4
5
data receiver
ACK 1
ACK 1, SACK 3-3
232 5ACK 6
ACK 1, SACK 3-4
ACK 1, SACK 3-5
77ACK 7
10
unordered
3 5
6 3 5 6ACK 1, SACK 3-6
unordered
11
We argue that tolerating reneging is wrong
• Suppose SACKs were a guarantee of delivery, not advisory– All SACKs are non-renegable (NR-SACKs)– Data receiver takes responsibility for all selectively acked data– Data sender can remove NR-SACKed data from send buffer
Part I: What is the gain in forbidding selective acks?
always improved send buffer utilization (TCP and SCTP)“Non-renegable selective acks for SCTP” Int'l Conf on Network Protocols (ICNP), Orlando, 10/08
sometimes improved throughput (SCTP)“Throughput analysis of Non-Renegable Selective Acks for SCTP” Computer Communications, 33(16), 10/10
Send buffer utilization (SACK)
13
data sender receive bufferdata receiver
send buffer
11
ACK 1 1100%
221100%
3 3ACK 1, SACK 3-3
2 31100%
44
ACK 1, SACK 3-4 42 31100%
55
ACK 1, SACK 3-5 42 3 5100%
send buffer blocking
2ACK 5
42 3 525%2
42 3 575%
42 3 525%42 3 550%
66100%
Send buffer utilization (NR-SACK)
data sender receiver bufferdata receiver
send buffer
11
ACK 11100%
ACK 7 2262 5 7100%
72 6 8100%ACK 8 8
8
982 7 9 9100%
ACK 9
221100%
4ACK 1, NR-SACK 3-4
42 31100% 4
5ACK 1, NR-SACK 3-5
42 3 5100% 5
52 4 6100% 6ACK 1, NR-SACK 3-6 6
3ACK 1, NR-SACK 3-3
2 31100% 3
62 5 7100% 7ACK 1, NR-SACK 3-7 7
1092 8 10100% 10
ACK 10100% 11108 9 11
no send buffer
blocking
15
NR-SACK ns-2 simulation
16
NR-SACK FreeBSD implementation
17
Send buffer utilization (ns-2)
NR-SACK
As traffic load increases, NR-SACKs better utilize
send buffer
Send
Buff
er U
tiliza
tion
SACKSACK 64KSACK 32K
∞
18
Send buffer utilization (FreeBSD)Se
nd B
uffer
Util
izatio
n
∞
19
Throughput gains (ns-2) (only for SCTP not TCP)
-1% 0% 1% 2% 3% 4% 5% 6%0%
2%
4%
6%
8%
10%
12%
14%
16%SCTP 32K 1000Mbps 45ms (BDP>32K)SCTP 32K 100Mbps 45ms (BDP>32K)SCTP 32K 10Mbps 45ms (BDP>32K)SCTP 32K 3Mbps 45ms (BDP=32.95K)SCTP 32K 2Mbps 45ms (BDP<32K)SCTP 32K 1Mbps 45ms (BDP<32K)
Loss Rate
Thro
ughp
ut G
ain
NR-SACKs never do worse than
SACKs
20
Changing TCP or SCTP to non-reneging protocol is easy:• SACK semantics changed from advisory to permanent• if data receiver needs to renege, data receiver MUST
RESET the connection ( this is the penalty)
Let’s assume transport protocols are designed to forbid data reneging
Part II: What is penalty in forbidding selective acks?
We argue that tolerating reneging is wrong
• Suppose reneging occurs 1 in 100,000 TCP (or SCTP) flows• Case A (current practice): reneging allowed
– 99,999 non-reneging connections underutilize send buffer (and for SCTP may achieve lower throughput)
– 1 reneging connection continues (maybe...)• Case B (proposed change): reneging forbidden
– 99,999 connections have equal or better send buffer utilization (and for SCTP potential greater throughput)
– 1 reneging connection is RESET
Hypothesis: “data reneging rarely if ever occurs in practice”
• Data reneging has never been studied– Does data reneging happen or not?– If reneging happens, how often?
How big is the penalty?answer: depends on how often reneging happens
receive buffer
1
3
1
2
3
3 4 55
22
Data Sender Data Receiver
Router
OS reneges
State of receivebuffer
Detecting TCP reneging at a router
4
ACK 1, SACK 3-4 3 4ACK 1, SACK 3-4
6 3 4 5 6ACK 1, SACK 3-6
ACK 1, SACK 3-6
77ACK 2, SACK 7-7
ACK 2, SACK 3-6 ? reneging detected
Model to detect reneging• Current state (C) and new SACK (N) are compared• 4 possibilities:
SACK 12-17 SACK 12-15 NewCurrent
SACK 12-13 SACK 12-17
SACK 22-25 SACK 12-17
SACK 12-17 SACK 15-20
Model to detect renegingCurrent state (C)New SACK (N)Reneging (R)
Model to detect reneging
CAIDA* trace TCP flow filter
RenegDetect
tsharkeditcapmergecap
~4600 lines of C codeACK reordering check
TCP flows with SACKs reneging?
yesorno
.pcap
*Cooperative Association for Internet Data Analysis
Model verification• RenegDetect was tested with synthetic TCP flows
– created reneging flows with text2pcap– all reneging flows were identified correctly
• RenegDetect was tested with real TCP flows from CAIDA Internet traces– at first, reneging seemed to occur frequently– on closer inspection, we found that many SACK
implementations are incorrect !
“Misbehaviors in TCP SACK Generation” (Ekiz, Rahman, Amer) (ACM SIGCOMM Computer Communication Review, April 2011)
Incorrect SACK implementations
Operating System MisbehaviorA B C D E F G
FreeBSD 5.3, 5.4 Y YLinux 2.2.20 (Debian 3) YLinux 2.4.18 (Red Hat 8) YLinux 2.4.22 (Fedora 1) YLinux 2.6.12 (Ubuntu 5.10) YLinux 2.6.15 (Ubuntu 6.06) YLinux 2.6.18 (Debian 4) YOpenBSD 4.2, 4.5, 4.6, 4.7 Y YOpenSolaris 2008.05 Y YOpenSolaris 2009.06 Y YSolaris 10 YWindows 2000 Y Y Y Y YWindows XP Y Y Y Y YWindows Server 2003 Y Y Y Y YWindows Vista Y YWindows Server 2008 Y YWindows 7 Y Y
• Event A: TCP flow reneges• Hypothesis:
• We want to design an experiment which rejects H0 with 95% confidence to conclude
• Our experiment will observe n TCP flows hoping to NOT find even a single instance of reneging
• Using MAPLE, n ≥ 299,572
Experiment design – how to “prove” reneging does not happen?
31
Questions ?
(thank you)
TCP Send buffer utilization (SACK)
32
data sender receiver buffer
1
2
3
4
5
6
2
data receiver
ACK 1
ACK 1, SACK 3-3
ACK 5
ACK 1, SACK 3-4
ACK 1, SACK 3-5
send buffer
1
21
2 31
42 31
42 3 5
42 3 5
42 3 5
42 3 5
6
100%100%
100%100%
100%
75%
50%
25%
100%
send buffer blocking
1
3
32 4 5
3 4
3 4 5
3 4 5
3 4 5
3 4 5
ACK 6 6
TCP Send buffer utilization (NR-SACK)
33
data sender receiver buffer
1
2
3
4
5
8
data receiver
ACK 1
ACK 1, NR-SACK 3-3
ACK 7
ACK 1, NR-SACK 3-4
ACK 1, NR-SACK 3-5
send buffer
1
21
2 31
42 31
42 3 5
52 4 6
62 5 7
72 6 8
100%
100%
100%100%
100%
100%100%
100%
82 7 9100%
1
3
3 4
5 3 4
5 3 4 6
5 3 4 6 7
5 2 3 4 6 7
8
6
7
2
ACK 1, NR-SACK 3-6
ACK 1, NR-SACK 3-7
9
100% 10 9
10
ACK 9
ACK 10
62 5 7100%
ACK 8
92 8 10
100% 11108 9 11
NO send buffer blocking
Data reneging in OSes
• Reneging in Linux (version 2.6.28.7)– tcp_prune_ofo_queue() deletes out-of-order data
• Reneging in FreeBSD, Mac OS– net.inet.tcp.do_tcpdrain sysctl turns reneging on/off– tcp_drain() deletes out-of-order data
Data reneging in Linux
3. Inferring the state of receive buffer
TCP Segments with n SACK options
Enough space for another SACK
option
Not enough space for another SACK
option
n=1 ~88% 0%
n=2 ~11% 0%
n=3 0.7% 0.20%
n=4 n/a 0.15%
Total number of TCP segments 780,798 (100%)
3. Inferring the state of receive buffer
TCP Segments with n SACK options
Enough space for another SACK
option
Not enough space for another SACK
option
n=1 ~88% 0%
n=2 ~11% 0%
n=3 0.7% 0.20%
n=4 n/a 0.15%
Total number of TCP segments 780,798 (100%)
38
NR-SACK Negotiation
INIT – NR-SACKs Supported
INIT-ACK – NR-SACKs Supported
COOKIE-ACK
COOKIE-ECHO
SCTP Data Transfer
SCTP Association Startup
DATA
NR-SACKs(cum-ack,gap-ack,nr-gap-ack,dup-
TSN)
39
NR-SACK Chunk
Type = 0x10 Chunk Flags Chunk Length
Cumulative TSN Ack
Advertised Receiver Window Credit
Number of NR-Gap-Ack Blocks = MNumber of Gap Ack Blocks = N
Gap Ack Block #1 Start Gap Ack Block #1 End
Duplicate TSN 1
Duplicate TSN X
Gap Ack Block #N Start Gap Ack Block #N End
Number of Duplicate TSNs = X Reserved
NR-Gap Ack Block #1 Start NR-Gap Ack Block #1 End
NR-Gap Ack Block #M Start NR-Gap Ack Block #M End
ACK
SACK
NR-SACK
D-SACK
40
When is data non-renegable? case 2: multistreaming
data senderreceiver buffer
data receiver
SID : Stream Identifier SSN : Stream Sequence Number
41
When is data non-renegable? case 2: multistreaming
data senderreceiver buffer
11
data receiver
ACK 1 SID: 1 SSN: 1
SID : Stream Identifier SSN : Stream Sequence Number
42
When is data non-renegable? case 2: multistreaming
data senderreceiver buffer
1
1
2
data receiver
ACK 1 SID: 1 SSN: 1
SID: 2 SSN: 1
SID : Stream Identifier SSN : Stream Sequence Number
43
When is data non-renegable? case 2: multistreaming
data senderreceiver buffer
1
3
1
2
3
data receiver
ACK 1
ACK 1, SACK 3-3
SID: 1 SSN: 1
SID: 2 SSN: 1
SID: 1 SSN: 2
SID : Stream Identifier SSN : Stream Sequence Number
44
When is data non-renegable? case 2: multistreaming
data senderreceiver buffer
1
3
4
1
2
3
4
data receiver
ACK 1
ACK 1, SACK 3-3 ACK 1, SACK 3-4
SID: 1 SSN: 1
SID: 2 SSN: 1
SID: 1 SSN: 2
SID: 2 SSN: 2
SID : Stream Identifier SSN : Stream Sequence Number
45
When is data non-renegable? case 2: multistreaming
data senderreceiver buffer
1
3
4
1
2
3
4
5
data receiver
ACK 1
ACK 1, SACK 3-3 ACK 1, SACK 3-4
SID: 1 SSN: 1
SID: 2 SSN: 1
SID: 1 SSN: 2
SID: 2 SSN: 2
SID: 1 SSN: 3
SID : Stream Identifier SSN : Stream Sequence Number
? ?
46
When is data non-renegable? case 2: multistreaming
data senderreceiver buffer
1
3
4
4 5
1
2
3
4
5
data receiver
ACK 1
ACK 1, SACK 3-3 ACK 1, SACK 3-4
ACK 1, SACK 3-5
SID: 1 SSN: 1
SID: 2 SSN: 1
SID: 1 SSN: 2
SID: 2 SSN: 2
SID: 1 SSN: 3
SID : Stream Identifier SSN : Stream Sequence Number
47
When is data non-renegable? case 2: multistreaming
data senderreceiver buffer
1
3
4
4
4
5
6
1
2
3
4
5
6
data receiver
ACK 1
ACK 1, SACK 3-3 ACK 1, SACK 3-4
ACK 1, SACK 3-5
ACK 1, SACK 3-6
SID: 1 SSN: 1
4 6
SID: 2 SSN: 1
SID: 1 SSN: 2
SID: 2 SSN: 2
SID: 1 SSN: 3
SID: 2 SSN: 3
SID : Stream Identifier SSN : Stream Sequence Number
48
When is data non-renegable? case 2: multistreaming
data senderreceiver buffer
1
3
4
4
4
5
6
1
2
3
4
5
6
242 6
data receiver
ACK 1
ACK 1, SACK 3-3
ACK 6
ACK 1, SACK 3-4
ACK 1, SACK 3-5
ACK 1, SACK 3-6
SID: 1 SSN: 1
4 6
SID: 2 SSN: 1
SID: 1 SSN: 2
SID: 2 SSN: 2
SID: 1 SSN: 3
SID: 2 SSN: 3
SID: 2 SSN: 1
SID : Stream Identifier SSN : Stream Sequence Number
49
When is data non-renegable? case 2: multistreaming
data senderreceiver buffer
1
3
4
4
4
5
6
1
2
3
4
5
6
7
242 6
7
data receiver
ACK 1
ACK 1, SACK 3-3
ACK 6
ACK 1, SACK 3-4
ACK 1, SACK 3-5
ACK 1, SACK 3-6
SID: 1 SSN: 1
4 6
ACK 7
SID: 2 SSN: 1
SID: 1 SSN: 2
SID: 2 SSN: 2
SID: 1 SSN: 3
SID: 2 SSN: 3
SID: 2 SSN: 1
SID: 1 SSN: 4
SID : Stream Identifier SSN : Stream Sequence Number
50
When is data non-renegable? Case 3: OS guarantee
Data SenderReceiver Buffer
Data Receiver
51
When is data non-renegable? Case 3: OS guarantee
Data SenderReceiver Buffer
11
Data Receiver
ACK 1
52
When is data non-renegable? Case 3: OS guarantee
Data SenderReceiver Buffer
1
1
2
Data Receiver
ACK 1
53
When is data non-renegable? Case 3: OS guarantee
Data SenderReceiver Buffer
1
3
1
2
3
Data Receiver
ACK 1
ACK 1, SACK 3-3
54
When is data non-renegable? Case 3: OS guarantee
Data SenderReceiver Buffer
1
3
3 4
1
2
3
4
Data Receiver
ACK 1
ACK 1, SACK 3-3 ACK 1, SACK 3-4
55
When is data non-renegable? Case 3: OS guarantee
Data SenderReceiver Buffer
1
3
3
3
4
4 5
1
2
3
4
5
Data Receiver
ACK 1
ACK 1, SACK 3-3 ACK 1, SACK 3-4
ACK 1, SACK 3-5
56
When is data non-renegable? Case 3: OS guarantee
Data SenderReceiver Buffer
1
3
3
3
3
4
4
4
5
5
1
2
3
4
5
66
Data Receiver
ACK 1
ACK 1, SACK 3-3 ACK 1, SACK 3-4
ACK 1, SACK 3-5
ACK 1, SACK 3-6
3 4 5 6
57
When is data non-renegable? Case 3: OS guarantee
Data SenderReceiver Buffer
1
3
3
3
3
4
4
4
5
5
1
2
3
4
5
6
22
6
Data Receiver
ACK 1
ACK 1, SACK 3-3
ACK 6
ACK 1, SACK 3-4
ACK 1, SACK 3-5
ACK 1, SACK 3-6
3 4 5 6
3 4 5 6
58
When is data non-renegable? Case 3: OS guarantee
Data SenderReceiver Buffer
1
3
3
3
3
4
4
4
5
5
1
2
3
4
5
6
7
2
6
7
Data Receiver
ACK 1
ACK 1, SACK 3-3
ACK 6
ACK 1, SACK 3-4
ACK 1, SACK 3-5
ACK 1, SACK 3-6
3 4 5 6
ACK 7
2 3 4 5 6
59
When is data non-renegable? Case 3: OS guarantee
Data SenderReceiver Buffer
1
3
3
3
3
4
4
4
5
5
1
2
3
4
5
6
7
2
6
7
Data Receiver
ACK 1
ACK 1, SACK 3-3
ACK 6
ACK 1, SACK 3-4
ACK 1, SACK 3-5
ACK 1, SACK 3-6
3 4 5 6
ACK 7
*
* OS guarantees not to renege
2 3 4 5 6
top related