1 Forward Error Correction Shimrit Tzur-David School of Computer Science and Engineering Hebrew University of Jerusalem.

11

Forward Error Forward Error CorrectionCorrection

Shimrit Tzur-DavidShimrit Tzur-David

School of Computer Science and School of Computer Science and EngineeringEngineering

Hebrew University of JerusalemHebrew University of Jerusalem

22

AgendaAgenda

IntroductionIntroductionThe FEC algorithmThe FEC algorithm

ImplementationImplementation

PerformancePerformance

33

IntroductionIntroductionLoss of packets is a fact of life in computers networks. Loss of packets is a fact of life in computers networks. The usual approach to recover from losses is to The usual approach to recover from losses is to retransmit missing packets on request. The presented retransmit missing packets on request. The presented approach is to add an error recovery mechanism that approach is to add an error recovery mechanism that enable the receiver to recover all useful data without enable the receiver to recover all useful data without sending explicit request for missing packets. sending explicit request for missing packets.

Such a mechanism can be implemented by applying a Such a mechanism can be implemented by applying a redundant encoding to the source data, so that even in redundant encoding to the source data, so that even in presence of packet losses, sufficient information is presence of packet losses, sufficient information is conveyed to the receiver to allow successful conveyed to the receiver to allow successful reconstruction of the original data. reconstruction of the original data.

44

Multicast CommunicationMulticast Communication

The possible lack of correlation between losses The possible lack of correlation between losses at different receivers can cause severe at different receivers can cause severe scalability problems.scalability problems.

It will be helpful if in this situation, the It will be helpful if in this situation, the receiver will enable to recover all useful data receiver will enable to recover all useful data without sending explicit requests for missing without sending explicit requests for missing packets.packets.

55

The goalThe goal

The idea of this research is to show that under some The idea of this research is to show that under some circumstances, incorporating error correction circumstances, incorporating error correction schemes inside the TCP achieves better performance schemes inside the TCP achieves better performance than that of retransmission schemes.than that of retransmission schemes.

Error correcting schemes for missing packets can be Error correcting schemes for missing packets can be classified as (k, n) schemes, where classified as (k, n) schemes, where kk is the number of is the number of data packets in a FEC block and data packets in a FEC block and nn is the number of all is the number of all the packets in the FEC block (including data packets the packets in the FEC block (including data packets and the error correcting packets).and the error correcting packets).

66

AgendaAgenda

IntroductionIntroduction

The FEC algorithmThe FEC algorithmImplementationImplementation

PerformancePerformance

77

The FEC AlgorithmThe FEC Algorithm

Implemented by applying a redundant Implemented by applying a redundant encoding to the source data.encoding to the source data.

In presence of packet losses, sufficient In presence of packet losses, sufficient information is conveyed to the receiver to information is conveyed to the receiver to allow successful reconstruction of the original allow successful reconstruction of the original data.data.

Such an encoding is called erasure code. Such an encoding is called erasure code.

88

Erasure CodeErasure Code

The idea: encode a set of k source data packets The idea: encode a set of k source data packets into a set of n > k encoded data packets.into a set of n > k encoded data packets.

Any subset of k encoded packets allow the Any subset of k encoded packets allow the reconstruction of the original source.reconstruction of the original source.

Such a code is called an (n, k) erasure code.Such a code is called an (n, k) erasure code.

The goalThe goal: produce the encoded packets given : produce the encoded packets given arbitrary values for k and n and make the arbitrary values for k and n and make the encoding/decoding procedure sufficiently fast encoding/decoding procedure sufficiently fast for practical use.for practical use.

99

Erasure Code – Cont.Erasure Code – Cont.

1010

A Simple Erasure CodeA Simple Erasure Code

A polynomial of degree k-1 is completely A polynomial of degree k-1 is completely specified by its value in k different points.specified by its value in k different points.Consider the source data packets as the Consider the source data packets as the coefficients of a polynomial P of degree k-1.coefficients of a polynomial P of degree k-1.The encoded packets are the values of P The encoded packets are the values of P computed in n different points.computed in n different points.The decoding: recover the coefficient of P given The decoding: recover the coefficient of P given its values in k points.its values in k points.This code is called Vandermond code.This code is called Vandermond code.

1111

Systematic CodesSystematic Codes

The transmission include a verbatim copy of The transmission include a verbatim copy of the source data.the source data.

No decoding effort is necessary in absence of No decoding effort is necessary in absence of errors.errors.

The Vandermond code is not systematic, but it The Vandermond code is not systematic, but it can be turned into a systematic code by simple can be turned into a systematic code by simple algebraic manipulations.algebraic manipulations.

1212

Systematic Codes – Cont.Systematic Codes – Cont.

- the coefficients

- the values of the polynomial

1313

Vandermond matrix (G)Vandermond matrix (G)

1414

Systematic Codes – Cont.Systematic Codes – Cont.

Any minor of degree k extracted from this matrix is Any minor of degree k extracted from this matrix is invertible.invertible.

This property still holds if we do linear combinations This property still holds if we do linear combinations of the columns.of the columns.

We can manipulate the matrix in such a way that the We can manipulate the matrix in such a way that the upper k rows become the identity matrix.upper k rows become the identity matrix.

After the transformation: G*X will give us - After the transformation: G*X will give us -

We have obtained a systematic code.We have obtained a systematic code.

1515

The encodingThe encoding

The source data:

G is an (n,k) matrix, any subset of k rows of G, G`, is an invertible matrix.

The sender sends , but the receiver gets y which contains k values from it.

The encoding data is:

1616

The decodingThe decoding

The original data can be reconstruct by using k The original data can be reconstruct by using k equations on the known values in y.equations on the known values in y.

1717

AgendaAgenda

IntroductionIntroduction

The FEC algorithmThe FEC algorithm

ImplementationImplementationPerformancePerformance

1818

ImplementationImplementation

End-to-end error correction should be addressed at the End-to-end error correction should be addressed at the transport layer, since it responsible for packet transport layer, since it responsible for packet reliability.reliability.

In the handshake process, the connection initiator add a In the handshake process, the connection initiator add a FEC flag inside the TCP options inside the ‘Syn’ FEC flag inside the TCP options inside the ‘Syn’ packet.packet.

If the receiver also knows how to deal with FEC, its If the receiver also knows how to deal with FEC, its ‘Syn-Ack’ packet will also contain the FEC flag.‘Syn-Ack’ packet will also contain the FEC flag.

The sender now knows that it can encode its data. The sender now knows that it can encode its data.

1919

SenderSender

For each new packet, it adds the FEC header.For each new packet, it adds the FEC header.The FEC header contains n, k and the index of The FEC header contains n, k and the index of the packet in the block.the packet in the block.It saves the packet payload in the FEC buffer.It saves the packet payload in the FEC buffer.It sends the packet.It sends the packet.When it sends a full data block (k packets), it When it sends a full data block (k packets), it encodes the data using the data in FEC buffer, encodes the data using the data in FEC buffer, creates the FEC packets and sends them as creates the FEC packets and sends them as well.well.

2020

Sender – Pseudo CodeSender – Pseudo Code

Sendmsg(buf){Sendmsg(buf){

while (length(buf) > 0)while (length(buf) > 0)

create new packetcreate new packet

fec_index++fec_index++

add fec header to packet {fec_index, k, n}add fec header to packet {fec_index, k, n}

data_len=min(mss, length(buf))data_len=min(mss, length(buf))

take data_len bytes from buf and add to the take data_len bytes from buf and add to the payloadpayload

save payload in fec_bufsave payload in fec_buf

send packetsend packet

2121

Sender – Pseudo Code – Cont.Sender – Pseudo Code – Cont.

if fec_index == kif fec_index == k

for(i=k; i<n; i++)for(i=k; i<n; i++)

fec_payload=encode(fec_buf,i)fec_payload=encode(fec_buf,i)

create_fec_packet(fec_payload)create_fec_packet(fec_payload)

send fec packetsend fec packet

fec_index = 0;fec_index = 0;

}}

2222

ReceiverReceiver

For each packet that it gets, it pulls the FEC For each packet that it gets, it pulls the FEC header.header.If this is redundant FEC packet it throws its If this is redundant FEC packet it throws its payload, else it adds the packet to the FEC queue.payload, else it adds the packet to the FEC queue.If this is a FEC packet, it doesn’t send its payload If this is a FEC packet, it doesn’t send its payload to the application layer.to the application layer.If there is a block with up to n-k missing packets, If there is a block with up to n-k missing packets, it decodes it (using FEC queue), handles queues, it decodes it (using FEC queue), handles queues, and sends the reconstructed data to the and sends the reconstructed data to the application layer.application layer.

2323

Receiver – Pseudo Code Receiver – Pseudo Code

dataqueue(packet){dataqueue(packet){

get_fec_header (packet,fec_index, k, n)get_fec_header (packet,fec_index, k, n)

If (!redundand && !exist)If (!redundand && !exist)

add packet to fec_queue in the right order (by s.n.)add packet to fec_queue in the right order (by s.n.)

If (fec_index > k)If (fec_index > k)

payload len = 0payload len = 0

2424

Receiver – Pseudo Code – Cont.Receiver – Pseudo Code – Cont.for (i = 0; i < n - k; i++)for (i = 0; i < n - k; i++)

if there is a full block with ‘i’ holesif there is a full block with ‘i’ holesif i == 0if i == 0

delete block from queuedelete block from queuebreakbreak

elseelsepayloadpayload[][] = decode(fec_queue) = decode(fec_queue)simulating receiving the missing simulating receiving the missing packets with the decoded payloadpackets with the decoded payloaddelete block from queuedelete block from queuebreakbreak

}}

2525

AgendaAgenda

IntroductionIntroduction

The FEC algorithmThe FEC algorithm

ImplementationImplementation

PerformancePerformance

2626

PerformancePerformance

The measurement were taken in the LAN, with The measurement were taken in the LAN, with n=9 and k=8.n=9 and k=8.We compared the transferring time with files We compared the transferring time with files in sizes: 50MB, 100MB, 150MB and 200MB.in sizes: 50MB, 100MB, 150MB and 200MB.We demonstrated packet loss by throwing the We demonstrated packet loss by throwing the picked packets at the receiver side.picked packets at the receiver side.We made the measurements with different loss We made the measurements with different loss probabilities: 0.025, 0.05, 0.075, 0.01, 0.0125, probabilities: 0.025, 0.05, 0.075, 0.01, 0.0125, 0.015, 0.0175, 0.02, 0.025, 0.03, 0.035, 0.04.0.015, 0.0175, 0.02, 0.025, 0.03, 0.035, 0.04.

2727

50MB File50MB File

2828

100MB File100MB File

2929

150MB File150MB File

3030

200MB File200MB File

3131

ConclusionsConclusions

As we can see, all the graphs show better results until As we can see, all the graphs show better results until the loss probability reaches to 3%.the loss probability reaches to 3%.In loss probability higher than 3%, the probability to In loss probability higher than 3%, the probability to lose the FEC packets increases, in this case there is no lose the FEC packets increases, in this case there is no decoding process to reconstruct it.decoding process to reconstruct it.The receiver can lose more than one packet in a The receiver can lose more than one packet in a block, in this case it needs to wait until it gets one of block, in this case it needs to wait until it gets one of the missing packets (by retransmission).the missing packets (by retransmission).If one of the FEC packets is lost, and another packet If one of the FEC packets is lost, and another packet in the next block is lost, we reconstruct the packet in in the next block is lost, we reconstruct the packet in the next block, but we still can’t transfer the data to the next block, but we still can’t transfer the data to the application layer.the application layer.

3232

TODOTODO

The system should sense its traffic and if there The system should sense its traffic and if there are losses in which the FEC is useful (and still are losses in which the FEC is useful (and still TCP friendly) turn it on. TCP friendly) turn it on.

If there is a high percentage of losses, turn off If there is a high percentage of losses, turn off the FEC algorithm.the FEC algorithm.

The system should be able to handle change of The system should be able to handle change of n and k during a connection.n and k during a connection.

3333

THE ENDTHE END