SCHOOL OF COMPUTING SCIENCE SIMON FRASER UNIVERSITY CMPT 820 : Error Mitigation Schaar and Chou, Multimedia over IP and Wireless Networks: Compression,

SCHOOL OF COMPUTING SCIENCESIMON FRASER UNIVERSITY

CMPT 820 : Error MitigationSchaar and Chou, Multimedia over IP and Wireless Networks: Compression, Networking, and Systems

Presented by: Azin Dastpak

2

Outline

Motivation Operational Encoder Control Adaptive Intra Updates Interactive Error Control

Mode 1:Acknowledged Reference Area Only Mode 2:Synchronized Reference Frames Mode 3:Regular Prediction with Limited Error

Propagation Mode 4:Unrestricted Reference Areas with Expected

Distortion Update Conclusion Questions

3

Motivation

Error propagation is the major problem when transmitting MCP-coded video over lossy channels.

Encoder shall change its encoding behavior if it’s aware that decoder has experienced loss, which may sacrifice some compression efficiency.

4

Motivation

a)no error robustness b)adaptive intra updates c)interactive error control

5

Operational Encoder Control

The encoder is responsible to select one of many encoding parameters Motion vectors MB modes Quantization parameters Reference frames Spatial and temporal resolution

considering constraints bit rate encoding and transmission delay complexity Buffer size

6

Operational Encoder Control

7

Operational Encoder Control

Bad decisions in parameter selection may lead to poor results in: Coding efficiency Error resilience Both..!

8

Operational Encoder Control – Compression Efficiency

The distortion of MB b coded with option m:

The coding mode selected for MB b:

i

imbibmb ssd2

,,,, '

b )(minarg ,,*

mbOmbOmb rdm

9

Adaptive Intra Updates

In the presence of errors, introducing more intra-coded MBs is important.

Sequence characteristics and bit rate influence the appropriate percentage of intra updates.

Modify the selection of the coding modes to take into account the influence of lossy channels. Replace the encoding distortion by decoder

distortion.

2

,,, ),(')(' mCssCd ttbtbtmb

10

Adaptive Intra Updates

The channel behavior is not deterministic and channel realization observed by the decoder is unknown

The receiver can send reports on the experienced loss and delays statistics for the encoder to compute expected distortion:mbd , }),'('{)}'('{

2

,,',' mcssEcdE ttbtbCtmbC tt

11

Interactive Error Control

In previous scenario only the statistics of the channel process C’ was known to the encoder.

In this scenario the -frame delayed version of the loss process experienced at the receiver is known at the encoder. Sending Acks for correctly received data units Sending Nacks for missing slices Both

The goal is to reduce, limit or even completely avoid error propagation by integrating the decoder state information in the encoding process

tC

12

Interactive Error Control

At encoder side, each generated data unit Pi is assigned a decoder state.

This information will be integrated in a modified rate-distortion optimized operational encoder control

},,{, OAKNACKACKC ienc

b )'(minarg ,','*

mbOmbOmb rdm

13

Interactive Error Control

We have four different operation modes, differing only in set of coding option available at encoder, O’ and the applied distortion metric

Mode 1: Acknowledged Reference Area Only Mode 2: Synchronized Reference Frames Mode 3: Regular Prediction with Limited Error

Propagation Mode 4: Unrestricted Reference Areas with

Expected Distortion Update.

mbd ,'

14

Mode 1: Acknowledged Reference Area Only

Only the decoded representations of data units that have positively acknowledged at the encoder are allowed to be referenced in the encoding process.

Encoding options are restricted to acknowledged areas only,

If no reference area is available or no satisfying match is found in the accessible reference area, intra coding is applied.

tACKOO ,'

15

Mode 1: Acknowledged Reference Area Only

Error may be visible in the presentation of a single frame but error propagation and reference frame mismatch are completely avoided.

16

Mode 1: Acknowledged Reference Area Only

Average PSNR versus bit rate for feedback mode 1

17

Mode 2: Synchronized Reference Frames

In this mode not only positively acknowledged data units but also a concealed version data units with decoder state Nack are allowed to be referenced.

For the reference frames to be synchronized the encoder must apply exactly the same error concealment as the decoder.

18

Mode 2: Synchronized Reference Frames

19

Mode 2: Synchronized Reference Frames

The advantage of of mode 2 is for : Low bit rates Delays < Nref -1

20

Mode 3: Regular Prediction with Limited Error Propagation

Mode 1 and 2 are mainly suitable in cases of higher loss rates, when the loss rates are low the performance is degraded by longer prediction chains due to feedback delay.

In this case we only alter the prediction in the encoder in case of the reception of a NAK. In this case the reference area and option set will be changed to acknowledged and concealed image parts.

21

Mode 3: Regular Prediction with Limited Error Propagation

22

Mode 3: Regular Prediction with Limited Error Propagation

For delay=0 mode 2 and 3 are the same. For increasing delays, mode 3 performs

significantly worse than mode 2.

23

Mode 4: Unrestricted Reference Areas with Expected Distortion Updates

In this case the set of encoding options is not altered, that is O’=O.

Only for the data units with outstanding acknowledgement at the encoder, the randomness of the observed channel state is considered. For all other data units the observed channel state is no longer random.

mbd ,')}'('{ ,}'{ imbC CdE

i

)}('{ ,}{ imbC CdEi

OAKC ienc ,

OAKC ienc ,

24

Mode 4: Unrestricted Reference Areas with Expected Distortion Updates

Compared to mode 1 and 2 this method is beneficial if the feedback is significantly delayed.

Compared to mode 3, it reduces the unsatisfying performance in case of error propagation

For this mode turns to mode selection without any feedback at all

For this mode is identical to mode 2 and 3.

0

25

Conclusion

Error propagation is the most important issue in error-prone video transmission

If no feedback is available, increased percentage of intra MBs, selected by channel-adapted optimization schemes, performs best

If feedback is available, interactive error control can be applied For short delays or low error rates: Modify the

prediction only in case of Nack message reception. In all other cases: Reference only those areas for

which the encoder is sure that the decoder has exactly the same are

26

Questions?