Top Banner
CENIC - QoS Workshop VIP - Magda El Zarki 1 Video Coding and Quality Issues Video Coding and Quality Issues Magda Magda El El Zarki Zarki ( ( elzarki elzarki @ @ uci uci . . edu edu ) ) VIP Lab. (www. VIP Lab. (www. ics ics. uci uci. edu edu /~ /~ vip vip) CAL-(IT) CAL-(IT) 2 2 & Dept. of ICS & Dept. of ICS UC, Irvine UC, Irvine CENIC - QoS Workshop 1/24/02
78

CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

Apr 06, 2020

Download

Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki1

Video Coding and Quality IssuesVideo Coding and Quality IssuesMagdaMagda El El Zarki Zarki ( (elzarkielzarki@@uciuci..eduedu))

VIP Lab. (www.VIP Lab. (www.icsics..uciuci..eduedu/~/~vipvip))CAL-(IT)CAL-(IT)2 2 & Dept. of ICS& Dept. of ICS

UC, IrvineUC, Irvine

CENIC - QoS Workshop 1/24/02

Page 2: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki2

Outline of Talk1. Part 1:

1. Overview of Video Compression2. The MPEG suite3. Video Quality4. Losses

2. Part 2:1. Delivery over IP Networks2. Feedback Control

3. Part 3:1. MPEG-42. Layered and Object based Coding

4. Part 4:1. QoS Issues2. Conclusions

Page 3: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki3

Part 1

1. Overview of Video Compression

2. The MPEG suite

3. Video Quality

4. Losses

Page 4: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki4

1. Video Compression: Goal

• Goal of video compression is tominimize the bit rate in the digitalrepresentation of the video signal while:– Maintaining required levels of signal quality

– Minimizing the complexity of the codec

– Containing the delay

Page 5: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki5

1. Video Compression: Tradeoffs

• The choice of a compression methodinvolves a tradeoff along the following 4dimensions:

Signal Quality(PSNR, MOS,etc.)

Efficiency(bpp, bpf,etc.)

Complexity(MIPs,memory,

Coding delay

Page 6: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki6

1. Video Compression: Why?• Video signals are amenable to compression

due to the following factors:– Spatial correlation: correlation among neighboring

pixels– Spectral correlation: color images– Temporal correlation: correlation among pixels in

different frames

• There is considerable irrelevant (from aperceptual viewpoint) information containedin video data.

Page 7: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki7

1. Video Compression: Lossless Coding

• Lossless coding is a reversible process -perfect recovery of data -> before and afterare identical in value. Used regardless ofmedia’s specific characteristics. Lowcompression ratios.– Example: Entropy Coding

• data taken as a simple digital sequence• decompression process regenerates data completely• e.g. run-length coding (RLC), Huffman coding, Arithmetic

coding

Page 8: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki8

1.Video Compression: Lossy Coding

• Lossy coding is an irreversible process -recovered data is degraded -> thereconstructed video is numerically notidentical to the original. Takes into accountthe semantics of the data. Quality isdependent on the compression method andthe compression ratio.– Example: Source Coding

• degree of compression depends on data content.• E.g. content prediction technique - DPCM, delta

modulation

Page 9: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki9

1. Video Compression: Hybrid Coding

• Used by most multimedia systems– combines entropy with source encoding

– E.g. JPEG, H.263, MPEG-1, MPEG-2,MPEG-4

Page 10: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki10

1. Video Compression: Design Choices

• Lossless or lossy or both• Compression ratio• Variability in compression ratio (fixed or

variable quality)• Resilience to transmission errors• Complexity tradeoffs in codec (memory,

processing, etc.)• Nature of degradations• Hierarchical representation

Page 11: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki11

1. Video Compression - Standards

• Broadcast (high bit rate):– MPEG-1– MPEG-2

• Video Conferencing (low bit rate):– H.261– H.263

• Interactive (full range of bit rates):– MPEG-4

Page 12: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki12

1. Video Compression: Deficiencies ofexisting standards

• Designed for specific usage– H.263 cannot be stored (no random access)– MPEG-1 & MPEG-2: not optimized for IP transport

• No universal file format for both local storageand network streaming

• Output cannot be reused efficiently aftercomposition - encoded once, no versatility

Page 13: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki13

1. Video Compression: Requirements forNew Standard

• Efficient coding scheme– Code once, use and reuse everywhere– optimized for both local access and network streaming

• Works well in both error prone and error freeenvironment– Scalable for different bandwidth usage– Video format can be changed on the fly– Transparent to underlying transport network

• Support efficient interactivity over network

Page 14: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki14

1. Video Compression: A solution - MPEG-4

• Internet in the future– Not only text and graphics, but also audio and video

• Fast and versatile interactivity– Zoom in, zoom out (remote monitoring)– Fast forward and fast backward (video on demand)– Change viewing point (online shopping, sports)– Trigger a series of events (distance learning)– On the fly composition– Virtual environments

• Support both low bandwidth connections(wireless/mobile) and high bit rates (fixed/wireline)

Page 15: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki15

1. Video Compression: What is MPEG-4?

“A coded, streamable representationof audio-visual objects and theirassociated time-variant data alongwith a description of how they arecombined.”

Page 16: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki16

2. MPEG: Overview• MPEG exploits not only spatial redundancy in each

frame, but also temporal (i.e frame-to-frame)redundancy present in all video sequences.

• Two Categories: intra-frame (spatial) and inter-frame(temporal) encoding– Intra: DCT based compression for the reduction of spatial

redundancy - I frame– Inter: Block-based motion compensation for exploiting

temporal redundancy• Causal (predictive coding) - current picture is modeled as

transformation of picture at some previous time - P frame• Non-causal (interpolative coding) - uses past and future picture

reference - B frame

Page 17: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki17

2. MPEG: Stream Components

I

P

P

B

B

B

B

Page 18: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki18

2. MPEG: The Quantization Parameter

• The quantization step is the main knob used tocontrol the output bit rate of MPEG based encoders.

• For CBR encoders MPEG quantization is adjusted asfollows:– If data rate increases over threshold, then quantization

enlarges the step size

– If data rate decreases, then quantization is performed withfiner granularity

Bitrate

% of frame

Increase Q

Decrease Q

threshold

Page 19: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki19

3. Quality

• What is video quality?– Generally judged using PSNR

• Easy to computeBUT• Not a good estimate of quality

– New objective quality measurements• Hard to computeBUT• More accurate

Page 20: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki20

3. Quality: Assessment Techniques

• Traditional Objective Assessment - PeakSignal to Noise Ratio (PSNR)

• Subjective Assessment – DSCQS (DoubleStimulus Continuous Quality Scale)

• Perceptual Objective Assessment -– Human visual perception based

– Capturing image imperfections

Page 21: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki21

3. Quality: Peak Signal to Noise Ratio

• For a video sequence of K frames ofNxM dimension with 8 bit depth:

RMSE

PSNR

255log20 10

⋅=

[ ]∑=

∑=

∑=

⋅⋅

=

Kk

Ni

Mj

kjixkjix

KMN

RMSE 1 1 1

2),,(̂),,(

1

Page 22: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki22

3. Quality: Advantages of PSNR

• Very easy to compute

• Well understood by most researchers

• Results are close to DSCQS but theydo not translate accurately to humanperception

Page 23: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki23

3. Quality: Disadvantages of PSNR

Some reconstructed images with different errors have the same PSNR values

Page 24: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki24

3. Quality: Subjective Assessment: DSCQS

• Source (A) and Processed (B) videoclips are presented in pairs

• The video presentation sequences arerandomized

Ai Bi Bj Aj

Page 25: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki25

3. Quality:DSCQS scoring

• Viewers grade each clip’s quality

• Data is gathered in pairs

A B

Excellent GoodFair

Poor

Unacceptable

Page 26: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki26

3. Quality: Issues with DSCQS

• Until now the most reliable qualitymeasurement method

• Requires special viewing room andequipment

• Needs a large group of people

• Large amount of post processing ondata

Page 27: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki27

3. Quality: Objective Assessment (OA)

• Establish a good quality assessment model

• The model takes as inputs the source and theprocessed video clips.

• Compare the model output to DSCQS test score

• If the result is consistent with DSCQS measurement,the model can substitute DSCQS

Page 28: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki28

3. Quality: OA Requirements

• Ability to predict subjective quality withlow error

• Predictions agree monotonically withthe relative magnitudes of subjectivequality ratings

• Prediction is robust with respect to avariety of video impairments

Page 29: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki29

3. Quality: OA Models - 2 approaches

1. Establish a model that simulates thehuman visual stimulation

2. Find the relationship betweenmeasurable parameters and perceptualdistortion (blurring, tiling, noise)

Page 30: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki30

3. Quality: Issues related to Method 1

• Advantages:– Considers both luminance and chrominance

– Some methods show very high correlation withDSCQS for some video sequences

• Disadvantages:– Not capable of in-service evaluation

– Not consistent over all video bit rate ranges

– Computationally complex

Page 31: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki31

3. Quality: Method 2 - ITS Model

• Institute for Telecommunication Studies(ITS) were the first group to propose anobjective measure several years ago.

• They have since refined (or fine tuned)the model to capture more of the imageimperfections.

• They map image imperfections ontomeasurable mathematical parameters.

Page 32: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki32

3. Quality: Perceptual Impairment Factor Vs. AMeasurable Parameter

Image blur

Step response rise time

Edge busyness

Step response jitter width

False contouring

Sp-p to minim

um

quantizing p-p

Granular noise

Equivalent analog signal SNR

express as

Sp-p/

Nrm

sDirty W

indow

Maxim

um noise am

plitudeM

ovement blur

Rise time of a m

oving edgeJerkiness

Field or frame difference in term

s

of moving edge position

Page 33: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki33

a) Original, b) Blur, c) Tiling, d) Noise

3. Quality: Perceptual Impairment Factor

a b

c d

Page 34: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki34

3. Quality: Advantages of ITS Model

• Works well for a wide range of bit rates

• Produces results that are consistentwith subjective tests (DSCQS)

• Computationally efficient

• Bandwidth efficient (384:1)

• In service quality monitoring

Page 35: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki35

3. Quality: Disadvantages of ITS Model

• Based on no visual model (vs.method 1)

• Only considers luminance value

Page 36: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki36

3. Quality: Video Quality Experts Group(VQEG)

• Several models have been underevaluation

• Tested video bit rate from 768 kbps to50Mbps ( 4:2:0 - 4:2:2 MPEG-2 )

• Both NTSC and PAL signals tested• Viewing Distance limit to 6:1• Carefully calibrated and aligned display

equipment

Page 37: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki37

3. Quality: DSCQS vs ITS

Page 38: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki38

3. Quality: Conclusions• All models have strengths and weaknesses, not one

can substitute DSCQS

• Some display fairly consistent behavior for differentvideo resources

• Developed to judge encoder quality not to assessdamage caused by packet losses

• No quality measures developed yet for shape coding

Page 39: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki39

4. Losses

• Packet losses may cause the quality ofthe video to degrade to unacceptableviewing levels

• It is not always easy to assess thedegree of degradation - highlydependent on what portion is lost

• Error concealment techniques canimprove quality substantially

Page 40: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki40

4. Losses: Error Concealment

(a). Unconcealed Image; (b). Frequency concealment (FC); (c). The 16thframe after initial FC on the first image; (d). Spatial Concealment

Left: Unconcealed Image. Middle: Concealed by simple motion vector estimation,simply averaging the top and bottom mvs. Right: Concealed with more motionvectors. All the adjacent mvs are used directly or indirectly.

Page 41: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki41

Part 2

1. Delivery over IP Networks

2. Feedback Control

Page 42: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki42

1. Delivery: Bit rate & Quality (VBR)

VBR_Bitrate

0

2000

4000

6000

8000

10000

12000

14000

16000

1 23 45 67 89 111 133 155 177 199

Time(200 msecs)

Q=4 Q=8 Q=12

VBR_Join

0

0.1

0.2

0.3

0.4

0.5

0.6

1 23 45 67 89 111 133 155 177 199

Time(200 msecs)

Join

Q=4 Q=8 Q=12

Page 43: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki43

1. Delivery: Bit Rate & Quality (CBR)

CBR_Bitrate

0

2000

4000

6000

8000

10000

12000

14000

16000

1

23

45

67

89

111

133

155

177

199

Time(200 msecs)

Bit

rate

(kb

ps)

Q4 Q8 Q12

CBR_Join

0

0.1

0.2

0.3

0.4

0.5

0.6

1 23 45 67 89 111 133 155 177 199

Time(200 msecs)

Jo

in

Q4 Q8 Q12

Page 44: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki44

1. Delivery: Bit Rate & Quality(Constrained VBR)

Constrained_VBR_Bitrate

0

2000

4000

6000

8000

10000

12000

14000

16000

1 23 45 67 89 111 133 155 177 199

Time(200 msecs)

Bit

rate

(kb

ps)

Q4 Q8 Q12

Constrained_VBR_Join

0

0.1

0.2

0.3

0.4

0.5

0.6

1 23 45 67 89 111 133 155 177 199

Time(200 msecs)Jo

in

Q4 Q8 Q12

Page 45: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki45

1. Delivery: Comparison TableJoin PSNR Bit RateAverage Std_dev Average Std_dev Average Std_dev

VBR_Q4 0.202004 0.002655 35.91496 0.44465 10053 1545VBR_Q8 0.317597 0.004489 32.96177 0.52643 3813 1127VBR_Q12 0.409873 0.004677 31.42474 0.67014 2302 889CBR_Q4 0.204929 0.003747 35.72485 0.98738 10054 483CBR_Q8 0.329049 0.024604 32.80154 1.15316 3815 408CBR_Q12 0.458771 0.058536 31.36981 1.56755 2307 374Constrained_VBR_Q4 0.206998 0.003203 35.78648 0.83835 10066 947Constrained_VBR_Q8 0.320873 0.010033 32.87081 0.87554 3829 736Constrained_VBR_Q12 0.412427 0.016802 31.38822 0.92129 2326 647

Table 1. Quality and Bit rate: average and standard deviation comparison

Page 46: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki46

2. Feedback Control

• Feedback control can be used to controlthe source encoder - change the outputbit rate by changing the quantizationparameter (Q) based on someperformance metrics– Use packet loss rates (RTCP error reports)– Use TCP congestion information– Use the perceived quality at the receiver

Page 47: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki47

1. Feedback Control - Implementation

video quality degradation calculator

Video inputVideo output

Enc oder Decoder

local d ecoder

link 3TCP/IP feedback channel

Network

RTP/UDP/IP video data channel

link 2video quality feature extractor

local quality fe ature extractor

Page 48: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki48

2. Feedback Control: Impact

Picture-1 Picture-2

Quality degradation caused by 1) packet loss and 2) compression algorithm

Page 49: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki49

2. Feedback Control: Issues

• Fine tuning of feedback control:– Error Concealment

– Evaluation window

– Degree of correction

– Thresholds for increasing or decreasing “Q”• QoS issues

• Pricing

– Impact on perception - variability in quality

– Etc.

Page 50: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki50

2. Feedback Control: Example

Page 51: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki51

Part 3

1. MPEG-4

2. Layered and Object based Coding

Page 52: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki52

1. MPEG-4: Overview• MPEG-4 aims to pave the way towards a

uniform, high quality encoding and decodingstandard, that would replace the manyproprietary streaming technologies in use onthe Internet today

• MPEG-4 is object-based, multi stream• Can accommodate a wide range of bit rates

including very low bit-rate communication formobile receivers or wristwatches that candisplay video.

Page 53: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki53

1. MPEG-4: What’s new?• Improved Coding Efficiency

– Hybrid data coding: mixing of synthetic and natural– Arbitrary shape coding (as opposed to rectangular)– Coding of multiple concurrent data streams

• Content-based Interactivity– Does not deem video frame as a whole anymore– Code each audio/video/text/graphics object into separate stream– User can interact with each object in the scene

• Universal Access– Robustness, independent of environment– Content-based scalability based on client’s request– Dynamically adaptive to available network bandwidth

Page 54: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki54

1. MPEG-4: Object Coding• Improves reusability and coding efficiency of

individual components– Apply different coding algorithms on different objects

• Allows content-based scalability– High resolution only on interesting part

– Streaming object, pre downloaded object and local objectcan work together

– Object based QoS support

– Allows more flexible user interactivity - each object can bepaused, FF, removed, etc.

Page 55: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki55

Integration of Natural and Synthetic ContentIntegration of Natural and Synthetic Content

Page 56: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki56

Augmented Reality

Page 57: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki57

Tele Presence

Page 58: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki58

1. MPEG-4: Scene Description

• A ‘compositor’ composes objects in a scene(A&V, 2&3D) creating a composite scene

• A scene description defines how objects appear onEnd User screen (composition view)

• With the scene description an End User can– change the position of individual video object– Zoom in/out interesting object– Choose different audio track (language, music)– Turn on/off individual object– Change resolution of an object, etc.

• Binary Format for Scene Description : ‘BIFS’

Page 59: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

1. MPEG-4: Scene Composition1. MPEG-4: Scene Composition

3D Furniture3D Furniture

2D Background2D Background

SpeechSpeech

VideoVideo ObjectObject

AV PresentationAV Presentation

Page 60: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki60

1. MPEG-4: Composition of Scenes1. MPEG-4: Composition of Scenes

Scene

Person Audio-visualPresentation

2D Background Furniture

Globe TableSpeech Video

Page 61: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki61

1. MPEG-4: Object Streams

...

Scene Description Stream

Object Descriptor Stream

Visual Stream

Visual Stream

Visual Stream

Audio Stream

Interactive Scene Description

Page 62: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki62

2. Object Coding: Object Descriptor(OD)• Groups a set of Elementary Streams

(ESs) associated with a particularobject as a single entity (e.g. base andenhancement layers)

• Transported in object descriptorstream

• Object descriptors can be updateddynamically over time

Page 63: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki63

Generic Sample of an Object Descriptor

Page 64: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki64

Examples of Object Descriptors

Page 65: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki65

ES-Descriptors• Each describes one Elementary Stream (ES)

(audio stream, video stream, etc.)• Includes configuration information for

dedicated stream decoder (DecoderConfig)• Contains sync layer configuration information

for this stream (SLConfig)• Conveys QoS Requirements to transport

channel (optional QoS descriptor)

Page 66: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki66

Coding Modes of MPEG-4

• Baseline(Conventional

rectangular coding)

– Compression

– Error Resilience

– Scalability

• Extended(Object (shape)

coding)

– Content-basedCoding

– Still Texture Coding

Page 67: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki67

Comparison of Modes

Page 68: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki68

Shape Coding

0

10000

20000

30000

40000

50000

60000

70000

80000

90000

vopheader

motion texture shape

arbitrary-shapedakiyorectangular-coded akiyo

Comparison of Arbitrary-shape coding

and Rectangular-based coding (Q = 6).

Total no. of frames: 40, format: qcif

Page 69: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki69

Combining Objects

Page 70: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki70

Scalable Coding

• Object based spatial scalability

• Object based temporal scalability

• Both provide resilience to transmissionerrors

Page 71: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki71

Spatial Scalability

Page 72: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki72

Temporal Scalability

Page 73: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki73

Tradeoff of Layered Coding

Layered coding schemes incur an increase in bit rate or decreasein video quality in comparison to a single-layer codec ofequivalent quality.

0 20 40 60 80 100

0

2

4

6

8

10

12

14

Base bit rate (% of total)

Tota

l bit

rate

(% in

crea

se)

Percentage increase of total bit rate versuspercentage base layer of total bit rate

Page 74: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki74

Scalable Coding & ESs

• Each layer is coded into an individualES with unique ES_ID

• All layers belonging to the same object(i.e,. all ES that refer to the sameobject) are placed in the same ObjectDescriptor with its unique OD_ID

Page 75: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki75

Example of Layered Spatial and ShapeCoding

Page 76: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki76

Part 4

1. QoS Issues2. Conclusions

Page 77: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki77

1. QoS Issues• Need to understand the application

• Need to understand its usage

• Need to understand its content

• Need to understand its versatility

• Need to understand quality tradeoffs

• Need to familiarize ourselves with resilience of thedata, recovery and control mechanisms

• Finally we can discuss QoS

Page 78: CENIC - QoS Workshop 1/24/02magda/Presentations/CENIC_QoS.pdf · 2002-01-25 · CENIC - QoS Workshop VIP - Magda El Zarki 7 1. Video Compression: Lossless Coding • Lossless coding

CENIC - QoSWorkshop

VIP - Magda El Zarki78

2. Conclusions

We have still a long way to go -

• Layered coding combined with shape codingshows promise

• Multi streaming not supported over IP

• Quality tools not there yet

• Pricing/Quality trade offs have to be defined

• Finally: Guarantees of Service are requiredas Best Effort does not work well!