Top Banner
MPEG-21 Scalable Video MPEG-21 Scalable Video Coding Coding and the Stool (UniBS) system and the Stool (UniBS) system Riccardo Leonardi, Alberto Signoroni, Michele Brescianini, Nicola Adami Signals and Communications Lab. – DEA University of Brescia, Italy riccardo . leonardi @ ing . unibs .it alberto . signoroni @ ing . unibs .it Thessaloniki – October 7-8, 2004
18

MPEG-21 Scalable Video Coding and the Stool (UniBS) system Riccardo Leonardi, Alberto Signoroni, Michele Brescianini, Nicola Adami Signals and Communications.

Dec 18, 2015

Download

Documents

Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: MPEG-21 Scalable Video Coding and the Stool (UniBS) system Riccardo Leonardi, Alberto Signoroni, Michele Brescianini, Nicola Adami Signals and Communications.

MPEG-21 Scalable Video MPEG-21 Scalable Video Coding Coding

and the Stool (UniBS) systemand the Stool (UniBS) system

Riccardo Leonardi, Alberto Signoroni,Michele Brescianini, Nicola Adami

Signals and Communications Lab. – DEAUniversity of Brescia, Italy

[email protected]

[email protected]

Thessaloniki – October 7-8, 2004

Page 2: MPEG-21 Scalable Video Coding and the Stool (UniBS) system Riccardo Leonardi, Alberto Signoroni, Michele Brescianini, Nicola Adami Signals and Communications.

R. Leonardi - A. Signoroni University of Brescia, Italy

Scenarios (why SVC is desirable)MPEG Requirements (the “shall” and “should” of SVC)Technologies which enable SVCSVC architectures (t+2D, 2D+t, UniBS)Experimental results

OutlineOutline

Page 3: MPEG-21 Scalable Video Coding and the Stool (UniBS) system Riccardo Leonardi, Alberto Signoroni, Michele Brescianini, Nicola Adami Signals and Communications.

R. Leonardi - A. Signoroni University of Brescia, Italy

Classical video coding Code-decode at a single operating point (target frame

resolution, spatial resolution and quality) Code-transcode at operating points different from the

target (complexity, performance loss, cost) Scalability in current standards (few spatio-temporal

operating points MPEG-2, coarse or fine grain quality scalability MPEG-4, performance loss)

Scalable video coding Code once decode at S x T x Q operating

points (full scalability in space, frame-rate and quality)

Video coding paradigmsVideo coding paradigms

Page 4: MPEG-21 Scalable Video Coding and the Stool (UniBS) system Riccardo Leonardi, Alberto Signoroni, Michele Brescianini, Nicola Adami Signals and Communications.

R. Leonardi - A. Signoroni University of Brescia, Italy

Multi-channel content production and delivery: the same stream will be viewed on a variety of devices

with different spatial and/or temporal resolution will be viewed and charged on the basis of delivered

quality level devices can decide to trade-off received quality for a

longer battery life (complexity scalability)

Video streaming over heterogeneous IP networksAdvanced surveillance systemsVideo conferencingMobile streaming videoOthers...

SVC scenarios and SVC scenarios and applicationsapplications

Page 5: MPEG-21 Scalable Video Coding and the Stool (UniBS) system Riccardo Leonardi, Alberto Signoroni, Michele Brescianini, Nicola Adami Signals and Communications.

R. Leonardi - A. Signoroni University of Brescia, Italy

at Pattaya Meeting (Dec.2001) MPEG started an AHG to explore Interframe Wavelet Coding – activity successfully completed in Oct.2002RPI software (prof. J.W.Woods) released based on MCTF+EZBC

MPEG started an AHG on Scalable Video Coding (MPEG-21)Call for Evidence on SVC: 9 responses received by July 2003Call for Proposals on SVC: 14 full proposals (Wavelet based, AVC based)

Wavelet based: best selected Microsoft Research Asia (software released)

AVC based: best selected HHI-FraunhoferCore Experiments started at the Monaco (Mar.2004) and Redmond (Jul.2004) meetings.No reference architecture yet specified (MRSA, HHI, Samsung, UNIBS, DANAE-TOMPHSON, UNSW proposals at the next meeting Palma de Mallorca Oct.04)

MPEG-21 Ad Hoc Group on MPEG-21 Ad Hoc Group on SVCSVC

Page 6: MPEG-21 Scalable Video Coding and the Stool (UniBS) system Riccardo Leonardi, Alberto Signoroni, Michele Brescianini, Nicola Adami Signals and Communications.

R. Leonardi - A. Signoroni University of Brescia, Italy

RequirementsRequirements

shall should may

Spatial scalability

2 levels(e.g.

CIF+QCIF)4 levels

From QQCIF to

3610x1536

Temporal scalability

2 levels4 levels

(up to 60 Hz)

Non-dyadic

Quality (SNR)

scalability

Coarse grain (25% bit-

rate)

Medium grain

(10% bit-rate)

FGS or progressive

Last revision: MPEG output doc N6505, Redmond, July 2004

Page 7: MPEG-21 Scalable Video Coding and the Stool (UniBS) system Riccardo Leonardi, Alberto Signoroni, Michele Brescianini, Nicola Adami Signals and Communications.

R. Leonardi - A. Signoroni University of Brescia, Italy

Combined Scalability (at least for a finite number of points in the spatial-temporal-quality space)Challenging Reqs

Coding Efficiency performance (superior to H.264/AVC)

Base-layer compatibility (to H.264/AVC)

Complexity scalability (for power limited devices)

Others (less peculiar to SVC) ROI and object scalability, robustness to transmission

errors and on “best-efforts” networks, end-to-end delay, random access capability)

Requirements Requirements contd.

Page 8: MPEG-21 Scalable Video Coding and the Stool (UniBS) system Riccardo Leonardi, Alberto Signoroni, Michele Brescianini, Nicola Adami Signals and Communications.

R. Leonardi - A. Signoroni University of Brescia, Italy

Multiple Adaptation Scalability: “decoding paths”

Req. + Core Experiment Req. + Core Experiment (oct’04)(oct’04)

spatial resolution

temporal resolution

4CIF

CIF

QCIF

7.5153060

bit-rates

high

low

global bitstream

Scenario 1 – merged proposal for sub-extractionsCity, Crew, Harbour, Soccer

Page 9: MPEG-21 Scalable Video Coding and the Stool (UniBS) system Riccardo Leonardi, Alberto Signoroni, Michele Brescianini, Nicola Adami Signals and Communications.

R. Leonardi - A. Signoroni University of Brescia, Italy

A generic SVC encoderA generic SVC encoder

Video

Entropy Coding

Motion Coding

Texture Coding

Spatio-Temporal"Transform"

2D SpatialTransform

TemporalTransform

Bitstream

In order to have an efficient scalable codec, it is necessary that In order to have an efficient scalable codec, it is necessary that each one of the building blocks is operating in a scalable mode.each one of the building blocks is operating in a scalable mode.

MPEG-21 doc N6520 Scalable Video Model 2.0 July 2004, Redmond, WA, USA

Page 10: MPEG-21 Scalable Video Coding and the Stool (UniBS) system Riccardo Leonardi, Alberto Signoroni, Michele Brescianini, Nicola Adami Signals and Communications.

R. Leonardi - A. Signoroni University of Brescia, Italy

In the MPEG21-SVC framework a series of SVC enabling technologies have been identified

Spatial transform: DWT (Discrete Wavelet Transform) 9x7; IntegerDWT 5x3; Decimation/Interpolation; Block based transforms

Temporal transform: Haar MCTF (Motion Compensated Temporal Filtering); 5x3 MCTF; UnconstrainedMCTF; …

Motion coding: non-scalable; multires. quality scalable explicit (inside the method) / implicit (on the data structure); hierarchical B-frames; …

Texture coding: EBCOT, 3D-EBCOT; EZBC; layered H.264/AVC; SNR scalable CABAC based entropic coding; …others (e.g. embedded wavelet coders).

SVC Enabling TechnologiesSVC Enabling Technologies

Page 11: MPEG-21 Scalable Video Coding and the Stool (UniBS) system Riccardo Leonardi, Alberto Signoroni, Michele Brescianini, Nicola Adami Signals and Communications.

R. Leonardi - A. Signoroni University of Brescia, Italy

SVC implementationsSVC implementations

• Following the Call for Proposal for SVC (MPEG doc.N5958 Oct.2003) a certain number of implementations have been presented (MPEG doc.M10569 Mar.2004).

• On the base of the CfP inputs a SVC classification has been made which allows to distinguish some families of SVC implementations:

“t+2D” schemes “2D+t” schemes “2D+t+2D” schemes or “Multi-scale Pyramid t+2D”

Our STool scheme could be classified in the third category

Page 12: MPEG-21 Scalable Video Coding and the Stool (UniBS) system Riccardo Leonardi, Alberto Signoroni, Michele Brescianini, Nicola Adami Signals and Communications.

R. Leonardi - A. Signoroni University of Brescia, Italy

t+2D schemest+2D schemes

Video Bitstream

MultiplexMCTF

Motion Coding(Scalable)

Motion

TextureSNR Scalable

Entropy CodingDWT (N level) HP/LP

Enablestemporal scalability

Enablesspatial scalability

Enablesquality scalability

Page 13: MPEG-21 Scalable Video Coding and the Stool (UniBS) system Riccardo Leonardi, Alberto Signoroni, Michele Brescianini, Nicola Adami Signals and Communications.

R. Leonardi - A. Signoroni University of Brescia, Italy

2D+t schemes2D+t schemes

Video Bitstream

DWT (1 level) MultiplexMCTF

Motion CodingMotion

TextureSNR Scalable

Entropy CodingHP

DWT (1 level)

LP

LP

MCTF

Motion CodingMotion

TextureSNR Scalable

Entropy CodingHP

MCTF

Motion CodingMotion

Texture SNR ScalableEntropy Coding

DWT(N-3 level)

HP/LP

Page 14: MPEG-21 Scalable Video Coding and the Stool (UniBS) system Riccardo Leonardi, Alberto Signoroni, Michele Brescianini, Nicola Adami Signals and Communications.

R. Leonardi - A. Signoroni University of Brescia, Italy

UniBS SVC scheme: UniBS SVC scheme: “STool”“STool”

Orig4CIF

DWT

CIFMEC+MCTF

MEC+MCTF

Prediction +DWT

MEC+MCTF

EC

EC

EC

decoded

decoded

QCIF MVQCIF

MVCIF

MV4CIF

bit-stream

Prediction +DWT

2 SYSTEMS which differs in the EC part

1. 3D-ESCOT (MSRA software)

2. GOF-EMDC (UNI-BS)

MEC (Motion Estimation and Coding)

+ MCTF (MSRA software)

Page 15: MPEG-21 Scalable Video Coding and the Stool (UniBS) system Riccardo Leonardi, Alberto Signoroni, Michele Brescianini, Nicola Adami Signals and Communications.

R. Leonardi - A. Signoroni University of Brescia, Italy

Results: Results: CITY QCIF 15 Hz 64 kbps

Original AVC STool

zoom zoom

Page 16: MPEG-21 Scalable Video Coding and the Stool (UniBS) system Riccardo Leonardi, Alberto Signoroni, Michele Brescianini, Nicola Adami Signals and Communications.

R. Leonardi - A. Signoroni University of Brescia, Italy

Results:Results:

Original

AVC Stool

CITY CIF 30 Hz 256 kbps

Page 17: MPEG-21 Scalable Video Coding and the Stool (UniBS) system Riccardo Leonardi, Alberto Signoroni, Michele Brescianini, Nicola Adami Signals and Communications.

R. Leonardi - A. Signoroni University of Brescia, Italy

Results: AVC vs MSRA vs Results: AVC vs MSRA vs StoolStool

Play

HARBOUR

CITY

QCIF 15Hz 96kbps

CIF 30Hz 384kbps

QCIF 15Hz 64kbps

CIF 30Hz 256kbps

Page 18: MPEG-21 Scalable Video Coding and the Stool (UniBS) system Riccardo Leonardi, Alberto Signoroni, Michele Brescianini, Nicola Adami Signals and Communications.

R. Leonardi - A. Signoroni University of Brescia, Italy

ConclusionsConclusions

SVC systemsMultipoint spatial-temporal-SNR scalabilityComparable or superior in coding performance with respect to AVCA lot of work is currently made on SVC (AVC is mature, SVC is young… :-)

SToolSTool: a new 2D+t+2D system with good performance (UNIBS submission to the next MPEG meeting)