Scalable Video Conferencing Using Subband Transform Coding and Layered Multicast Transmission

Scalable Video ConferencingUsing Subband Transform Coding

and Layered Multicast Transmission

Mathias JohansonSwedish Research Institute for Information Technology

[email protected]

Scalability in Videoconferencing

• Large number of video receivers (and senders)

• Multiple quality levels in a single multipoint conference session

• Differentiated host and network requirements

• Realizable over public internetworks

• CODEC operates at fixed bandwidth

• Multipoint operation involves gateways

• Differentiated quality levels in a multipoint session require transcoders that are expensive and introduce latency

• Often dependent on level 2 network protocols (e.g. ISDN systems)

Limitations of Traditional Videoconferencing Systems

Approach...

• Scalable codec based on subband transform coding

• Receiver-driven layered IP-multicast transmission

• Software implementation + DSP-based implementation

Layered Video Coding

• Temporal layering– Increased number of refinement layers correspond to

higher framerate

• Spatial layering– Increased number of refinement layers correspond to

higher image resolution

• Layered quantization– Increased number of refinement layers correspond to

finer quantization

Temporal Layering

Channel 1

Channel 2

Channel 3

Channel 4

Transmission channels that can be received independently

Images of a video sequence

Spatial Layering

Channel 1

Transform

Channel 3

Channel 2

Original imageBase signal + refinement signals

Layered image and video encoding/compression formats

• Hierarchical JPEG

• MPEG-2 scalable mode– temporal, spatial, SNR scalability

• H.263 scalable mode

• Wavelets

Block-based DCT

Subband transform

Base layer

Refinementlayer

Down-sample

x(t)Encode

Encode

Decode

Upsample

Spatial scalability in block based image and video encodings

Wavelet-based approach to spatial scalability

Glow

x(t)

(t)y0

(t)y1

2

2Ghigh

base layer

refinement layer

Quadrature mirror filters implementing the wavelet transform

Encode

Encode

Wavelet transform

Iterate….

horizontal transform vertical transform

Original image

Transformed image

Wavelet compression

• Colorspace conversion and subsampling– RGB -> YCrCb 4:2:2

• Wavelet transform (separately on Y, Cr, Cb)– Subband decomposition

• Quantization of each subband/component– Lossy compression step

• Huffman encoding– entropy coding

Communication Architecture

• Transmit the subbands of the transformed images on separate channels that can be received independently

• Multicasting

• Leaf-initiated JOIN-mechanism

RLMReceiver-driven Layered (IP) Multicast

224.3.4.5

224.3.4.6

224.3.4.7

224.3.4.8

Refinement layers

Base layer

RInternet

Sender Receiver (4 layers)

Receiver (1 layer)

High bandwidth

Low bandwidthMulticast router

Wavelet RTP header

FOLW

HQ1

Q2Q3

Fragmentation OffsetLayer NumberWidth

HeightY Quantization Factor

Cr Quantization FactorCb Quantization Factor

H1H2H3

L1L2

L3

Y Huffman Table SizeCr Huffman Table SizeCb Huffman Table Size

Y Data LengthCr Data Length

Cb Data Length

FO L W H Q1 Q2 Q3 H1 H2 H3 L1 L2 L3

0 32 64 96 128 160 192

Prototype implementation

• Based on Smile!

• Software wavelet codec

• Receiver-driven layered IP multicast network module

• RTP/RTCP

• Spatial and temporal scalability

• SGI O2, MIPS R5000 processor

Usage Scenario highly heterogeneous environment

RHigh-speed LAN

Internet

Dial-up access

Medium qualityLow quality

High quality

Leased Line

Leased Line

Transmitter

Performance Tests

Image quality scalability Bandwidth scalability

Future work...

• Temporal compression

• DSP implementation (TMS320C80 or similar)

• Automatic selective refinement based on ”bandwidth discovery”

• Subband audio coding