Audio/Video compression An introduction

Audio/Video compressionAn introduction

Alain Bouffioux

December, 20, 2006

December, 20, 2006AV Compression / Alain Bouffioux

2

Agenda

Introduction - The evolution of Audio/Video consumer products and the role of compression techniques.

Audio & Video compression principles

Audio demonstration

Video demonstration


3



Audio demonstration

Video demonstration

Agenda


4

Moore’s law

Number of transistors per square inch doubles every 18 months


5

Moore’s law today

Cost of a transistor divided by one million in 30 years


6

Moore’s law today (2)

“Self-fulfulling prophecy” = roadmap for the semiconductor industry

Itanium 2 (2004, 592,000,000)


7

Moore’s law today (3)

Roadmap for semiconductor industry= only certainty in the current undefined future

Progress in semiconductors= fuels the innovation = fuels the software revolution= fuels the wireless revolution

(WLAN, WPAN, WBAN, …)

Examples:WBAN & sensors, RFID applications, camera to swallow, flexible display…

New products & related needs motivate semiconductor industry( Self-fulfilling prophecy)

Moore’s law will continue to apply: 10 years, 20 years ?– Economical limitation ? Investment (fixed) cost / globalisation– Power consumption (Moore’s low in reverse direction)– Architectural gap between IP-blocks & application (middleware still more

complex…)


8

The evolution of CE products (1)Past: every CE product was analogue

~1983: Music becomes digital (CD players)

Early 90ies: microprocessor enters CE devices(Early DVD players incorporated processing power equivalent to their comtemporary PC)

Late 90ies: Communication features incorporated within a CE devices.

ConsumerConsumerComputerComputer

CommunicationCommunication

CDCDDVDDVD

Receiver Digital TV orReceiver Digital TV orResidential gatewayResidential gateway


9

The evolution of CE products (2)

The Residential Gateway (Set-Top-Box, ADSL modem?) as the link between the home and the world-wide information infrastructure.

RG Home NetworkWorld-wide

communicationinfrastructure


10


The Residential gateway (in home) as the gateway to various services. Local Server provides 2 kind of services:

– BroadcastAnalogue & digital TV, NVOD, PPV

– Point-to-point (Home to local server)Home shopping, VOD, e-mail, Web browsing, PC connection...

Network

Local server

Internet

Local server

About 1000 homes


11


The Residential gateway as a key element of the home network

Home Network

Residential Gateway

To telephone Network

To satelliteNetwork

To cable Network

Computer

DVD Jukebox

Television

DiskRecorder


12


Everything will become digital: audio, telephone, video, photography, newspaper…The question is not if a selected product will really become digital; the question is when?

Consumer/Computer/Communication Convergence is progressive

New products combine all 3 domains(e.g. New GSM devices – Television on mobile)

Products always more and more complex

Products have always new features

Lifetime of products is always shorter


13

Compression is one among the various factors (all powered by semiconductor progresses) that enable multimedia technologies.

Factors enabling such evolution

Software (methodology,user interface ...)

International cooperation(interoperability &economy of scale)

Disc capacity (DVD), communicationgoing digital (xDSL, DVB, IP......)

Multimedia

Electronics(Memory capacity,clock frequency,µP architecture, ...--> decoding at low cost

Audio/VideoCompression(e.g. CCIR601 vs MPEG)


14

BUT !!

Convergence of technologies (consumer, communication, computer)

All products combine all three technologies

BUT !

Divergence of applications– Home consumer, Multimedia phone, Camera, PDA, Office

computer, Automotive…– High number of potential products

Technology push Market pull (user centric approach)


15



Audio demonstration

Video demonstration

Agenda


16

Compression in first A/V CE Products (1)

First Audio/Video products made compression without knowing it was compression.How ?By removal of irrelevancies (sampling rate, quantization)

Audio and Video characteristics

Audio VideoSpectral

SensitivityGood Bad

SpatialSensitivity

Bad Good


17

Compression in first A/V CE Products (2)

Audio productsFrom 2 to 7.1 channels are enough to provide the spatial resolution.

Video productsThree colours (RGB) are enough to provide the spectral resolution.


18

Audio: Compression needed in spectral domain

Bitrate of a stereo audio source (CD-DA encoding)

Sampling frequency : 44.1 kHzStereo16-bit per sampleBitrate = 44100 * 2 * 16 = 1.41 Mbit/sec

Audio waveform (time)

time

The need for more compression (1/5)


19

Video: Compression needed in spatial domain

Bitrate of a video source (CCIR 601 - 50 Hz countries)

25 images per secondYUV coding (Y: luminance - U,V : Chrominance)Y: 8 bit per pixel - U,V: 1 pixel on 2 coded, 8 bit per pixelBitrate = (576*720)*25*16 = 166 Mbit/sec


720 samples

576lines

Video image


20


Channels availables for AV transmission– Analog television channel (compatibility)

Cable (bandwidth = 8 MHz) Satellite (Bandwidth = 30-40 MHz)

Capacity around 40 Mbit/sec

– Compact disc (CD)For 74 min. play time : 1.41 Mbit/sec


21


MPEG-1 target (Moving Picture Expert Group)(Video-CD : 74 min. constraints)

But quality was judged too poor (about VHS quality)

Compression

Video : 166 Mbit/sec

Audio : 1.4 Mbit/sec

1.4 Mbit/sec


22


MPEG-2 target– Program stream (DVD)

– Transport stream (DVB)

Compression

1 program(video, multichannelaudio, ....)

= motivation for the capacityincrease of the CD (--> DVD)

3-9 Mbit/sec (variable bitrate)(but higher quality than MPEG-1)

Compression

n programs(video, multichannelaudio, ....)

about 40 Mbit/sec (constant bitrate)(DVB-Satellite & DVB-Cable)


23

Principles of compression (1/2)

Compression (or source coding) is achieved by suppressing information :

– redundant information – irrelevant information

Suppression of redundant information lossless compression example: PCM to DPCM,DCT

The original signal and the one obtained after encoding and decoding are identical

DecompressionCompressionFc(x,y,t)

Rc kbps Ri < Rc

Fp(x,y,t) = Fc(x,y,t)

Rp = Rc


24

Principles of compression (2/2)

Suppression of irrelevant information lossy compression Example: bandwidth limitation, masking in audio

The original signal and the one obtained after encoding and decoding are different but are perceived as identical

DecompressionCompressionFc(x,y,t)

Rc kbps Ri < Rc Rp = Rc

Fp(x,y,t) <> Fc(x,y,t)


25



Audio demonstration

Video demonstration

Agenda


26

Audio Demonstration

From “Borderline” Madonna - Stereo - 16 bit/channel

Compression used AAC

Compression

Decompression

Original

-

32kbps

128kbps

64kbps

16kbps

705 kbps


27

MOS scale (1/2)

Signal distortion is not a good measure of the performance of a lossy compression method an other method is necessary: MOS scale (Mean Opinion Score)

The five-grade CCIR impairment scale (Rec.562)1(Very annoying), 2(Annoying), 3(Slightly annoying), 4(Perceptible but not annoying), 5(Imperceptible)

Example:Double blind test


28

MOS scale (2/2)

Compressed

3

1

4

Impairmentscale5

Min value

Sequence2 3

Original

Mean value

Max value

Original signal

Listener answers to :1. Which signal is the original ? 2 or 3 ?2. Grade the other one ?

Exchange box(Random for listener)

Selector(controlled by listener)

Compressed signal


29



Audio demonstration

Video demonstration

Agenda


30

Compression to VBR or to CBR

CBR (Constant Bit Rate) vs VBR (Variable Bit Rate)

Scene more complex Higher bit rate for same quality

CBR variable quality (example : Video CD artefact)

Constant quality VBR necessary (e.g.: DVD-Video)

Constantbit rate

Bitrate

Complex

Distorsion

Simple

Constant quality


31

Video demonstration

MPEG-1 Example Video CD standard(288*352)video – CBR 1.4 Mbps

MPEG-2 ExampleDVD standard(576*720)video – VBR ~3Mbps


32

The compression trade-offCompression techniques are still making progress

Trade-off Complexity/Quality/Bit Rate

New technique may result in new trade-off

Quality

Bitrate

Complexity

MPEG Layer 1

MPEG Layer 2

MPEG Layer 3

MPEG AAC

Other TechniqueSpeech coding


33

Audio/Video compression An introduction

Documents

evolution of ce products

role of compression

progressivenew products

ce devices early dvd

residential gateway

local server home shopping

semiconductor industryitanium

broadcast analogue digital