Avlm 2009 Compression Erik Luyten

compression

from production format to distribution format

What's this ?

2AVLM Training26/05/09

René Margritte: La Trahison des Images (1928-29)


Moving images !?


Temporal resolution

2 images/second 5 images/second

12,5 images/second 25 images/second


Spatial resolution

7 x 10 = 7028 x 40 = 1120

14 x 20 = 280

112 x 160 = 1792056 x 80 = 4480

448 x 640 = 309120224 x 320 = 71680


Color resolution

24 bit color pallet244 KB file

8 bit color pallet28 KB file

4 bit color pallet16 KB file4 bit color palletNo “dithering”

8 bit optimized pallet28 KB file


Video signal(uncompressed)768 (720) pixels

576

pixe

ls

25 images/secondEach pixel has 3 values: (3 bytes)• Red: 256 steps (1 byte)• Green: 256 steps (1 byte)• Blue: 256 steps (1 byte)

Datarate for the component video signal:720(pix) x 576(lines) x 2(byte) x 25(images)=20 MByte/sec(160 Mbit/sec)

Datarate for the RGB video signal:768(pix) x 576(lines) x 3(byte) x 25(images)=32 MByte/sec(256 Mbit/sec)


Audio signalSampling Rate: 44,1 kHz (CD)Bit Depth: 16 bit (65.536 steps)Stereo: x 216 bit x 2 x 44.100 = 1.411.200 bps 1,4 Mbit/sec

Production systems normally use a sampling rate of 48 kHz16 bit x 2 x 48.000 = 1.536.000 bps


Video distribution ?

Disk

floppy disk = 0,05 seconds

CD-rom = 20 seconds

1 Gbyte hard disk = 32 seconds

16 Gbyte array = 8 min 30 sec

Network

Ethernet 100 Mbit/sec (shared) LAN1/2.5

Ethernet 10 Mbit/sec (shared !) LAN1/25

ISDN lijn ( 2B + D) : 128Kbit/sec 1/2000

Modem connection : 56 Kbit/sec 1/4600

1 second video: 32 Mbyte (256 Mbit/sec) 1 minute video: 1,9 GByte 1 hour video: 111 GByte


Compression techniques

Intraframe (spatial compression)Compression within a single frame

Interframe (temporal compression)Compression between successive frames

LosslessNo information loss

Lossy(Important) information is lost (f.i. medical applications)

SymmetricCoding and decoding equally processor intensive

AsymmetricCoding is more processor intensive than decoding


Intraframe compression (in one frame)

Run Length Encoding• without compression:

one line of 100 (white) pixels 100 x 3 bytes = 300 bytes

• RLE compression 100 identical white pixels in a row 2 bytes: total amount of pixels involved 3 bytes for the colour information 2 + 3 = 5 bytes of info for 100 pixels

• Compression factor of 60:1 without information loss

100 pixels


Other Intraframe techniquesDiscrete Cosine Transformation

• JPEG, MPEG, DV,…• Type of Fourier Transformation • Blocks of 8 x 8 pixels and/or 16 x 16

Wavelet Compression• MPEG-4 (still image), JPEG2000• Improving image resolution as more data are received

Vector QuantizationFractal Compression


Interframe compression

Image 124

Image 125


Interframe: MPEG (Moving Picture Expert Group)

I B B P B B P B B P B B I

GOP Group Of PicturesI-pictures: (Intra pictures) reference image with complete information

for one frame (scene changes, ...)

P-pictures: (Predicted pictures) images are “predicted” using the information in earlier I or P pictures.

B-pictures: (Bi-directional predicted) pictures are recomposed from information in earlier or later I or P pictures. B-pictures are never used as reference for other images.

Group Of Pictures (GOP)


Movie Formats

File Format:Specifies how data are arranged in a file

(interleaving video and audio)Codec:

Specifies how (image) data is compressedArchitecture:

Synchronisation of different media components (video, audio, language tracks, text,…. tracks)

Control for interactivity, alternate tracks,…


Movie Architectures

• QuickTime• AVI• Windows Media Player• Real Media• Flash ?• MPEG-4

Media platformplatform distribution cost alternates remarks

QuickTimeMacWindows(Linux)

CD romDownload (HTTP)Streaming (RTSP)

Player = freePro = €30Server= open srce

RTSP alternatesHTTP alternatesMixed alternatesSkip prot.,instant -on

QTVR, text, midi, SMIL,..production and distribution

H.264

RealWindowsMacLinux

Streaming (RTSP)Download (HTTP)

Player = freeSuperpass=$13/mServer=payable

RTSP alternates(Surestream techn.)Dynamic

only distribution and mainly for streaming

WindowsMedia

Windows(Mac)(Linux)

Streaming ( )Download (HTTP)CD rom

Player=freeServer=Windows2003

RTSP alternatesDynamic

Production anddistributionVersion 9 on windows platform

Flash(Spark,

On2, h264)

WindowsMacLinux

CD romDownload (HTTP)Streaming

Player=freeServer=€4759

? H.264

MPEG-4WindowsMacLinux

Streaming (RTSP)Download (HTTP)CD romDVD, GSM, ….

Player=QT, Real,…Server=open srceLicense fee

scalable (in the future)

Industrie standaardfrom handheld to HD

H.264

17AVLM training24/05/09


File Formats

• DV• MPEG-1• MPEG-2• MPEG-4


Codec’s• Picture:

• Cinepak• Indeo (Intel Indeo)• H.261, H.263, H.264• Sorenson• M-JPEG• MPEG-1, MPEG-2, MPEG-4• DV

• Sound:• IMA • Qualcomm PureVoice, Qdesign Music,… • MP3• AAC (Advanced Audio Coding)• WMA, RealAudio


H.264

Codec’s and bitrate

MPEG-2MPEG-2

MPEG-4

56 kbit 256 Mbit1 Mbit 9 Mbit 25 Mbit200 kbit

MPEG-1

H-261H-263

SorensonDV

uncompr.

Distribution codec’s Production codec’s

profile 0 profile 1

MPEG-4

HD distribution Studio productionPIM, GSM, ...

Web ?


MPEG-4 differenceMPEG-1, MPEG-2video

subtitles

background

graphics

Ren

derin

g an

dC

ompo

sitio

nC

ompr

essi

ontransport

Dec

ompr

essi

on

displayand

scaling

video

subtitles

background

graphics

transport

transport

transport

transport

com

pres

sion

Sca

ling,

Ren

derin

g an

dC

ompo

sitio

n

Dec

ompr

essi

on

display

Bitmap stream

ASCI code

Bitmap image

Vector code

MPEG-4

displayand

scaling

display


Video to Computer• Video recording

• Interlaced, 25 frames - 50 fields per second

• Restricted to 576 effective lines (PAL) and 3x4 picture aspect ratio.

• Computer image• Non-interlaced, 60 to 80 frames per

second• No restrictions on picture format and

aspect ratio• De-interlacing of video material is

desirable.


De-interlacing

1

2

3

311

312

313

313

314

315

623

624

625

Field 1Field 2


Delivery methods• DVD, CD, USB stick,…• Internet

• Progressive download (HTTP, FTP)• “file transport” type (TCP protocol)• No data loss: resent lost packages• No real-time events• Complete file must be downloaded to allow random

access. • No special server needed (normal web service over

http)• Streaming media (RTP)

• RTSP transport (UDP protocol)• Lost data (probably) will not be recovered• Real-time events possible• “Random access” possibility (only needed fragments will be

downloaded)• Needs one or more(worldwide proxy delivery)

streaming servers


IP distributionHTTPProgressive Download

RTSP StreamingUnicast

RTSP StreamingMulticast

Protocol/Port TCP (http)Port 80

UDP (RTP)Port 554, 7070,... (80)

UDP (RTP)Port 554, 7070,...

Firewall Same as Web accessno problems

UDP(RTP) accessProxy server, http tunnel

Only in multicast enabled segments

Live broadcast No Ja (through relay server) Yes

Recorded mat. Yes Yes Yes

Video on Demand Yes(after complete download) Yes No

Alternates Yes Yes Yes

Copy protected Can be circumvented Can not be saved (easily) Can not be saved (easily)

Packet loss (quality) No (resent) Yes (discard) Yes (discard)

Buffering 100 %complete file

5-8 secondslatency

5-8 secondslatency

Servers WWW serverApache, NT,... Streaming Server RTP Streaming Server RTP


QuickTime ?• QuickTime is a container that allows other

media to work in synchronization with video and audio.

• QuickTime is more than video and provides functionality for:• QuickTime VR• Interactivity (Flash, JavaScript, Wired Sprites, Pop-up

menu, chapter track, http track, ....)• Components (Zoomify, Pulse, Kaydara, ...)

• FastStart download and/or RTSP streaming• Alternates for:

• Network connection(datarates)• Language versions (audio track and/or subtitle track)• Processor speed, QT version, choke speed


Compression settings ?The download data rate for progressive download can be higher than the

network connection limit. The client application simply waits until enough data have been received before activating the play button. (no information is lost)

With streaming the stream data rate has to be lower than the network connection limit. There will be no time to recover lost or late packages and the client application will only receive partial information resulting in frozen images and broken sound.

download streamingsmall mediu

mlarge 112 256 384

Image size 240x180

320x240

480x360

240x180

240x180

320x240

Images/sec 12,5 25 25 12,5 12,5 25Codec h.264

VBRh.264VBR

h.264VBR

h.264CBR

h.264CBR

h.264CBR

Audiocodecsample rate

MPEG-416000

MPEG-432000

MPEG-448000

MPEG-422kMon

oMPEG-416000

MPEG-416000

Target bitrateV+A kbits/sec

168+32200

436+64500

896+128

102464+16

80176+32

208276+32

308


External web server

Toledo server

Reference movie with alternates

QT Streaming server

client


Toolswindows mac OSX

Digitalisation

Adobe PremiereSorenson Squeeze (499$)Movie Maker, AVID,...

Final Cut ProSorenson Squeeze (499$)iMovie, AVID,....

Editing Adobe PremiereMovie Maker, AvidQT Pro (€40)

Final Cut ProiMovie, AVIDQT Pro(€40)

Compression

Episode ($495) Sorenson SqueezeCleaner XL 1.5 ($595)Premiere, QT Pro,Super, ffmpeg,....

Episode ($495) Sorenson SqueezeCleaner 6.5 ($595)Compressor (FCP), QT Pro,VisualHub, ffmpegX, ...

Interaction LiveStage, Flash CS4Cleaner XLQT Pro

LiveStage, CS4Cleaner 6QT Pro


Books• Compression for Great Digital Video (©

2002)Ben Waggoner, CMP Books, ISBN 1-57820-111-X

• Video Compression Demystified (© 2001)Peter Symes, McGraw-Hill, ISBN 0-07-136324-6

• Streaming Media (© 2003)Tobias Künkel,Wiley Computer Publishing,ISBN 0-470-84724-7

• QuickTime for the Web 3th ed. (© 2004)Apple,Morgan Kaufmann Publishers,ISBN 1-55860-904-0

Avlm 2009 Compression Erik Luyten

Technology

qt pro

download

streaming

video

lost

avid

udp

data