IEE 5037 Multimedia CommunicationsLecture 1: Introduction
Dr. Tian-Sheuan [email protected]. Electronics EngineeringNational Chiao-Tung University
Dept. Electronics Engineering,N
ational Chiao T
ung University Special Thanks to Prof. Hang to provide most of the slides
in this lecture, except highlighted by other sources.
2
Chapter 1 IntroductionWhy need compression
How compression possible
Principles of Compression Techniques
Multimedia Standards
Subjects to be discussed
3
Why Compression?
-- Massive dataSpeech: 8 bits (per sample) x 8K (samples/sec) = 64Kbits/sCD audio:
16 bits x 44.1K (samples/sec) x 2 (channels) = 1.411Mbits/sec(44.1K = 60 (fields) x 245 (lines) x 3 (samples) (J. Watkinson,The Art of Digital Audio, p.28, Focal Press, 1989) )
Digital TV: (4:2:2, NTSC in CCIR 601)Still picture: 720 (pels) x 483 (lines) x 2.0 bytes = 5.564 MbitsMotion picture: 5.564 Mbits x 29.97 (frames/sec) = 167Mbits/sec
Digital HDTV: (ATSC)1920 (pels) x 1080 (lines) x 1.5 bytes x 30 (frames/sec)=746Mbits
4
Picture: “Hotel“, 720x576, 414,720 Byte
Source: T. Wiegand “Digital image communication”
5
JPEG-2000 Compressed to 12,960 Byte
Source: T. Wiegand “Digital image communication”
6
JPEG-2000 Compressed to 3,240 Byte
Source: T. Wiegand “Digital image communication”
7
The Power of Compression
Source: T. Wiegand “Digital image communication”
8
Digital Image Formats
Source: T. Wiegand “Digital image communication”
9
Digital Image Formats
Source: T. Wiegand “Digital image communication”
10
Digital Image Formats and Applications
Source: T. Wiegand “Digital image communication”
11
Examples of Storage Media
Source: T. Wiegand “Digital image communication”
12
Examples of Transmission Media
Source: T. Wiegand “Digital image communication”
13
An Example of Comm. Systems< A Digital TV System >
Our focus
Source: T. Wiegand “Digital image communication”
14
Examples of Data Sources-- Classified based upon compression techniques
General digital data: various types of computer filesTextSpeech: human conversationAudio: musicBi-level images: fax, dithered (two-tone) imagesStill gray-level pictures: graphics, natural imagesMotion pictures: video conferencing, television.
-- Data characteristics can be used to reduce bit rate.
15
How Compression Possible?Characteristics of data:
Stationary statistical model -- Shannon information theoryNon-stationary properties such as local correlation
Characteristics of human perception:Finite resolution of hearing and visionAuditory masking effectColor representationVisual masking effect
16
Signals (Waveforms)
Time/ Space
Amp.
Analog Signals x(t)
Conti. Conti.
Discrete-time (discrete-space) (sampled-data) signal x(m)
Discrete Conti.
Digital signals x(m)
Discrete Discrete
17
Audio SamplesPiano (fur Elise) samples and spectrogram (McClellan et al., DSP First, Prentice-Hall, 1998)
18
Data Compression TechniquesInformation lossless: Redundancy reduction ─ The
original data can be completely recovered.Direct: Huffman codes, arithmetic coding, Ziv-Lempel coding, … (narrow-sense data compression)Predictive: Run length coding, …
Information lossy: Information (entropy) reduction --The reproduced data are approximations of the original data. This may not be meaningful for a computer file.Block coding: vector quantization, transform coding, …Sequential: DPCM, tree coding, …Multi-resolution (non-block): sub-band, wavelet, ...
19
Compression TechniquesWaveform coding: Reproduce waveform,e.g., DPCM, transform, sub-band, …-- Universal but lower efficiency.Content-based coding: Reproduce contents, e.g., (speech) vocoder, (image) contour-texture coding, (video) model-based coding.International standards: JPEG, MPEG, H.261/3, …
20
Compression Algorithm Development
Phase 1: Modeling. Describe redundancy in the form of a model. E.g., prob. distribution, Markov model, …--Techniques are then invented to remove redundancy; e.g.,: predictor, transform, …Phase 2: Coding. Representations of “model”and “residual” (unpredictable) information. --Techniques: e.g., quantization and entropy coding of the prediction errors, …
21
Elements in Waveform CodingDecorrelation: Reduce spatial and temporal redundancy. Techniques: prediction, transform, …Selecting representatives: Reduce the number of possible signals. Techniques: quantization, …Entropy coding: Equalize the probability distribution of the output symbols. Techniques: Huffman codes, Ziv-Lempel coding, ...
22
Multimedia Coding Standards
Complete, practical coding algorithms-- A balance between (compression) performance and (implementation) complexity (and a compromise among various interest parties)Critical for telecommunication products and consumer audio/video media products
23
Standards Organizations
CCITT – Comité Consultaitif International Télégraphique et Téléphonique (International Telegraph and Telephone Consultative Committee) ITU – International Telecommunication UnionISO – International Standardization
OrganizationIEC – International Electrotechnical
Commission
24
Image/Video StandardsISO/IEC JTC1 SC29 – ISO and IEC Joint Technical Committee (on Information Technology) Subcommittee 29 (Coding of audio, picture, multimedia and hypermedia)– Working Group (WG) 1:
JBIG (Joint Bi-level Image Group) – 1-bit to 4/5-bit still pictures
JPEG (Joint Photographic Experts Group) – 8-bit or more still picturesISO/IEC JTC1 SC29– WG 11: MPEG (Moving Picture Experts Group) – Motion pictures– WG 12: MHEG (Multimedia-Hypermedia Experts Group) –Multi/Hyper-media exchange format
25
Video Coding StandardsStandards Typical rates ApplicationsITU-T (CCITT) H.261 128 384k bits/s Videophone over ISDN ISO MPEG-1 (11172-2) 1.2 Mbits/s Video CDISO MPEG-2 (13818-2) 4–10 Mbits Digital TV/HDTV (ITU-T H.262) 20 Mbits/s Over air/networksITU-T H.263 < 64k bits/s Videophone over PSTNISO MPEG-4 (14496-2) Low/high-rates Object-orientedISO MPEG-7 (15938) Database Content descriptionITU-T H.263 v2 < 64k bits/s PSTN/wireless VideophoneITU-T H.264 (JVT,AVC) < 40k bits/s Net/wireless Videophone
ISDN: Integrated Services Digital NetworkPSTN: Public Switched Telephone Network
26
MPEG Audio StandardsStandards (Year) Coding rates Sampling rates/
(channel)
MPEG-1 Layer 1 (92) 32k – 448 kbits/s 32, 44.1, 48k (1-2 chs)
MPEG-1 Layer 2 (92) 32k – 384 kbits/s 32, 44.1, 48k (1-2 chs)
MPEG-1 Layer 3 (93) 32k – 320 kbits/s 32, 44.1, 48k (1-2 chs)
MPEG-2 Layers 1,2,3 (94) 32, 44.1, 48, 16, 22.05, 24k (1-5.1)
MPEG-2 AAC (97) 8-64 kbits/s/ch 8, …, 96k (1-96 chs)
MPEG-4 AAC (99) 8-64 kbits/s/ch 8, …, 96k (1-96 chs)
27
Speech Coding StandardsStandards Typical rates (Year) Quality: MOS (1-5)PCM 64 kbits/s (1972) 4.4 (PSTN)G.721 ADPCM 32 kbits/s (1984) 4.1 (PSTN)GSM 13 kbits/s (1991) 3.6 (Cellular) G.728 (low delay) 16 kbits/s (1992) 4.0IS-96A (CDMA) 0.8-8.55 kbits/s (1993) ~3.4 (Cellular) G.729 8 kbits/s (1995) ~4.2G.723.1 5.3, 6.3 kbits/s (1995) ~4.0Half-rate GSM 5.6 kbits/s (1995) ~3.4 (Cellular) AMR 5.15-12.2 kbits/s (1999) ~3.9 (3GPP)
MOS: Mean Opinion Score -- 5=excellent, 4=good, 3=fair, 2=poor, 1=bad
28
Subjects To Be DiscussedBasic theory: Shannon theory (outline)Lossless coding algorithms: Huffman, arithmetic, run-length.Lossy coding algorithms: (vector) quantization, differential coding, Transform coding, subband coding, wavelet-coding, motion estimation/compensation. Standards: JPEG, JPEG2000, H.261/3 basic, MPEG-1/2/4 video.
29
Organization - from ISO/IEC to WG
Where does MPEG fit into the scheme of things
ISOInternation Standards Organization
IECInternational Electrotechnical Commission
ISO/IECJTC 1
Joint Technical Committee 1
Source: MPEG 101, N6706
30
Organization - from ISO to WG
Where does MPEG fit in the scheme of things
ITTFInternational Technical Task Force
Geneva, CH
SC xxSubcommittee
WG 1JPEG
WG11MPEG
Moving Picture Experts GroupMilan, IT
WG12MHEG
(last meeting 2001/03
SC 29Subcommittee 29
Secretariat - ITSJC/IPSJTokyo, Japan
JTC 1Joint Technical Committee - Information Technology
Secretariat - ANSINew YorK City
ISO/IECGeneva, CH
Source: MPEG 101, N6706
31
Organization - from WG to adhoc
WG11 MPEG Committee
Leonardo Chiariglione
WG11 MPEG Committee
Leonardo Chiariglione
SystemsOlivier Avaro
SystemsOlivier Avaro
VideoJen-Rainer Ohm
VideoJen-Rainer Ohm
AudioSchuyler Quackenbush
AudioSchuyler Quackenbush
MDSIan Burnett
MDSIan Burnett
TestTobias Oelbaum
TestTobias Oelbaum Requirments
Rob KoenenRequirmentsRob Koenen
LiaisonJan Borman
LiaisonJan Borman ISG
Marco MattevelliISG
Marco Mattevelli
IntegrationJean-Claude Durford
IntegrationJean-Claude Durford
SNHCMickael Borgas-Sevenier
SNHCMickael Borgas-Sevenier
Source: MPEG 101, N6706
32
Organization - from WG to adhoc
WG11 MPEG Committee
Leonardo Chiariglione
WG11 MPEG Committee
Leonardo Chiariglione
SystemsOlivier Avaro
SystemsOlivier Avaro
VideoJen-Rainer Ohm
VideoJen-Rainer Ohm
AudioSchuyler Quackenbush
AudioSchuyler Quackenbush
MDSIan Burnett
MDSIan Burnett
TestTobias Oelbaum
TestTobias Oelbaum Requirments
Rob KoenenRequirmentsRob Koenen
LiaisonJan Borman
LiaisonJan Borman ISG
Marco MattevelliISG
Marco Mattevelli
IntegrationJean-Claude Durford
IntegrationJean-Claude Durford
SNHCMickael Borgas-Sevenier
SNHCMickael Borgas-Sevenier
Source: MPEG 101, N6706
33
Membership
MPEG experts
National Bodies
WG11
SC29 NB members of SC29
JTC1 NB members of JTC1
ISO/IECNational Standard
Organisations
Source: MPEG 101, N6706
34
The MPEG Modus Operandi
•ExplorationThe search for new technologySeek Industry expertsOpen seminars
•RequirementsEstablish the scope of workCall for Proposals
•Competitive phaseDo HomeworkResponse to CfPInitial technology selection
•Collaborative phaseCore ExperimentsWorking Drafts
Source: MPEG 101, N6706
35
The MPEG Modus Operandi (cont’d)
•StandardizationBallotsNational Body Comments
•AmendmentAdding new technology
•CorrigendaCorrective actions
•New subdivisionsAdd new non-compatible technology
Source: MPEG 101, N6706
36
Where to from here?
• Ongoing search for technology enhancements
• Call for evidence of new/improved technology
• Significant benefits need to be demonstrated
• Actions include:•amendments or •new subdivisions to existing standards or •entirely new standards
Source: MPEG 101, N6706
37
Where to from here
Amendments • compatible enhancements to an existing specification
Corrigenda• Corrective actions to any element of the specification
Collaboration• Establish liaison with other duly constituted organizations• Exchange requirements and specification.
New subdivision of specification• non-compatible enhancements and additions to technology
Source: MPEG 101, N6706
38
Specification development time-lines
No fix duration specified or required -
•Exploration6-12 months depending on extent of search
•Requirements development6-12 months partly in parallel with exploration
•Competitive phase3-6 months partly in parallel with requirements
•Collaborative phase1 year following completion of competitive phase
total approximately 2 yearsSource: MPEG 101, N6706
39
Completion phase of development
Fixed minimum timeline - balloting
3 months 2 months publish
Specification CD FCD FDIS IS
Amendment PDAM FPDAM FDAM AMD
Corrigenda X DCOR X COR
TechnicalReport PDTR X DTR TR
Documenttype
4 months
All documents start out in life as working drafts (WD)
Source: MPEG 101, N6706
40
Alphabet soup
WD - Working Draft
CD - Committee DraftFCD - Final Committee DraftFDIS© - Final Draft International StandardIS © - International Standard© indicates ISO copyright
PDAM - Proposed Draft AmendmentFPDAM - Final Proposed Draft AmendmentFDAM © - Final Draft AmendmentAMD © - Amendment
Source: MPEG 101, N6706
41
More Alphabet soup
DCOR - Draft CorrigendaCOR © - Corrigenda
PDTR - Proposed Draft Technical ReportDTR © - Draft Technical ReportTR © - Technical Report
DoC - Disposition of Comments
Source: MPEG 101, N6706
42
Step by step
WDs become CDs (PDAM, PDTR, DCOR)(changes from here on out are by ballot / comments only)
CDs are balloted by National Bodies
Ballots are submitted by NB’s to SC 29 with or without comments(at FDIS ballots are JTC 1 level)
Comments are processed and approved at MPEG meetings (DoC)
Specification moves on to next stage
Source: MPEG 101, N6706