Top Banner
ECE8873 MPEG-7 Deryck Yeung
39

ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

Jan 02, 2016

Download

Documents

Neil Cameron
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

ECE8873MPEG-7

Deryck Yeung

Page 2: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

Overview

Summary of MPEG-1,MPEG-2 and MPEG-4

Why another standard? MPEG-7 What’s next? Conclusion

Page 3: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

Origin of MPEG (Movie

Picture Expert Group)

An Evolution Theory of MPEG

Page 4: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

MPEG-1

MPEG-1

Page 5: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

Summary MPEG-1 MPEG-1(1988-1993)

-Goal: coding of moving picture with associated audio at rate up to 1.5Mbps-Entertainment and storage-Video storage with VHS quality-Video CD is full application of MPEG-1-3 layers in audio coding.

-Audio Layer 3 is the famous MP3

Page 6: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

MPEG-2

MPEG-1

MPEG-2

Page 7: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

Summary MPEG-2

MPEG-2(1990-1995)- Higher bit rate at 80Mbps- mainly broadcast- Application in DVD, HDTV etc.- Backward compatibility with MPEG-1

Page 8: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

MPEG-3

MPEG-1

MPEG-2MPEG-3

Page 9: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

MPEG-3

MPEG-1

MPEG-2MPEG-3

Page 10: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

MPEG-4

MPEG-1

MPEG-2

MPEG-4

Page 11: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

Summary MPEG-4

MPEG-4 embraces several areas: Coding of visual objects Coding of audio objects Scene description and user

interaction Most important: object coding This set the stage for MPEG-7.

Page 12: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

Q&A session:

Do we need another standard?If so, why?

Page 13: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

Why MPEG-7? Let’s take a step back! Almost all standards up to this point

involve either some form of waveform coding (e.g. linear prediction etc) or transform coding (e.g. DCT etc).

Look at linear prediction, for example. Speech waveforms are modeled as

output of a filter.

Page 14: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

Why MPEG-7? Now, given the input to the filter

and the filter coefficients we can reconstruct the waveform, albeit with some distortion.

{Waveform } {input, filter coeffs}

Hence, we have a different (more efficient) representation of the waveform.

Represented by

Page 15: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

Why MPEG-7?

Now look at transform coding.

Data block ---------------------> ------------ --------

Transform allows data to be ‘rotated’ to bring out the structure of the waveform for efficient coding.

T Q T-1

New representation

Page 16: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

Why MPEG-7?

Classical Fourier Series expansion is an example Need uncountably infinite number of

‘samples’ to represent a sinusoid in time domain

Fourier series expansion needs only 3 parameters to represent the same signal

{sinusoid} {ω0, 2 FS coefficients.}

Represented by

Page 17: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

Big picture so far.

waveform

Newrepresentation

quantization

Page 18: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

Why MPEG-7?

Page 19: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

Why MPEG-7? One motivation for MPEG-7 is to allow

meaningful search for data. We are missing one thing in our

picture. MPEG-7 allows us to go one step

further. It gives meaning to the ‘bits’ by

allowing metadata. Or ‘information’ about ‘information’.

Page 20: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

Why MPEG-7?

waveform

Newrepresentation

?

quantization

Page 21: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

Why MPEG-7?

waveform

Newrepresentation

description

quantization

MPEG-7

Page 22: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

Link between MPEG-4 and MPEG-7 MPEG-4 codes contents as objects. But an object can be described in many

different ways, just like how we can describe the object ‘apple’ in for example French, English, Russian etc.

MPEG-7 defines the ‘universal language’ as to how these objects are described and the ‘grammar’ as to how ‘sentences’ of these objects can be made.

Page 23: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

MPEG-7(1998-2001) Also known as ’Multimedia Content Description

Interface’. An ISO/IEC standard Strictly speaking, MPEG-7 is not a data

compression scheme. MPEG-7 is mainly a software implementation. MPEG-7 specifies the rules as to how to describe

audiovisual data content whereas MPEG-1,2 and 4 make content available.

MPEG-7 is not targeted at specific application. It aims to be as generic as possible for further extension.

Page 24: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

Scope

Page 25: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

Main elements MPEG-7 consists of 3 parts:

Description tools (like tool box in Matlab):

-Descriptors (D) -Description Schemes (DS).

Description Definition Language (DDL). System tools.

Page 26: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

Multimedia Description Tools:

Descriptors (D): just like ‘data types’ in Matlab Describes low-level audio or visual features such

as color, motion, texture etc as well as audiovisual content such as location, time etc

Description Schemes (DS): Just like ‘functions’ in Matlab Describes high-level audiovisual (AV) features

such as regions, segments, events etc. DS not only describes relationships among D’s, but relationships among basic DS’s as well.

Page 27: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

Big picture now

Page 28: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

Description Tools:

Over 100 description tools (just like Matlab toolboxes) have been defined. They can be divided as follows:

Basic datatypes

Links & media localization

Basic Tools

Models

Basic elements

Navigation & Access

Content management

Content description

Collections

Summaries

Variations

Content organization

Creation & Production

Media Usage

Semantic aspects

Structural aspects

User interaction

User Preferences

Schema Tools

User History Views Views

Page 29: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

Example:

One DS in “content description” ‘toolbox’ is the Segment DS.

Page 30: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

Another example: SegmentRelationship DS

Page 31: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

Other Multimedia Description Tools

MPEG-7 have a general set of multimedia Description Tools.

From this general set 2 other sets are defined: MPEG-7 Visual: Description tools

dealing with only visual description. MPEG-7 Audio: Description tools

dealing with only audio description.

Page 32: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

Description Definition Language (DDL). DDL is based on XML schema Language. Since XML is not specifically designed

for audiovisual description and real-time usage, it has been extended by MPEG-7 to suite the purpose.

DDL defines syntax as to how D’s and DS’s can be described and combined.

DDL allows designers to create their own D’s and DS’s.

Page 33: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

System Tools

A set of tools that support binary representation of D’s and DS’s.

Broadly, they are divided into 2 groups: Textual format (TeM) Binary format (BiM)

Page 34: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

Problem in searching

Let’s go back to the problem of searching using MPEG-7

3. Spatial edge distribution

2. Spatial color distribution

4. Object shape

1. Color histogram

Page 35: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

Other uses of MPEG-7

Digital libraries Multimedia directory services (e.g.

Tourist information, Geographical information systems)

Home Entertainment And many more!

Page 36: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

What has been done so far

The ‘wrapping’ effect of all these MPEG standards.

Page 37: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

What is next? MPEG is developing another

standard, called MPEG-21. Alternatively known as ‘Multimedia

Framework’. The goal is interoperability. It promises an environment where

delivery and use of all content types will be possible.

Page 38: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

Summary Development of MPEG family

MPEG-1 and MPEG-2 are data compression MPEG-4 is the start to toward data

description (object coding). MPEG-7 is data description

Move from merely data compression to data description

Next step is interoperability (MPEG-21)

Page 39: ECE8873 MPEG-7 Deryck Yeung. Overview Summary of MPEG-1,MPEG-2 and MPEG-4 Why another standard? MPEG-7 What’s next? Conclusion.

Bibliography Tarbell, Richard: MPEG-1 slides Tian, Dihong: MPEG-2 slides Dechaza, Olivier: MPEG-4 slides Martinez,Jose: ‘MPEG-7 Overview’ Chiariglione, Leonardo: ‘MPEG: achievements and

current work’ Chiariglione, Leonardo: ‘The MPEG generation, new

information-age elements’ Koenen, Rob: ‘Object-based MPEG offers flexibility’ at

www.eetimes.com Day, Neil; Martinez Jose: ‘Introduction to MPEG-7(V4.0)’ http://www.chiariglione.org/mpeg/ http://www.mpegif.org/resources.php