MPEG-7

MPEG-7Cochin University of Science And Technology

Multimedia Content Description Interface

Kunal Kishore

A Presentation By :-

MPEG-7Cochin University of Science And Technology

Contant

Introduction Need of MPEG-7 History of MPEG Elements Of MPEG-7 Parts of MPEG-7 Application Conclusion

Introduction In September 2001, Moving Picture Experts Group (MPEG) specified an international standard called Multimedia Content Description Interface, or MPEG-7. MPEG-7 is a standardized description, based on XMLSchema, of various types of AV content. Material (e.g., still pictures, graphics, audio, video) with associated MPEG-7 data can be efficiently indexed and searched for.

MPEG-7Cochin University of Science And Technology

MPEG-7Cochin University of Science And Technology

Why do we need MPEG-7 ?

Need

Fast And accurate acess Personalized Content Production and Consumption Content Management’ Automation

+VisualAudioSketch

Support for Advanced Query

MPEG-7Cochin University of Science And Technology

MPEG- History…1

MPEG: Moving Picture Experts Group

ISO / IEC/JTC1/SC29/WG11

Established in 1988

A Working Group of ISO/IEC in charge of the Development of Standards for Coded epresentation of Digital Audio and Video

MPEG-7Cochin University of Science And Technology

MPEG- History…2

MPEG-1: Interactive CD and MP3 11 / 1992

MPEG-2: DTV, STB, DVD 11 / 1994

MPEG-4: Web and Mobility ver1: 09 /1998, ver2:

11 /1999

MPEG-7: ??? 08 / 2001

MPEG-7Cochin University of Science And Technology

Scope of MPEG-7

The description generation (feature extraction, indexing) is not a normative part of the standard The description consumption (search engine, retrieval process) is also not a normative part of the standard. Just the description is normative

Standardizes the minimum needed to ensure interoperability

MPEG-7Cochin University of Science And Technology

MPEG: Main Elements…1

MPEG-7Cochin University of Science And Technology

Descriptors (Ds): representations of features To describe various types of features of multimedia

information To define the syntax/semantics of each feature

representation Color example – specify histogram in RGB color space

Description Schemes (DSs) Specifies the structure and semantics of the relationships

between its components, which may be either Ds or other DSs.

MPEG: Main Elements…2

MPEG-7Cochin University of Science And Technology

Description Definition Language (DDL) The language to specify DSs and possibly Ds Will allow the creation of new DSs (and possibly

Ds) and the extension modification of existing DSs . System tools

To support multiplexing of description, synchronization issues, transmission mechanisms, file format, etc.

MPEG: Main Elements…3

MPEG-7Cochin University of Science And Technology

Systems Description Definition Language Visual Audio Multimedia Description Schemes Reference software Conformance Testing Extraction and use of descriptions

Parts of MPEG-7 Standered

MPEG-7: Systems

It defines tools to:

Provide for efficient storage and transport

Synchronize between content and description

Manage and protect intellectual property

MPEG-7Cochin University of Science And Technology

MPEG-7Cochin University of Science And Technology

Grid Layout Time Series 2D-3D Multiple View Spatial 2D Coordinates Temporal Interpolation

Basic Structure of Visual Descriptors

MPEG-7Cochin University of Science And Technology

Color Texture Shape Motion Localization Face Recognition

MPEG-7 Visual Descriptors

MPEG-7Cochin University of Science And Technology

Sound Effects

Music Instrument Timbre

Spoken Content

Melody Contour

MPEG-7 : Audio Descriptors

Color Camera motionMotion activityMosaic

Color Motion trajectoryParametric motionSpatio-temporal

shape

Color ShapePositionTexture

Video segments Still regions

Moving regions Audio segments

Spoken content Spectral

characterizationMusic: timbre,

melody

Low level Audio Visual descriptors

MPEG-7Cochin University of Science And Technology

MPEG-7Cochin University of Science And Technology

Multimedia Description Schemes- MDS

• MPEG-7 Multimedia Description Schemes (DSs) are metadata structures for describing and annotating audio-visual (AV) content. • The DSs provide a standardized way of describing in XML the important concepts related to AV content description and content management in order to facilitate searching, indexing, filtering, and access. • The DSs are defined using the MPEG-7 Description Definition Language (DDL), which is based on the XML Schema Language, and are instantiated as documents or streams.

MPEG-7Cochin University of Science And Technology

Organization of MDS tools

MPEG-7Cochin University of Science And Technology

Example of Segment Tree

19

Foreground

SR1:· Creation, Usage meta

information· Media description· Textual annotation· Color histogram, Texture

SR2:· Shape· Color Histogram· Textual annotation

SR6:· Color Histogram· Textual annotation

SR5:· Shape· Textual annotation

SR4:· Shape· Color Histogram· Textual annotation

SR3:· Shape· Color Histogram· Textual annotation

Background

MPEG-7Cochin University of Science And Technology

XML was created to structure, store, and transport information.

• It is just plain text. Software that can handle plain text can also

handle XML.

• It is just pure information wrapped in tags. Someone must write a

piece of software to send, receive or display it.

• XML-aware applications can handle the XML tags specially. The

functional meaning of the tags depends on the nature of the

application.

DDL: XML & format

MPEG-7Cochin University of Science And Technology

Example

MPEG-7Cochin University of Science And Technology

Binary Format- BiM

BiM is designed for transmission and storage BiM encoders and decoders can deal with any XML language. BiM is a schema oriented encoding scheme

• The binary format is deduced from the schema definition

• There is no need to define coding tables or a specific encoding mechanism

MPEG-7Cochin University of Science And Technology

Application Scenario

MPEG-7Cochin University of Science And Technology

Mobile phone Personal video recording (PVR) systems Content delivery Surveillance, remote sensing, visual control

equipment Database, server Storage and retrieval of audiovisual databases

(image, film, radio archives)

Additional Application

Some Industrial Tools

• IBM MPEG-7 Annotation Tool• Ricoh MovieTool• Canon MPEG-7 Speech Recognition Engine

MPEG-7Cochin University of Science And Technology

MPEG-7Cochin University of Science And Technology

MPEG-7 Timeline Work plan

Call for proposalsOct 1996Call for proposalsOct 1996

20012000199919981996

Working draftMar 2000Working draftMar 2000

Committee draftOct 2000Committee draftOct 2000

Draft internationalStandardJuly 2001

Draft internationalStandardJuly 2001

Final committee DraftJan 2001Final committee DraftJan 2001

InternationalStandardOct 2001

InternationalStandardOct 2001

Divergence Convergence

• MPEG-7:

– AV content description for interoperable application

• Description Definition Language:

– XML Schema (flexibility) + Binary version (efficiency)

• Description Schemes:

– Library of description tools

– Covers a wide range of generic needs

Conclusion

MPEG-7Cochin University of Science And Technology

MPEG-7Cochin University of Science And Technology

Refrences

Multimedia System Standered & n/w :Puri

MPEG-7 Awareness event(http://www.mpeg-industry.com)

http://www.mpeg.chiariglion.org (A MPEG Home Page )

http://www.mpeg-alliance.com

http://www.w3.org/1999/05/06-xmlschema-1

http://www.dlib.org/dlib/september99/hunter/09hunter.html

MPEG-7Cochin University of Science And Technology

Questions ?

MPEG-7Cochin University of Science And Technology