Unit 1

Principles Of Multimedia

610905MCA Semester III

Faculty : Swapna Kolhatkar

MCA Department

Unit 1

UNIT I: Introduction 8 HrsWhat is multimedia, goals and objectives, characteristics of multimedia presentation,multimedia application, multimedia building blocks, multimedia and internet.

Unit 1 ...

Multimedia Architecture:User interface: GTK +, Qt, Windows multimedia support, hardware support, distributed multimedia application, streaming technologies, multimedia database systems, multimedia authoring tools, overview of multimedia software tools, multimedia document architecture (MHEG, SGML, ODA, OMF, etc.)

What is Multimedia

Multimedia means multiple intermediaries between the source and sink of information or multiple means by which information is stored, transmitted, presented or perceived.

The multiple means by which we perceive information are :

1.Text 3. Sound2. Images 4. Video

Characteristics of Multimedia

Multimedia is any combination of text, graphic, art, sound, animation and video delivered by any electronic means.

Multiple medianon linearityscope of interactivityintegrity Digital Representation.

1. Multiple Media

Pictures1.images – captured by camera2.graphics – hand drawn

Time1.static – text, images, graphics2. variable – sound, movies (motion pictures or animation)

2. Non Linearity

It is the capability of jumping or navigating from one point within a presentation to another point without apparent delay.

Eg. Multimedia based presentation

Non linear eg. - TV shows, motion pictures.

3. Interactivity

Interaction is made possible through a set of interactivity elements embedded within the presentation like buttons, menu items or hyperlinks. It helps in taking user inputs.

4. Digital Representation

Analog – audio and video cassette on magnetic tapes. Also called sequential access.

Digital – random access, hard disk, floppy disk and compact disc.

5. Integrity

Several media types present and playing continuously, need to be integrated or be a

part of a single entity which is the presentation.

Purpose of Multimedia

To present visualizations of concepts

To present animations

To require interactive participation of learner

Multimedia Application

Home EntertainmentEducational PurposeIndustrial TrainingInformation kioskscorporate PresentationsBusinessTourism and travel industryElectronic shoppingcommunication and networks.Medicineengineering applicationscontent based storage and retrieval system (CBSR)

Multimedia Building Blocks

TextSound ImagesGraphicsAnimationVideo

Multimedia and Internet

The Internet has been home to multimedia -- graphics, animation, and sound files -- for years and years.

The intertwining of multimedia and the Internet (slowly and irrevocably) makes sense. People like multimedia on their computers. It's engaging, entertaining, and often makes otherwise complex computers a little easier to use. Certainly it works: Some multimedia educational CD-ROMs are meant to be enjoyed by children who are barely old enough to walk. The Internet, once an ugly, text-only system, certainly needed some pictures, sound, and point-and-click simplicity to make it understandable to the masses.

This example is only the beginning of how multimedia is changing the Internet.

Consumer products and services, such as Mosaic in a Box and America Online, are making the Net still easier for all to enjoy.

GoogleEarth

User interface: GTK +, Qt,

GTK (GIMP Toolkit) is a library for creating graphical user

interfaces. It is licensed using the LGPL (Lesser General

Public License) license, so you can develop open software,

free software, or even commercial non-free software using

GTK without having to spend anything for licenses or

royalties.

Hello World in GTKNow for a program with a widget (a button).

#include <gtk/gtk.h>

static void hello( GtkWidget *widget, gpointer data ){ g_print ("Hello World\n");}static gboolean delete_event( GtkWidget *widget, GdkEvent

*event, gpointer data ) { g_print ("delete event occurred\n"); return TRUE;}

GTK+ contd..

The GTK+ library itself contains widgets, that is, GUI

components such as #GtkButton or #GtkTextView.

The GtkButton widget can hold any valid child widget. That is

it can hold most any other standard GtkWidget. The most

commonly used child is the GtkLabel.

Syntax : GtkWidget *gtk_label_new( const char *str );

Qt+

Widgets are the basic building blocks for graphical user

interface (GUI) applications built with Qt. Each GUI

component (e.g. buttons, labels, text editor) is a widget that

is placed somewhere within a user interface window, or is

displayed as an independent window. Each type of widge is

provided by a subclass of QWidget, which is itself a

subclass of QObject.

Distributed Multimedia Applications

In a true distributed system not only the hardware but the software also remains dispersed.

The reasons for distribution of multimedia are those of traditional data processing objects plus some new ones.

i.e. Moving and sharing.

Moving and sharing...

Moving data to the point of useSharing of data and resources.

Eg. An architectural blueprint can be worked upon by different groups of construction engineers, dispersed geographically.

More Examples

Companies can communicate product information to customers in format and location they prefer.

Advertising companies Medical professionals can collaborate

Architecture

Creation of subsystem where the content is created.

A storage subsystem where the created content is stored.

A distribution subsystem using which the content may be distributed to multiple users.

A management subsystem where the entire operations are managed.

Distributed Multimedia Applications

Videophone, video conference and multimedia conference

Video on Demand (VoD)Movie on Demand (MoD)Interactive Television (ITV)Voice over IP (VoIP)

Streaming Technologies

Streaming media is media that is consumed (read, heard, viewed) while it is being delivered.

It is more a property of the delivery system than the media itself.

2 Techniques1. Progressive download2. Real time streaming.

Progressive download

A media file containing audio or video is downloaded and stored on the clients local file system. As the file is being downloaded, the client is able to play back the media file without having to wait for the file to be downloaded in its entirety.

Mostly used for short video clips.

Real time Streaming

The media file is streamed to the client but is only played and not stored by the client.

Mostly used for long videos, internet radio and TV broadcasts.

A streaming media system is made of many interacting technologies. Video cameras and audio recorders

Multimedia Database Systems

Multimedia database systems (MMDBS) can be defined as collections of media elements like text, image, audio and video that can be stored, manipulated and retrieved.

1. Storage and retrieval of multimedia

a) Textual NotationsMMDB needs to be populated first before

it can be utilized.For retrieval, text notations were manually

generated by individuals. Disadvantages :

Subjective nature of manual desc.More time and effortAll multimedia may not have text desc.

1. Storage and retrieval of multimedia

b) Low-Level Featurescolorshapeloudnesspitch.

c)Semantic featuresA Semantic Object (SO) is a collection of

image pixels or audio samples that corresponds to the projection of a real object or event in an image, audio, video sequence eg a cluster of trees, a car moving on a road.

1. Storage and retrieval of multimedia

d) Knowledge based InterpretationExtraction and representation of low level

features and semantic objects is not sufficient to appropriately handle user queries because the system still cannot interpret what the features or objects mean.

e) Queries and searching.'display all images containing the color red''display all audio files containing a laugh'

General Architecture of MMDBS

A multimedia database consists of 2 levels of abstractions

1. physical level consisting of the media objects (Media DB)

2. logical level consisting of the features characterizing the media objects. (Features DB)

Basic overview of an MMDBS

Symbolic links

Media file

Media DB

Feature ExtractionFeature Info

Feature DB

Extended view of an MMDBS

Media DB Feature DB

Media element

Result list of query

Feature Extraction

Query Interface

Semantic Info

Query

Symbolic Link

Prototype Implementations

1. MINOS , MULTOS2. REDI3. MIM4. QBIC5. OVID6. CORE7. VisualSEEk8. SCORE9. CHABOT10. CANDID11. VISION12. JACOB13. FACE etc

Multimedia Authoring Tools

Also known as authorware, a program that helps you write hypertext or multimedia applications. Authoring tools usually enable you to create a final application merely by linking together objects, such as a paragraph of text, an illustration, or a song. By defining the objects' relationships to each other, and by sequencing them in an appropriate order, authors (those who use authoring tools) can produce attractive and useful graphics applications. Most authoring systems also support a scripting language for more sophisticated applications.

Examples

AuthorwareFormula Multimedia

HyperStudioiShell 3

MaxMediaMediaMixer

OpusPlatypus

Revolution

Overview of Multimedia Software Tools

* Digital Audio * Music Sequencing and Notation * Image/Graphics Editing * Image/Graphics Editing * Animation * Multimedia Authoring

Ref : http://www.cs.cf.ac.uk/Dave/ Multimedia/node40.html

Digital Audio

Macromedia Soundedit -- Edits a variety of different format audio files, apply a variety of effects

CoolEdit -- Edits a variety of different format audio files

Music Sequencing and Notation

Cakewalk

Cubase

Logic Audio

Marc of the Unicorn Performer

Image/Graphics Editing

Adobe Photoshop

Adobe Premiere

Macromedia Freehand

Animation

Avid SoftImage

Animated Gif building packages e.g.

GifBuilder

Multimedia Authoring

Macromedia Director

Authorware

Microcosm

Flash : Macromedia

HyperCard

Multimedia Document Architecture

Document

A document consists of a set of structural information using which some content is displayed.

A document is aimed at the perception of a human and is accessible for computer processing.

A multimedia document is a document which comprises of information coded in at least one continuous (time dependent) medium and in one discrete (time independent) medium.

Document Architecture

It specifies the content of a document, the physical layout of document like pages and frames and logical layout of the document like paragraph, sections, headings etc.

For exchanging of documents between different applications, the documents themselves must adhere to a standard architecture.

Standardized Architectures

SGML (Standard Generalized Markup Language)

ODA (Open Document Architecture)specifically for multimedia ....

OMF (Open Media Framework)

MHEG (Multimedia and Hypermedia Information Coding Expert Group)

DCA (Document Content Architecture)

MODCA (Mixed Object Document Content Architecture)

SGML

It prescribes a standard format for embedding descriptive markup within a document.

SGML allows you to set up hierarchical models for each type of document produced.

It forces each element in the structure (labeled chapter, title, paragraph) to fit in the logical, predictable structure of document.

A typical document has 3 layers – structure, content and style.

SGML ...

1. StructureDTD – (Document Type Definition) which provides a framework and rules for the definition of the elements that constitute a document.

A DTD accompanies a document wherever it goes.

SGML ...

2. Content

Content is the information itself. Creating an SGML document

involves inserting tags around content.

<topic><par>Multimedia is interesting !! </par></topic>

3. Style

2 standard based style sheets :

FOSI – Formatting Output Specification Instance

DSSSL – Document Style Semantics and Specification Language

SGML Structures1. Elements – element names are always case – insensitive. It has 3

parts, start tag, content, end tag [<pre> example </pre>]

2. Attributes – attribute/value pair appear before the final '>'

[<h1 align = 'center'> ................. </h1>]

3. Entities – named part of a marked up document, irrespective of

any structural considerations. When parser encounters any entity

reference, it immediately substitutes the value declared for the

entity name.

SGML DTD

Rules are the first stage in the creation of a formal specification for the structure of an SGML document ie SGML DTD

Minimization Rules determine whether or not start and end tags must be

present in every occurrence of the element concerned.

Content Model for specifying the content in terms of other elements or

using special reserved words.

Occurrence indicators, group connectors and entity definitions together

make the DTD.

Example

<! ELEMENT poem - - (title?, stanza+)>

<! ELEMENT title - 0 (#PCDATA)>

- : must be present (start tag is present for both)

0 : may be omitted. (for title, end tag may be omitted)

#PCDATA : valid character data

?, *, + : Occurrence indicators

ODA

The main property of ODA is the distinction among content, logical structure and layout structure. This is in contrast to SGML where only logical structure and contents are defined.

1. The ODA Architecture – enables compound documents to be

encoded and interchanged electronically between systems

conforming to the standards.

2. Logical, layout and content information – indept of page layout

3. Generic structures – reports as defining class of the document.

4. Style – Layout (figure + caption) and presentation(indentation of

first line)

5. The document profile – author name, create date, keywords etc

MHEGAn MHEG document can be distributed over the network or in

CD ROM format, but for presentation only, no editing is possible.

MHEG is intended to produce electronic multimedia versions

of printed books.

It supports encoding of interactive multimedia presentations

defining a platform independent set of multimedia objects

like buttons, text fields etc. Facility of an embedded

scripting language for event handling is also provided.

MHEG ...

It is based on object hierarchy where each object has a set of components and behaviours.

MH-Object

Behavior

Action

Link

Script

Component

content

Interaction

Descriptor

HyTIME (extension of SGML)

It is meta language similar to SGML. The structure allows for:

1. The specification of spatial interrelations between

document elements.

2. The specification of temporal interrelations between

document elements (synchronization and time

dependencies)

3. The specification of the overall logical structure

4. The specification of pointers between elements within or

outside the document.

OMF

It is a standard format for the interchange of digital media data among heterogeneous platforms. The primary emphasis is video production.

1. Source (analogue) material object represent videotape and film

so that the origin of the data is readily identified.

2. special track types store (SMPTE) time codes for segments of

data.

3. Transitions and effects for overlapping and sequences of

segments are predefined.

OMF ...

The OMF provides structures for 3 distinct elements :

1. Digital media data – audio, video and graphics for m only part of the

information.

2. Media Sources – describes the original physical sources of data.

3. Composition – describe the arrangement of sections of sources and

how they are played over time.

OMF interchange keeps sources separate from compositions. More than one

composition can reference sections of the same digital media data, reducing

storage requirements.

OMF ...

OMF uses objects called Mobs (media Objects) to uniquely describe compositions and sources. Each Mob contains an identifier called a MobID

It uses SMPTE time code for numbering frames.

It also has support for motion control.