Multimedia Systems Design

Multimedia Systems DesignIntroduction

Introduction “A Picture is worth a thousand words” There have been ongoing attempts to

improve productivity of knowledge workers. Reducing paper flow (reports, memos) has

been one important area where electronic mail and groupware technologies are beginning to have impact.

As the demands of business increased, the 1980s were distinguished by overnight mail and fax as the means of communicating important information that did not require face to face meeting.

Introduction Electronic Mail Electronic Meetings Video Conferencing ISDN HDTV

Introduction The technologies at the core of

the computing revolution have reached a point where one can envision a computing system composed of a number of elements together by various communication methods, all striving to serve user.

Introduction The machine user interaction will be at

conversational level rather that through the typed cryptic commands of yesterday or the mouse movements still in use.

As screens become larger and feature increasingly higher resolutions, the demand to see higher-resolution images is increasing.

New technologies especially developed for GUI allows multiple applications to operate simultaneously using multiple windows, thereby placing increasing demands on screen estates.

Introduction The main challenge for the designers of

multimedia systems will be to pack large volumes of information in compact packets.

Local area network provides bandwidth ranging from 10mbits/sec to several gigabits per second.

WAN depends on land, sea or satellite based communication channels that carry a large number of conversations.

With such bandwidths compression and decompression of data is important.

The size of compressed data is important because it determines the bandwidth necessary to meet acceptable performance requirements.

What constitutes Multimedia? Multimedia meant a combination of text

with images. Document image management was an

outgrowth of facsimile technology. Facsimile provided a means of scanning and

converting a document into coded information that described each pixel as white or black.

When the number of pixel per inch was low, the information was easily manageable.

When the pixel densities increased as better fax machines were developed, the information became very large.

What Constitutes Multimedia? Wide variety of multimedia applications are

in use or under development like:1. Video Conferencing2. Medical applications, such as analysis of surgical procedures and high resolution.3. Real estates on-line video clips with property descriptors4. Multimedia help and training material5. Security systems for employee identification

Multimedia Elements Multimedia applications require

dynamic handling of data consisting of a mix of text, voice, audio components, video components, and image animation.

Integrated multimedia application allows user to cut sections of all or any of these components and paste them in a new document or in another application.

There are several components of Multimedia:

Facsimile Fax. A process by which fixed graphic material including pictures, text, or images is scanned and the information converted into electrical signals which are transmitted via telephone to produce a paper copy of the graphics on the receiving fax machine. Facsimile transmissions were the first practical means of transmitting document images over a telephone line. It uses group3 compression standards. Typical pixel densities used for facsimile are

in the 100 to 200 dpi range. The higher resolution are used to enhance

the clarity of documents.

Document Images Document images are used for storing

business documents that must be retained for long periods of time or may need to be accessed by a large number of people.

Providing multimedia access to such documents removes the need for making several copies of the original for storage or distribution.

For storage of such documents minimum dpi required is 300 dpi.

An uncompressed A-size(8 -½ inch x 11 -½ inch image) is over 1 Mbyte.

Even with group 3 compression this size reduces to approximately 75kbytes.

Document Images For images that are gray scaled or color, the

sizes are much larger to accommodate the pixel color information.

Scanning of document images at that a high resolution requires very efficient compression and decompression technologies.

Photographic Image Photographic images are used for wide

range of applications such as employee records for instant identification at a security desk, real estate systems with photographs of house in the database containing the descriptions of house.

For bank signature cards, patient medical history, fingerprint cards.

To create photographic image with laser printer resolution of 600 dpi is considered essential .

Geographic information system maps Maps created in GIS are widely used for

natural resource and wild life management as well as urban planning.

These systems store the graphical information of the map along with a database containing information relating high lighted map elements.

Two kinds of technologies are used for storage and display of geographic maps.

Raster technology to display natural resources.

Another application combines raster image and vector overlay showing railroads or highways and other human made structure.

Voice Commands Voice commands allow the user to direct

computer operation by spoken commands. Voice commands allow hands free usage of

computer applications by allowing command via short voice commands rather than a keyboard or pointing device.

Recognition of commands requires specialized techniques and powerful processing capabilities to compensate for differences in pitch, accents, and voice modulation techniques of users.

Voice Synthesis Transforming computer output to voice or

audio output. It is easier to achieve in compare to voice

recognition. The cadence(the consistency with which the

spoken words are stung together) of the output should be good for message to be clear.

Digital signal processors designed for such an application have the processing power to perform the computations and maintain correct cadence.

Audio Messages Audio messages are substitute of text

messages. Computers having microphone can record

audio message and can embed it into electronic mail.

Audio messages also require large volumes of storage.

Compression techniques are required to manage the storage efficiently.

With image speed of decompression and display is important, speed of decompression and playback of audio message with proper cadence is important for audio message.

Video Messages Similar to audio messages, video messages

can be embedded into electronic mail messages.

The storage and playback requirements are even more complex with video messages because of storage of each video shots.

Full motion stored and live video Full motion video act as very useful idea for

online training manuals. Full motion video can be used for video

conferencing, live video presentations. Full motion video requires high bandwidth

for communication, massive storage requirements and high compression techniques.

High-definition television (or HDTV) is a digital television broadcasting system with higher resolution than traditional television systems

Again digital HDTV(High definition TV) places another major demand on the design- that of special effects such as zoom, freeze frame, image merging and image reconstruction.

Holographic Images Holographic images extend the concept of

virtual reality by allowing the user to get inside a part, such as, an engine and view its operation from the inside.

Multimedia Applications Document Imaging Image Processing and Image

Recognition Full Motion Digital Video

Applications Electronic Messaging

Document Imaging Organizations such as insurance agencies,

law offices, state governments, and the federal government, including the department of defense, manage large volumes of documents.

Document imaging makes it possible to store, retrieve, and manipulate very large volumes of drawings, documents, and other graphical representations.

Ex: Sending large volumes of engineering data about complex systems in electronic form rather than on paper.

Document Imaging Almost all document image systems use

below work flow:1. Scanning images2. Performing Quality checks3. Performing data entry based on the contents of the images.4. Indexing them5. Storing them on optical media

Image Processing and Image Recognition Image processing involves:

Image RecognitionImage EnhancementImage SynthesisImage Reconstruction

Image Processing and Image Recognition The original image is not altered in

document image workflow. Image processing system actually alter the

contents of the image itself. Ex: Recognition of images: in factory floor

quality assurance systems, Image Enhancement: Satellite

reconnaissance systems Image Synthesis: Law enforcement

suspect identification Image Reconstruction: In Plastic surgery

design systems

Image Enhancement

Image enhancement improves the quality of images for human viewing.

Removing blurring and Noise, increasing contrast .

For example an image might be taken of any cell of body which might be of low contrast range could enhance the image.

Increasing sensitivity and contrast makes picture darker by making borderline pixels black or increasing the gray scale level of pixels.

Image Animation

Computer created or scanned images can be displayed sequentially at controlled display speeds to provide image animation.

Image animation is a technology developed by Walt Disney and brought into every home in the form of cartoons.

Optical Character recognition

OCR technology is used for data entry by scanning typed or printed words in a form.

OCR technology used as a means for data entry may be used for as simple a task as reading the number of an invoice, or for capturing entire paragraphs of text.

Handwriting Recognition Handwriting Recognition has been subject

of intensive research now a days. Pen based systems are designed to allow

user to write commands on an electronic tablet.

Key Design Constraint: Ability to recognize writer independent continuous cursive handwriting accurately in real time.

Two factors are important: Strokes or shapes being inputted and velocity of input.

Parse recognizer will identify topology of strokes.

Handwriting Recognition The stroke is compared with the prototype

character set until a match is found or all predefined prototypes are checked.

Full Motion Digital Video Applications Full Motion video is the most complex and the

most demanding component of multimedia applications.

Full Motion Video Applications:1. Training and Manuals:

On-Line Reference CD-Rom Training

2. Business Applications; E-Mail Video Conferencing Presentations Demos

Full Motion Digital Video Applications3. Games and Entertainment

Interactive TV CD-ROM Interactive Games

Full Motion Digital Video Requirements It should be possible to attach full motion video

clips to other documents such as memos, presentations.

Users should be able to take sections of a video clip and combine the sections with sections from other video clips to form their own new video clip.

Features such as rewind, fast-forward, play, and search should be available.

It should be possible to view the same clip on a variety of display terminal types with varying resolution capabilities without storing multiple copies in different formats.

It should be possible for users to move and resize the window displaying the video clip.

Full Motion Digital Video Requirements The users should be able to adjust the contrast

and brightness of the video clip and also adjust the volume of the associated sound.

Evolving Technologies for multimedia systems Hypermedia documents

HypertextHyper speech

HDTV and UDTV 3-D Technologies and Holography Fuzzy Logic Digital Signal Processing

Hypermedia documents Technical and business documents in

electronic form. It contains text, embedded or linked

multimedia objects such as image, audio or full motion video.

Hypermedia has its roots in hypertext.

Hypertext Hypertext allows authors to

Link information together, Create information paths through a

large volume of related text in documents

Explicit connections or links allow readers to move from one location to another in a document or to other documents.

Hypermedia is an extension of hypertext in that these electronic documents, in addition to containing text will also include any kind of information that can be stored electronically.

HypertextAn important issue arise with

Hypermedia document is storage. It does not make sense to embed full

copies of each component of a hypermedia document within document file.

Rather, since the embedded components may also be included in other documents.

Hypermedia document only store references to the documents.

The user sees a single document, but the locations of the various components that are in document are transparent.

HDTV and UDTV The electronics industry is attempting to raise the resolution levels

of commercial television broadcasting. Among the better-known television broadcasting standards are

NTSC, PAL, SECAM, NHK, and others. These standards range in resolution from 525 lines for NTSC to

819 lines for the French standard. A hot debate has been in progress for bringing the world together

on a single high-definition television (HDTV) broadcasting standard.

A 1125-line digital HDTV has been developed and is being commercialized.

NHK of Japan is trying to leapfrog the digital technology to develop ultra definition television (digital UDTV) featuring approximately 3000 lines. Figure 1-5 shows the progression in the resolution of television pictures..

HDTV and UDTV There are some key technologies necessary to make

the jump to a 3000-line UDTV standard. It requires the development of ultra- resolution

displays at a commercially viable price, high-speed video-processing ICs, and ultra broadband communications bandwidths for WAN services such as ISDN.

Fuzzy Logic It includes graphics rendering, data

compression for images, voice and video, voice recognition and synthesis, as well as signal processing for video, high resolution facsimile.

Graphics rendering involves the painting of three dimensional objects onto a two-dimensional display

Any application that is computationally intensive can benefit from the mathematical principles behind fuzzy logic.

Types of images

Visible Images Visible Images include

Drawings such as Blueprints, Engineering Drawings, Space maps for Offices,n Town Layouts, Paintings, Photographs, Documents, Still Frames

Non-visible Images: Non-visible Images are not stored as images but it is displayedas images. The examples include Pressure Gauges Temperature Gauges Other metering displays

Abstract Images Abstract Images are not really images. They exist

as real world objects or representations. They are computer generated images based on

some arithmetic calculations. The examples are Kaleidoscope – Shows different patterns due to

relative positions of glass beads when it is rotated The mathematical used for generating Abstract

Images: Discrete Functions – It results in still images that

remain constant. Continuous Functions – It is used to show

animated images and operations such as image fading or dissolving into another image

Multimedia data interface Standards

Video Processing Standards: Intel’s DVI

Apple’s Quick Time Microsoft’s AVI

Intel’s DVI DVI is actually both the name of the Digital Video

Interactive hardware system sold by Intel and the file format associated with that system. DVI technology is essentially a PC-based interactive audio/video system used for multimedia applications

Apple’s Quick time QuickTime (sometimes called QTM) is the native

method of storing audio and motion video information on the Apple Macintosh platform.

It is used to record and play back multimedia information and store the data on magnetic or optical media.

QuickTime, however, is not only a data-storage format. It is also a collection of tools (the Movie Toolbox) that allows QuickTime movies to be modified (edit, cut, copy, paste, and so on), just as a word processor is capable of modifying an ordinary text file.

Apple’s Quick time Quick time includes:

System Software File formats Compression/Decompression

algorithms Human interface standards

Microsoft’s AVI Microsoft’s Audio video interleave standard, similar

to apple’s quick time offers low cost, low resolution video processing.

It allows users to set parameters such as window size, quality, compression algorithm