Module 4

Module 4

ELEMENTS OF MULTIMEDIA

Multimedia is a term that was coined by the advertising industry to mean buying ads on TV, radio, outdoor and print media. It was originally picked up by the PC industry to mean a computer that could display text in 16 colors and had a sound card. The term was a joke when you compared the PC to the Apple Macintosh which was truly a multimedia machine that could show color movies with sound and lifelike still images.

When Windows reached about version 3, and Intel was making the 386, the SoundBlaster equipped PC was beginning to approach the Mac in sound capabilities it but still had a long way to go as far as video. The Pentium processor, VGA graphics and Windows 95 nearly closed the gap with the Mac and today's with fast Pentiums, new high definition monitors and blazing fast video cards the PC has caught up with the Mac and outperforms television.

There are a number of terrific software packages that allow you to create multimedia presentations on your computer. Perhaps the best and most widely known is Microsoft's PowerPoint. With PowerPoint a user can mix text with pictures, sound and movies to produce a multimedia slideshow that's great for boardroom presentations or a computer kiosk but difficult to distribute.

Eventually, in the not too distant future, the digital movie imbedded in web pages will become the presentation delivery system of choice relegating PowerPoint to the dustbins of software. If you have ever browsed a DVD movie disk on your computer you've seen that future.

The basic elements of multimedia on a computer are:

• Text • Graphics • Still images • Audio • Movies • Animations • Special Effects

Text, still images and the video portion of movies are functions of your monitor, your video card and the software driver that tells Windows how your video card works. Your monitor is essentially a grid of closely spaced little luminous points called pixels which can be turned on and off like tiny

light bulbs. For the sake of simplicity we'll extend our above example to say that the little bulbs can be lighted with a number of colors. Just how close together those points of light are packed is a function of your monitor. The number of colors that the luminescent points can display is a function of the monitor in concert with the video card. (If you're wondering what a video card is, follow the cable from your monitor to your computer.)

Text When PC's were in their infancy running under MS-DOS they only displayed text in one size and one color. Most of those early monitors were outgrowths of dumb terminals and had green or amber displays on a black background but for dramatic effect it could be reversed. They ran in two modes: Text and graphic. In graphic mode a program had to light up (turn on) each pixel. The screen was 640 pixels wide and 480 high so to clear the screen in graphics mode a program had to turn off 307200 pixels.

Text on those early PC's was displayed using the ASCII charter set which was a series of 2 numbers that could be sent to the monitor. Each of those two digit numbers represented an alpha-numerical character. For example, the ASCII character code for a lower case a is 97 while the uppercase is 65. A text charter was 8 pixels high and 8 pixels wide so when a program sent a character 65 to the screen a helper systems program called ANSI.SYS would send a signal to the proper location turning on and off 64 dots appropriately to make an image of the letter A.

The 8 by 8 matrix made the screen 80 characters wide and 60 charters high. To clear the screen only 4800 characters had to be cleared and only two digits had to be sent for each character. Had the processors been as fast as they are today everything could have been painted in graphics mode but they were slow and charter based programs were the standard.

The first color monitors had multiple size modes which made it possible to display larger text (still using the ASCII system) but the whole screen had to be shifted to the larger text. The first version of Windows was awful, especially when you compared it to the Mac or the Commodore Amiga simply because the PC's ability to display graphics was inferior but as clock speeds and processing power increased so did the quality of Windows. Windows and other graphical operating systems, used a font (a miniature picture) to paint text on the screen in graphical mode. The system is similar to the ASCII concept. Now the program specified the font and the charter.

The full evolution of text was the vector based, true type, proportional font introduced by Adobe which was as smart as a skilled typesetter. It was Adobe's early efforts at creating graphics based fonts that put them into the imaging business.

Still Images As we learned in the last lesson, turning on or off monitor pixels in graphics mode can create an alpha numeric character. Obviously the same process can be also be used to create a picture.

In the above representation of an upper case A and in the below picture of an eye we've used only white and black (on and off).

In the below example we've added two shades of gray, two shade of blue to the original black and white.

There are dozens of computer color mixing systems and an endless number of custom palettes. Essentially however, all computer colors are created by mixing red green and blue. Typically pallets are calculated using 255 parts as the maximum amount of any one color thus 255 parts of of all three colors creates white while no parts of any of the primary colors produces black. Below is an example of some of the palettes available to you.

In the below example we mixed our own custom gold with 255 parts red, 211 parts green and 101 parts blue.

The number of colors that you can display, paint with or print with depends upon the capablity of your video card.

To get true photo-realistic images you need 32 bit capabilities.

Graphics

Another interesting element in multimedia is graphics. As a matter of fact, taking into consideration the human nature, a subject is more explained with some sort of pictorial/graphical representation, rather than as a large chunk of text. This also helps to develop a clean multimedia screen, whereas use of large amount of text in a screen make it dull in presentation.

Unlike text, which uses a universal ASCII format, graphics does not have a single agreed format. They have different format to suit different requirement. Most commonly used format for graphics is .BMP or bitmap pictures. The size of a graphics depends on the resolution it is using. A computer image uses pixel or dots on the screen to form itself. And these dots or pixel, when combined with number of colors and other aspects are called resolution. Resolution of an image or graphics is basically the pixel density and number of colors it uses. And the size of the image depends on its resolution. A standard VGA (Virtual Graphics Arrays) screen can display a screen resolution of 640 ´ 480 = 307200 pixel. And a Super VGA screen can display up-to 1024 ´ 768 = 786432 pixel on the screen. While developing multimedia graphics one should always keep in mind the image resolution and number of colors to be used, as this has a direct relation with the image size. If the image size is bigger, it takes more time to load and also requires higher memory for processing and larger disk-space for storage.

However, different graphics formats are available which take less space and are faster to load into the memory.

There are several graphics packages available to develop excellent images and also to compress them so that they take lesser disk-space but use higher resolution and more colours. Packages like Adobe PhotoShop, Adobe Illustrator, PaintShop Pro etc. are excellent graphics packages. There are Graphics gallery available in CD’s (Compact Disk) with readymade images to suit almost every requirement. These images can directly be incorporated into multimedia development.

Audio

The image below is a map of the above wave file.

Sample Rate - To be clear we should explain that the size of a wave file depends largely to the Sample Rate of the recording. Sample rate is the number of samples of a sound that are taken per second to represent the event digitally. The more samples taken per second, the more accurate the digital representation of the sound can be. For example, the current sample rate for CD-quality audio is 44,100 samples per second. This sample rate can accurately reproduce the audio frequencies up to 20,500 hertz, covering the full range of human hearing.

Sample Size - Sound may be sampled at a size of 8, 12,16 or 32 bits.

Mono and Stereo - Monaural sound has only one track which plays as identical sound through each speaker on a stereo sound system where stereophonic sound has two tracks that play independently from each stereo speaker . Stereo data is interleaved.

The source file above was sampled as an unsigned 8 bit mono file at 11,025 hertz then encoded as a 16 bit mono file at 22,050 hertz. The target specifications were chosen because they are the optimum settings for FM quality monaural sound downloaded over a 28.8 modem. A CD audio recording has a sampling rate of 44100Hz. A Dialogic VOX file can have a rate of 6000Hz or 8000Hz.

Format - The basic format classification of the audio data are designated by sample size groups and method of compression. The PCM format encompasses samples containing binary data of 8, 12, 16, or 32 bits in size. The Telephony format applies to samples are encoded Dialogic ADPCM (VOX), µ-Law, A-Law, or ISDN A-Law. The Text format is for plain text

numbers ranging from -1.0 to 1.0 or -32768 to 32767. Hybrid formats that further compress PCM files such as MPEG-3 and MPEG-4 are often used when sound files are part of a movie or to create a smaller footprint for files that were sampled at a high rate.

Attributes - A sound file's attributes specify the actual nature and organization of the sample. A sound file copied from a CD would be PCM format and the attributes would be "16-bit, stereo, signed". Dialogic VOX files often use Telephony, with "4-bit VOX ADPCM, mono" attributes, but can also use µ-Law or A-Law

Signed and Unsigned Samples - Amiga and Apple systems use signed 8 bit (-128 to 127) or signed 16 bit (-32768 to 32767). Wave and Sound Blaster files for PCs are usually unsigned 8 bit (0 to 255) or signed 16 bit (-32768 to 32767). Generally, all 12 bit and 16 bit samples are signed. Byte swap - When more than one byte is required for each sample, the order in which the bytes are stored can vary from system to system. Systems with Intel processors (0x86 & Pentium PCs) store bytes in a certain order (1, 2, 3, 4, 5, 6...). Systems with Motorola processors (Macs) store the data in a different order (2, 1, 4, 3, 6, 5, ...) where each pair of numbers has been swapped. Sound files created for Macs will not play back properly on PC's and visa versa.

Animation

Animation is a sequential series of still images that create an illusion of motion. In the examples on this page we're working with an animated GIF file with a transparent background. The rabbit example is a slide show where frames are held on the screen for specified periods of time that range from 10/1000 to 100/1000th of a second.

To create the most convincing illusion of motion frames should be played at movie speed of 30 frames per second. Unfortunately, that tends to create a very big file. The skeleton example below holds the frames for 100/1000th of a second producing a 66KB file.

Aside from the obvious problem of disk space, file size can have an impact on playback quality as well. When your computer plays a movie or animation sequence it will load as many frames as possible into memory and then fetch the next frame as it plays one. This technique is called cached sequencing. To create smooth playing animations you must consider the complexity and size of each frame as well as the color depth and number of frames per second. Only the very fastest of modern desktop computers can play back full screen, 24 bit color video at 30 frames per second

Movie

Unlike an animation that we can create from drawings or images a movie is created by a photographic process and converted or ported to a computer so each frame has data stored in every pixel. As we discussed in the animation lesson, a computer's resources can be sorely taxed by dense video and movies have the added element of sound making things even more difficult. You already know the basics of computer generated movies because they are no more than an array of still images

synchronized with a sound file. The synchronization is accomplished using key frames and the playback computer's onboard clock.

Movie size on disk and in memory depends upon the video playback window size, the frame rate (how many frames are played in each second), the audio sample rate and size and the codec used to encode the file.

Below is a table representing a 30 second clip from a movie captured from VHS saved at various sizes with different codecs and settings.

Width

Height

FPS CODEC Color

Depth

Audio

CODEC

Audio Sample

Rate

Audio Sample

Size

Mono Stereo

Total File Size

W=320

H=240

15 Sorenson 32 Bit

Millions

IMA 4:1 22.05 kHz 16 bits mono 2.8 MB

W=240

H=180

15 Sorenson 32 Bit

Millions

IMA 4:1 22.05 kHz 16 bits mono 1.7 MB

W=240

H=180

.53 Sorenson 32 Bit

Millions

Qdesign Music 2

8 kHz 16 bits mono 108 KB

W=160

H=120

15 Sorenson 32 Bit

Millions

Qdesign Music 2

22.05 kHz 16 bits mono 536 KB

W=320

H=240

15 Cinepak 32 Bit

Millions

DVI IMA 22.05 kHz 16bits mono 2.7 MB

W=160

H=120

15 Cinepak 32 Bit

Millions

DVI IMA 22.05 kHz 16 bits mono 1.7 MB

W=320

H=240

15 MPEG-4 32 Bit

Millions

Microsoft

IMA

ADPCM

22.05 kHz 16 bits mono 4.6

MB

W=320

H=240

30 MPEG-4 32 Bit

Millions

Windows

Media

22.05 kHz 16.bits stereo 1.8

MB