Dr.B.Yogameena
Jan 01, 2016
Dr.B.Yogameena
D6GB Multimedia Systems
Faculty: Dr.B.Yogameena ([email protected])
Outline for todayHighlevel introduction to multimedia systems
To discuss…
Course Introduction Multimedia – Definitions Multimedia - Applications Multimedia Data Hypermedia Digital Technology
Text Books: 1. Ze-Nian Li and Mark S. Drew, “Fundamentals of
Multimedia”, Pearson Prentice Hall, October 2003.2. K. Rammohanarao, Z. S. Bolzkovic and D. A.
Milanovic, “Multimedia Communication Systems”, Prentice Hall, May 2002.
3. Yao Wang, Joern Ostermann, and Ya-Qin Zhang, “Video Processing and Communications”, Prentice Hall, 2002.
4. Fred Halsall, “Multimedia Communications: Applications, Networks, Protocols and Standards”, Addison-Wesley, 2001.
Multimedia- ApplicationsMultimedia plays major role in following
areasInstructionBusiness
Advertisements Training materials Presentations Customer support services
Entertainment Interactive Games
Multimedia- Applications
Enabling Technology Accessibility to web based materials Teaching-learning disabled children & adults
Fine Arts & Humanities Museum tours Art exhibitions Presentations of literature
Multimedia- Applications
In Medicine
Source: Cardiac Imaging,YALE centre for advanced cardiac imaging
Multimedia- Applications
In training
Multimedia- Applications
Public awareness campaign
SourceInteractive Multimedia Project Department of food science& nutrition, Colorado State Univ
Example Multimedia ApplicationsVideo teleconferencing, distributed lectures,
telemedicine, tele symphonyWhite board, collaborative document editingAugmented realityDVDs, digital movies, VOIP telephony (Vonage,
Skype) …Networked gamesVideo on demand (from cable TV, satellite etc.),
IPTV (AT&T U-verse)Can you think of more applications?
YouTube.com, founded in Feb 2005Every minute, 10 hours of video is uploaded
Definition: MultimediaMultimedia means the computer
information/data being transferred over the network which is composed of one or more modality where every type of information/data can be represented, stored, transmitted and processed digitally to enrich its content and enhance communication.
Systems operating on multiple modalities: text, audio, images, drawings, animation, video etc.
Sychronizing multiple modalities is important and hard
Data Types in MM system
Text Data
ASCII stands for American Standard Code for Information
Interchange. Computers can only understand numbers, so an
ASCII code is the numerical representation of a character.
This included both unformatted text, comprising strings of characters from a limited character set, and formatted text strings as used for the structuring, access and presentation of electronic documents.
Text Data
Figure 2.1 ASCII table and description (copyright: www.asciitable.com)
Text Data
ASCII uses 7 bits to represent a character. As a result only 127
characters are defined as standard ASCII characters. Characters 128-
255 are called extended ASCII characters.
Figure 2.2 Extended ASCII codes (copyright: www.asciitable.com)
Text Data
EBCDIC
EBCDIC (Extended Binary Coded Decimal Interchange
Code) is a character set used on early IBM computers.
EBCDIC was first introduced in 1965, it was the new
character-coding scheme came with IBM System 360 series.
EBCDIC uses 8 bits to represent a character.
Text Data
UNICODE
The Unicode character uses 16 bits to represent a character, thus more than 65000 characters can be represented. While 65000 characters are sufficient for encoding most of the many thousands of characters used in major languages of the world.
Sound Data
Sound Data
A typical compact disc can hold up to 74 minutes of 16 bit,
44.1 kHz audio that is uncompressed – about 650
megabytes.
Sound Data
Table shows how the size of a file is affected by the sampling rate and
bit length. The file is a one-minute sound clip, recorded and saved in
various forms in the Microsoft Windows WAV file format.
Quality Sampling Rate Resolution File Size
CD 44 kHz 16 bit Stereo 10.3 MB
44 kHz 8 bit Stereo 5.18 MB
FM Radio 22 kHz 16 bit Stereo 5.18 MB
22 kHz 8 bit Stereo 2.59 MB
AM Radio 11 kHz 16 bit Stereo 2.59 MB
11 kHz 8 bit Stereo 1.29 MB
Table 2.1 Variation of file size and sampling rate (60 seconds audio clip in MS WAV format)
Image Data
Image Data
Images, or pictures, are two-dimensional arrays of data
called bitmaps, with each element is called pixel.
Image Data
UnitsDpi - Dots Per InchBit Depth - The number of bits used to hold a pixel. Also
called color depth and pixel depth, the bit depth determines
the number of colors that can be displayed at one time.
Color Depth Number of Colors
4 bits 16
8 bits 256
16 bits 65,536
24 bits 16,777,216
Table Color Depths
Video Data
Video Data
Video, or moving images, is a sequence of images.
To create a sense of continuity, video must be played at a
rate of at least 25 frames per second (fps).
MultimediaMultimedia Hardware Hardware PeripheralsPeripheralsInput devices
Output devices
Storage devices
Communication devices _Modems
_Network Interfaces
Input DevicesInput DevicesKeyboards And MiceScanners And Digital camerasMIDI Keyboards Touch screens Trackballs Tablets
Continue…Continue…Voice recognition systems
Infrared remotes
Continue…Continue…Magnetic Card Encoders And Readers
Video cameras
Output DevicesOutput DevicesMonitorsSpeakersVR helmet and VR immersive displayVideo Devices
Storage DeviceStorage DeviceSyquest drives
Continue…Continue…CD-ROM Drives
Magneto-optical drives
Laserdisc Player
To discuss…
Course Introduction Multimedia – Definitions Multimedia - Applications Multimedia Data Hypermedia Digital Technology
Hypermedia and MultimediaA hypertext system: meant to be read
nonlinearly, by following links that point to other parts of the document, or to other documents
• HyperMedia: not constrained to be text-based, can include other media, e.g., graphics, images, and especially the continuous media – sound and video. The World Wide Web (WWW) — the best
example of a popular hypermedia application.
f(x,y) = reflectance(x,y) * illumination(x,y)Reflectance in [0,1], illumination in [0,inf]
Original 8-bit image,256 gray levels
Quantized to 6 bits ,64 gray levels
Quantized to 3 bits ,8 gray levels
Quantized to 1 bits ,2 gray levels
Quantization-False Contouring
Problem: Limited bandwidthNeed for compressionAudio
CD quality: 44100 samples per seconds with 16 bits per sample, stereo sound
44100*16*2 = 1.411 MbpsFor a 3-minute song: 1.441 * 180 = 254 Mb =
31.75 MBVideo
For 320*240 images with 24-bit colors320*240*24 = 230KB/image15 frames/sec: 15*230KB = 3.456MB3 minutes of video: 3.456*180 = 622MB
Discrete Cosine Transform
DCT converts the information from spatial domain to frequency domain
Consider an unsorted list of 12 numbers between 0 and 3 -> (2, 3, 1, 2, 2, 0, 1, 1, 0, 1, 0, 0). Consider a transformation of the list involving two steps
sort the list Count the frequency of occurrence of each of
the numbers (4,4,3,1 ) spatial info lost, captured freq. info
Discrete Cosine Transform (DCT)Discrete Cosine Transform (DCT)
This is DCT
DCT is an orthogonal transformm so its inverse DCT is an orthogonal transformm so its inverse kernel is the same as forward kernelkernel is the same as forward kernel
This is inverse DCT
100
1
00
1
001
bygiven ,matrix al tridiagonsymmetric the
of orseigen vect theare DCT theof vectorsbasis 3.The
2-MEPG 1,-MPEG JPEG,
coding transformcompactionenergy excellent .2
.orthogonal and real is DCT 1.The
DCT theof Properties
1
c
c
Q
Q
tCCCC
Properties of DCT: real, Properties of DCT: real, orthogonal, energy-orthogonal, energy-
compacting, compacting, eigenvector-eigenvector-basedbased
• Each 8x8 block can be looked at as a weighted sum of these basis functions. • The process of 2D DCT is also the process of finding those weights.
DCT: Why does it do this?DCT takes advantage of redundancies in the
data by grouping pixels with similar frequencies togetherHigher frequencies = lower numberLower frequencies = higher number
If lossy compression is acceptable, then each data unit can then be divided by quantization coefficient (QC)
Zig Zag Scan
……..
8X8
1X64
• To group low frequency coefficients in top of the vector
• Maps 8 x 8 to a 1 x 64 vector.
DCT (cont)
DPCM on DC Components
The DC component value in each 8x8 block is large and varies across blocks, but is often close to that in the previous block.
Differential Pulse Code Modulation (DPCM): Encode the difference between the current and previous 8x8 block.
RLE on AC ComponentsThe 1x64 vectors have a lot of zeros in
them, more so towards the end of the vector
Encode a series of 0s as a (skip,value) pair, where skip is the number of zeros and value is the next non-zero componentSend (0,0) as end-of-block value
using −415 (the DC coefficient) and rounding to the nearest integer
DCT
Subtracting -128
quantization
8X8 block of pixel values taken from original image
(16 is the value of the first pixel from quantization matrix)
−26 −3 0 −3 −2 −6 2 −4 1 −4 1 1 5 1 2 −1 1 −1 2 0 0 0 0 0 −1 −1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Zig-zag scan
−26 −3 0 −3 −2 −6 2 −4 1 −4 1 1 5 1 2 −1 1 −1 2 0 0 0 0 0 −1 −1 EOB
Huffman coding