Audio & Video Representation CS105. Data Representation Types of data: – Numbers – Text – Images – Audio & Video.

Audio & Video Representation

CS105

Data Representation• Types of data:– Numbers– Text– Images– Audio & Video

What is sound?• A continuous wave created by oscillations of

pressure through any material (solid, liquid & gas)

• For a computer: Analog to Digital– Specialized Hardware

Analog signal

Digitizing SoundVoltage

Time

• Discrete time (sampling) & discrete voltage (quantization)• Reasonable sound production: 40,000 times per sec

• Quantization: Dividing vertical axis into pieces – process of mapping a continuous range of values by a small finite set of values• 8 bit quantization = 256 levels, 16 bit = 65536 levels

Quantization

Sampling

Digitizing Sound

Quantization into levels Sampling time intervals

How many bits are needed to represent per sample?- 3 bits- Power of 2How many bits are needed to digitize this signal?- 3 x 13 = 39 bits

Bit Depth and Bit Rate• Bit Depth: Number of bits per sample• Bit Rate: Number of bits that are sent per unit

of time Each tick on the time axis (x-axis) is 1 sec. All levels are represented by 3 bits. What is the bit rate?- 3 bits/s

Every 4 ticks on the time axis is 1 sec i.e. you sample 4 times a second. What is the bit rate now?- 12 bits/s 1 sec

Audio Formats• Wav, AU, AIFF, VQF and MP3• MP3 (MPEG1 or 2) – dominant format for

compressing audio data– Psychoacoustic models– Form of Huffman encoding– Lossy compression

• Bit rates: – Uncompressed audio: 1411 kbit/s– Compressed audio (such as mp3) : 128 to 320 kbit/s

Practice Digitizing Signals!

Name:

Date:

Time

Voltage

Digitize this graph in the sampling intervals using 2 bits

Level 3

Level 1

Level 2

Level 4

S1 S3 S2 S4 S5 S6

Video• Three dimensional array of color pixels– Succession of images – Each image in the video is called a frame

• Video: Duration, Frame Size with a bit depth, frame rate (fps)– Example video: Duration: 3600s (1 hr), Frame Size:

640W*480H with 24 bits for each pixel, 25 fps• Frame: 0.0009216 Gb, Every sec: 0.023 Gb, • Every hour: ~83 Gb

– Human eye and frame rate: film and video games

Video Codecs• Video files are big!– Typically gigabytes of data for an hour of video

• Codec: Compressor/Decompressor– Employs lossy compression– Temporal & Spatial compression• Temporal compression: Differences between frames

– Keyframe is used to compare differences between frames and encode only the differences

• Spatial compression: Similar regions within frames and groups them– Run length encoding

Container formats• Encapsulates both audio, video and metadata

(needed to synchronize audio and video streams)

• Stores the data, does not encode it– For access: program that opens the container and

decodes the data inside

• AVI, MOV

Video Data• MPEG 1, 2 and 4 lossy compression

– MPEG 1 and 2 – deals with differences in frames– MPEG 4 :

• Better video quality than MPEG 1 and 2• Widely used• Instead of dealing with each individual frame, it tackles objects within

each frame

• Bit rate:– 1.25 Mbit/s – VCD quality (MPEG-1 video compression)– 5 Mbit/s – DVD quality (MPEG-2 compression) – 8 to 15 Mbit/s – HDTV quality (MPEG-4 compression) – 40 Mbit/s max – Blu-ray Disc