Multimedia – Sound Dr. Lina A. Nimri Lebanese University Faculty of Economic Sciences and Business Administration 1 st branch.

Multimedia – Sound

Dr. Lina A. NimriLebanese UniversityFaculty of Economic Sciences and Business Administration 1st branch

What is sound?• Sound is vibrations in the air; that is, a series

of rising and falling pressures in the air, ▫deviating from the average, which is

represented by atmospheric pressure. To prove this, you could place something loud (like an alarm clock) inside a vacuum chamber, and notice that the initially noisy object no longer makes a sound if it isn't surrounded by air anymore.

• The simplest way to create a sound is to make an object vibrate. ▫In this manner, a violin makes a sound when

the bow makes its strikes vibrate, ▫a piano sounds a note when a key is struck,

because a hammer struck a string and made it vibrate.

What is sound?

•This is how sound waves are produced; they can be represented in a diagram as changes in air pressure (or in the electricity level of the magnet) as a function of time.

•This can be represented as:  

Sonogram

•A sonogram, on the other hand, depicts sound frequencies as a function of time.

•It should be noted that a sonogram shows fundamental frequency, on top of which higher frequencies, called harmonics, are superimposed.

•This is what allows us to distinguish between different sources of sound: low notes have low frequencies, while high notes have higher frequencies.

Sonogram

Sound sampling

•To play sound on a computer, it must be converted into a digital format, as this is the only kind of information computers can work with. ▫A computer program intersperses small

samples of the sound (which amount to differences in pressure) at specific intervals of time.

•This is called sampling or digitizing sound.

•The period of time between two samples is called the sampling rate.

Sound sampling

•To reproduce audio which sounds continuous to the ear, it is required to sample at least once every few 100,000ths of a second (10 micro sec)▫it is more practical to go by the number of

samples per second, expressed in Hertz (Hz).

Nyquist – Shannon theorem• Sampling frequency must be high enough to

preserve the form of the signal▫the sampling rate must be equal to or greater than

twice the maximum frequency contained in the signal.

▫Our ears can hear sounds up to about 20,000 Hz. Therefore, for a satisfactory level of sound quality, the sampling rate must be at least on the order of 40,000 Hz.

• There are several standardized sampling rates in use: ▫32 kHz: for digital FM radio (band-limited to 15

kHz) ▫44.1 kHz: for professional audio and compact discs ▫48 kHz: for professional digital multi-track

recording, and consumer recording equipment (like DAT or MiniDisc)

Sound for computers

•Each sample (corresponding to an interval of time) is associated with a value, ▫The sample determines the air pressure

value at that moment.

Sound for computers• A computer works with bits, so the number

of possible values that the sample could have must be determined. ▫This is done by setting the number of bits on

which the sample values are encoded. • With 8-bit coding, there are 28 (= 256)

possible values. • With 16-bit coding, there are 216 (= 65536)

possible values. • The second option clearly offers higher

sound fidelity, but at the cost of using more computer memory.

Sound for computers•Stereo sound requires two channels, with

sound recorded individually on each one. One channel is fed into the left speaker, while the other is broadcast from the right speaker.

• In computer processing, a sound is therefore represented by several parameters: ▫The sampling rate ▫The number of bits in a sample ▫The number of channels (one for mono, two for

stereo, and four for Quadraphonic sound)

Memory size to store a sound file• It is easy to calculate what size an

uncompressed audio sequence will be. ▫By knowing how many bits are used to code

the sample, you know its size (as the sample size is the number of bits)

•To find out the size of a channel, all you need to know is the sample rate, and thus the number of samples per second, and from that the amount of space taken up by one second of music.▫This comes to: Sampling rate x Number of

bits

Memory size to store a sound file•To find out how much space a sound extract

several seconds long will take up, just multiply the preceding value by the number of seconds: ▫Sampling rate x Number of bits x Number of

seconds •Finally, to determine the actual file size of the

extract, the above figure should be multiplied by the number of channels (it will be twice as large for stereo as for mono).

•The size, in bits, of a sound extract is equal to: ▫Sampling rate x Number of bits x Number of

seconds x Number of channels

Audio files

Frequency Quality

48,000 Hz DAT, DVD-audio

44,100 Hz CD

22,000 Hz Radio

8,000 Hz Telephone

CD stereo audio of one hour:44,100 x 16 x 2 x 3600 = 635 Mb

DVD quadraphonic audio of one hour:48,000 x 16 x 4 x 3600 = 1,382 Gb

Psychoacoustics

•Human ear •Audition - The ability to hear comes from

acoustic process, mechanical, hydraulic, nervous and mental from the outer ear to the brain

•Apparatus - The main hearing are: ▫the ear canal (outer ear) ▫ossicular chain (middle ear) ▫the cochlea (inner ear)

Human ear

Human ear

•Cochlea ▫Basilar membrane: mechanical frequency

analyzer ▫Corti Organ: vibration sensor membrane

Perception of sounds

•Frequency Range ▫The human ear, through the basilar

membrane, collects a continuous range: [20 Hz - 20, 000 Hz]

•Sensitive frequencies▫By the effects of filtering and carried

resonance in the ear canal and the ossicles, the most sensitive frequencies are : [1, 000 Hz - 5, 000 Hz]

Perception of sounds

•Fletcher-Munson Curves: ▫Contours measuring

perceived strength (loudness) of sound depending on the frequency The lowest frequency is

the Listening threshold and highest frequency is the threshold of pain

The curves evolve according to age

Perception of sounds

•Bass sounds▫The first “La” of the piano: 27 Hz ▫The perception is disproportionally reduced

Perception of sounds

•Medium sounds▫The standard “La” of the piano : 440 Hz ▫The hearing is very high in this range

Perception of sounds

•Treble Sounds ▫The last “La” of the piano: 3480 Hz ▫The hearing is highly variable and

decreases with age

Masking

•Masking of frequencies▫Two sounds of frequencies very close

together are merged in a single frequency Same time: one sound Closer moments: beating

Masking

•Temporal Masking ▫A high amplitude sound will be perceived

longer and hides the sounds that follow Echo phenomenon Exponentially decreases (10 - 50 ms)

Sound Compression – general method• Division

▫The audio signal is divided into 12 samples per second

• Conversion ▫Each sample is converted into a spectrum of 32

bands of frequencies• Filtering

▫The frequency spectrum is analyzed with the psychoacoustics model Threshold: any frequency below the threshold of

listening is eliminated Masking: any frequency obscured by another is

eliminated • Coding

▫Determine the number of bits needed to quantify the amplitudes while minimizing the quantization noise

Sound Compression – MP3

•MP3 encoding is a lossy method •Level I

▫Uniform Quantization ▫Only the frequency masking is used

•Level II ▫Uniform Quantization ▫Frequency and temporal masking

•Level III ▫Adaptive Quantization ▫Frequency and temporal masking ▫Huffman coding

Sound Compression – MP3•MP3 (“MPEG1 Audio layer 3”) is a lossy

audio data compression format, developed by the International Standardization Organization (ISO).▫This format is used to compress normal

audio formats (WAV or audio CD) at a rate of 1:12.

•MP3 format permits the equivalent of files of twelve CD music albums to take up the same space as one CD-ROM.

•Moreover, mp3 barely alters the sound quality perceptible by human ear.

MP3 - Context• MP3 compression involves removing the data

corresponding to inaudible frequencies by the average person under normal listening conditions.▫This compression analyses the spectrometric

components of an audio signal, and applies a psychoacoustic model to them so as to preserve only "audible" sound.

▫The average human ear is able to recognize sounds between 0.02 kHz and 20 kHz, with sensitivity being at its peak for frequencies between 2 and 5 kHz

• MPEG compression involves determining which sounds go unheard and can be deleted; it is therefore “lossy compression”

MP3 – The masking effect•Gabriel Bouvigne explains :

"When you look at the sun and a bird passes in front of it, you don't see it, because the light from the sun is too bright. Acoustics are like that. When there are loud sounds, you don't hear the quiet sounds. For example: When an organist isn't playing, you can hear whistling in the pipes, and when he is playing, you can't hear it anymore, because it's masked. This is why it isn't necessary to record every sound, and this is the main principle used in the MP3 format to save space.“

Huffman code• The Huffman algorithm is an encoding (not

compression) algorithm, which takes effect at the end of the compression process, by creating variable-length codes over a large number of bits. ▫The codes have the advantage of a unique prefix,

but they may be correctly decoded despite their variable length, and this can be done quickly with the use of tables.

▫This type of encoding saves, on average, a little under 20% of the space taken up.

• When sounds are “pure” (that is, there is no masking), the Huffman algorithm is very effective, as digital audio contains many redundant sounds.

MIDI - Definition• MIDI is a protocol for communication and control between

electronic musical instruments▫ one or more of these "instruments" can be a computer

• An instrument that supports a standard MIDI has a MIDI-IN port and a MIDI-OUT port▫ instruments can be linked in cascade (or daisy-chain)

• Sometimes the device has a MIDI-THRU, which is a direct copy of the MIDI-IN, without the delay caused by the copy on the MIDI-OUT port