Transmisión de Datos Multimedia – – Master IC 2007/2008 Tema 0: Transmisión de Datos Multimedia Clases de aplicaciones.

Transmisión de Datos Multimedia – http://www.grc.upv.es/docencia/tdm – Master IC 2007/2008

Tema 0: Transmisión de Datos MultimediaTema 0: Transmisión de Datos Multimedia

Clases de aplicaciones multimedia Redes basadas en IP y QoS

Computer Networking: A Top Down Approach Featuring the

Internet, 3rd edition.

Jim Kurose, Keith RossAddison-Wesley, July 2004.

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

2

What is multimedia?

Definition of multimedia Hard to find a clear-cut definition In general, multimedia is an integration of text, graphics, still

and moving images, animation, sounds, and any other medium where every type of information can be represented, stored, transmitted and processed digitally

Characteristics of multimedia Digital – key concept Integration of multiple media type, usually including video

or/and audio May be interactive or non-interactive

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

3

Various Media Types

Text, Graphics, image, video, animation, sound, etc. Classifications of various media types

Captured vs. synthesized media Captured media (natural) : information captured from the real

world– Example: still image, video, audio

Synthesized media (artificial) : information synthesize by the computer

– Example: text, graphics, animation

Discrete vs. continuous media Discrete media: space-based, media involve the space dimension

only– Text, Image, Graphics

Continuous media: time-based, media involves both the space and the time dimension

– Video, Sound, Animation

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

4

Classification of Media Type

SoundSound VideoVideo

ImageImage

AnimationAnimation

TextText GraphicsGraphics

Captured From real world

Synthesized By computer

Discrete Discrete

Continuous Continuous

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

5

Text

Plain text Unformatted Characters coded in binary form ASCII code All characters have the same style and font

Rich text Formatted Contains format information besides codes for characters No predominant standards Characters of various size, shape and style, e.g. bold, colorful

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

6

Plain Text vs. Rich Text

An example of Plain text

Example of Rich text

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

7

Graphics

Revisable document that retains structural information Consists of objects such as lines, curves, circles, etc Usually generated by graphic editor of computer programs

-4-2

02

4

-4

-2

0

2

4-10

-5

0

5

10

Example of graphics (FIG file)

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

8

Images

2D matrix consisting of pixels Pixel—smallest element of resolution of the image One pixel is represented by a number of bits Pixel depth– the number of bits available to code the pixel

Have no structural information Two categories: scanned vs. synthesized still image

Computer software

Computer software

Capture and A/D conversionCapture and

A/D conversion

Digital still imageDigital still image

Synthesizedimage

Scannedimage

Camera

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

9

Images (cont.)

Examples of images Binary image – pixel depth 1 Gray-scale – pixel depth 8 Color image – pixel depth 24

Binary image

Gray-scale imagecolor image

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

10

Video vs. Animation

Both images and graphics can be displayed as a succession of view which create an impression of movement

Video – moving images or moving pictures Captured or Synthesized Consists of a series of bitmap images Each image is called a frame Frame rate: the speed to playback the video (frame per

second) Animation – moving graphics

Generated by computer program (animation authoring tools) Consists of a set of objects The movements of the objects are calculated and the view is

updated at playback

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

11

Sound

1-D time-based signal

Speech vs. non-speech sound Speech – supports spoken language and has a semantic

content Non-speech – does not convey semantics in general

Natural vs. structured sound Natural sound – Recorded/generated sound wave

represented as digital signal Example: Audio in CD, WAV files

Structured sound – Synthesize sound in a symbolic way Example: MIDI file

0 100 200 300 400 500 600 700 800 900 1000-0.2

-0.15

-0.1

-0.05

0

0.05

0.1

0.15

0.2

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

12

Networked Multimedia

Local vs. networked multimedia Local: storage and presentation of multimedia information

in standalone computers Sample applications: DVD

Networked: involve transmission and distribution of multimedia information on the network

Sample applications: videoconferencing, web video broadcasting, multimedia Email, etc.

InternetInternetVideo server

Image serverA scenario of multimedia networking

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

13

Consideration of Networked Multimedia

Requirements of multimedia applications on the network Typically delay sensitive

end-to-end delay delay jitter:

– Jitter is the variability of packet delays within the same packet stream

Quality requirement Satisfactory quality of media presentation Synchronization requirement Continuous requirement (no jerky video/audio) Can tolerant some degree of information loss

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

14

Technologies of Multimedia Networking

Challenges of multimedia networking1. Conflict between media size and bandwidth limit of the

network2. Conflict between the user requirement of multimedia

application and the best-effort network3. How to meet different requirements of different users?

Media compression – reduce the data volumeAddress the 1st challenge Image compression Video compression Audio compression

Multimedia transmission technologyAddress the 2nd and 3rd challenges Protocols for real-time transmission Rate / congestion control Error control

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

15

Multimedia Networking Systems

Live media transmission system Capture, compress, and transmit the media on the fly

(example?) Send stored media across the network

Media is pre-compressed and stored at the server. This system delivers the stored media to one or multiple receivers. (example?)

Differences between the two systems For live media delivery:

Real-time media capture, need hardware support Real-time compression– speed is important Compression procedure can be adjusted based on network

conditions For stored media delivery

Offline compression – better compression result is important Compression can not be adjusted during transmission

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

16

Classes of multimedia applications

Streaming stored audio and video Streaming live audio and video Real-time interactive audio and video

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

17

Streaming Stored Multimedia: What is it?

1. videorecorded

2. videosent

3. video received,played out at client

Cum

ula

tive

data

streaming: at this time, client playing out early part of video, while server still sending laterpart of video

networkdelay

time

t>0

100%

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

18

Streaming vs. Download of Stored Multimedia Content

Download: Receive entire content before playback begins High “start-up” delay as media

file can be large~ 4GB for a 2 hour MPEG II

movie Streaming: Play the media file

while it is being received Reasonable “start-up” delaysReception Rate >= playback

rate. Why?

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

19

Streaming Stored Multimedia: Interactivity

VCR-like functionality: client can pause, rewind, FF, push slider bar

•10 sec initial delay OK•1-2 sec until command effect

OK•RTSP often used (more later)

timing constraint for still-to-be transmitted data: in time for playout

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

20

constant bit rate videotransmission

Cum

ula

tive

data

time

variablenetwork

delay

client videoreception

constant bit rate video playout at client

client playoutdelay

bu

ffere

dvid

eo

Streaming Multimedia: Client Buffering

Client-side buffering, playout delay compensate for network-added delay, delay jitter

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

21

Streaming Multimedia: Client Buffering

Client-side buffering, playout delay compensate for network-added delay, delay jitter

bufferedvideo

variable fillrate, x(t)

constant drainrate, d

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

22

Interactive, Real-Time Multimedia

applications: IP telephony, video conference, distributed interactive worlds

end-end delay requirements: audio: < 150 msec good, < 400 msec OK

includes application-level (packetization) and network delays higher delays noticeable, impair interactivity

session initialization how does callee advertise its IP address, port number,

encoding algorithms?

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

23

Internet multimedia: simplest approach

audio, video not streamed: no, “pipelining,” long delays until playout!

audio or video stored in file files transferred as HTTP object

received in entirety at clientthen passed to player

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

24

Progressive Download

browser GETs metafile browser launches player, passing metafile player contacts server server downloads audio/video to player

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

25

Streaming from a streaming server

This architecture allows for non-HTTP protocol between server and media player

Can also use UDP instead of TCP.

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

26

Multimedia Over Today’s Internet

TCP/UDP/IP: “best-effort service” no guarantees on delay, loss

But multimedia apps requires QoS and level of performance to be effective!

Today’s Internet multimedia applications use application-level techniques to mitigate (as best possible) effects of delay, loss

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

27

Streaming Multimedia: UDP or TCP?

UDP server sends at rate appropriate for client (oblivious to

network congestion!) often send rate = encoding rate = constant rate then, fill rate = constant rate - packet loss

short playout delay (2-5 seconds) to compensate for network delay jitter

error recover: time permittingTCP send at maximum possible rate under TCP fill rate fluctuates due to TCP congestion control larger playout delay: smooth TCP delivery rate HTTP/TCP passes more easily through firewalls

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

28

Multimedia, Quality of Service: What is it?

Multimedia applications: network audio and video(“continuous media”)

network provides application with level of performance needed for application to function.

QoS

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

29

Improving QOS in IP Networks

Thus far: “making the best of best effort” Future: next generation Internet with QoS guarantees

RSVP: signaling for resource reservations Differentiated Services: differential guarantees Integrated Services: firm guarantees

simple model for sharing and congestion studies:

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

30

Principles for QOS Guarantees

Example: 1Mbps IPphone, FTP share 1.5 Mbps link. bursts of FTP can congest router, cause audio loss want to give priority to audio over FTP

packet marking needed for router to distinguish between different classes; and new router policy to treat packets accordingly

Principle 1

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

31

Principles for QOS Guarantees (more)

what if applications misbehave (audio sends higher than declared rate) policing: force source adherence to bandwidth allocations

marking and policing at network edge: similar to ATM UNI (User Network Interface)

provide protection (isolation) for one class from others

Principle 2

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

32

Principles for QOS Guarantees (more)

Allocating fixed (non-sharable) bandwidth to flow: inefficient use of bandwidth if flows doesn’t use its allocation

While providing isolation, it is desirable to use resources as efficiently as possible

Principle 3

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

33

Principles for QOS Guarantees (more)

Basic fact of life: can not support traffic demands beyond link capacity

Call Admission: flow declares its needs, network may block call (e.g., busy signal) if it cannot meet needs

Principle 4

Tra

nsm

isió

n d

e D

ato

s M

ult

imedia

- M

ast

er

IC 2

00

7/2

00

8

34

Summary of QoS Principles