Top Banner
Chapter 10 Chapter 10 Video Video Multimedia Systems Multimedia Systems
67
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Chapter10

Chapter 10Chapter 10VideoVideo

Multimedia SystemsMultimedia Systems

Page 2: Chapter10

Key PointsKey Points The display of moving pictures depends on The display of moving pictures depends on

persistence of visionpersistence of vision. . Uncompressed video requires 26MBytes Uncompressed video requires 26MBytes

per sec (NTSC) or 31MBytes per sec (PAL). per sec (NTSC) or 31MBytes per sec (PAL). Digitization may be performed in the Digitization may be performed in the

camera (e.g. camera (e.g. DVDV) or using a ) or using a capture cardcapture card attached to a computer. attached to a computer.

NTSCNTSC, , PALPAL and and SECAMSECAM are analogue are analogue video standards. All three use video standards. All three use interlaced interlaced fieldsfields. .

Page 3: Chapter10

Key PointsKey Points CCIR 601CCIR 601 is a standard for digital video. It uses is a standard for digital video. It uses Y'CBCRY'CBCR

colour with 4:2:2 colour with 4:2:2 chrominance sub-samplingchrominance sub-sampling. The da. The data rate is 166Mbits per sec. ta rate is 166Mbits per sec.

Video compression can make use of Video compression can make use of spatialspatial (intra-fra (intra-frame) and me) and temporaltemporal (inter-frame) compression. Spatial (inter-frame) compression. Spatial compression is still-image compression applied to indcompression is still-image compression applied to individual frames. Temporal compression is based on ividual frames. Temporal compression is based on fraframe differencesme differences and and key frameskey frames. .

Motion JPEGMotion JPEG applies JPEG compression to each fram applies JPEG compression to each frame. It is usually performed in hardware. e. It is usually performed in hardware.

CinepakCinepak, , Intel IndeoIntel Indeo and and SorensonSorenson are popular softw are popular software are codecscodecs used in multimedia. They are based on used in multimedia. They are based on vecvector quantizationtor quantization. .

Page 4: Chapter10

Key PointsKey Points MPEGMPEG video is an elaborate codec that combin video is an elaborate codec that combin

es DCT-based compression of key frames (es DCT-based compression of key frames (I-piI-picturesctures) with forward and backward prediction ) with forward and backward prediction of intermediate frames (of intermediate frames (P-picturesP-pictures and and B-pictB-picturesures) using ) using motion compensationmotion compensation. .

QuickTimeQuickTime is a component-based multimedia is a component-based multimedia architecture providing cross-platform support architecture providing cross-platform support for video, and incorporating many codecs. It hfor video, and incorporating many codecs. It has its own file format that is widely used for disas its own file format that is widely used for distributing video in multimedia. tributing video in multimedia.

Digital video Digital video editingediting is is non-linearnon-linear (like film ed (like film editing). iting).

Page 5: Chapter10

Key PointsKey Points Most digital Most digital post-productionpost-production tasks are applic tasks are applic

ations of image manipulation operations to thations of image manipulation operations to the individual frames of a clip. e individual frames of a clip.

For delivery using current technology, it may bFor delivery using current technology, it may be necessary to sacrifice frame size, frame rate, e necessary to sacrifice frame size, frame rate, colour depth, and image quality. colour depth, and image quality.

StreamedStreamed video is played as soon as it arrives video is played as soon as it arrives without being stored on disk, so it allows for liwithout being stored on disk, so it allows for live transmission and `video on demand'. ve transmission and `video on demand'.

Page 6: Chapter10

Moving PicturesMoving Pictures All current moving pictures depend on All current moving pictures depend on

the following phenomenathe following phenomena– Persistence of visionPersistence of vision

A lag in the eye's response results 'after-A lag in the eye's response results 'after-images'images'

– Fusion frequencyFusion frequency If a sequence of still images is presented If a sequence of still images is presented

above this frequency, we will experience a above this frequency, we will experience a continuous visual sensationcontinuous visual sensation

Depend on brightness of image relative to Depend on brightness of image relative to viewing environmentviewing environment

Below this frequency will perceived flickering Below this frequency will perceived flickering effecteffect

Page 7: Chapter10

Generate Moving PicturesGenerate Moving Pictures VideoVideo

– Use video camera to capture a sequence Use video camera to capture a sequence of framesof frames

AnimationAnimation– Generate each frame individually either Generate each frame individually either

by computer or by other meansby computer or by other means

Page 8: Chapter10

Digital VideoDigital Video

A video sequence consists of a A video sequence consists of a number of framesnumber of frames

Each frame is a single image Each frame is a single image produced by digitizing time-varying produced by digitizing time-varying signal generated by video camerasignal generated by video camera

Page 9: Chapter10

Digital VideoDigital Video Think about the size of the uncompresseThink about the size of the uncompresse

d digital videod digital video– NTSC video formatNTSC video format

Bitmapped images for video frameBitmapped images for video frame– 640 640 480 pixels with 24-bit color = 0.9 MB/frame 480 pixels with 24-bit color = 0.9 MB/frame

30 frames per second30 frames per second– 900 kb/frame 900 kb/frame 30 frames/sec = 26 MB/sec 30 frames/sec = 26 MB/sec

60 seconds per minute60 seconds per minute– 26 MB/sec 26 MB/sec 60 secs/minute = 1,600 MB/minute 60 secs/minute = 1,600 MB/minute

Strains on current processing, storage and data transmission !

Page 10: Chapter10

Create Digital VideoCreate Digital Video Get analog/digital video signal fromGet analog/digital video signal from

– video cameravideo camera– video tape recorder (VTR)video tape recorder (VTR)– broadcast signalbroadcast signal

Digitize analog video & compress itDigitize analog video & compress it

Page 11: Chapter10

Digitizing Analog VideoDigitizing Analog Video In computerIn computer

– Video capture cardVideo capture card Convert analog to digital & compressConvert analog to digital & compress Can also decompress & convert digital to Can also decompress & convert digital to

analoganalog

– Compress throughCompress through Video capture card (hardware codec)Video capture card (hardware codec) Software (software codec)Software (software codec)

Page 12: Chapter10

Digitizing Analog VideoDigitizing Analog Video In cameraIn camera

– Digitize and compress using circuitry Digitize and compress using circuitry inside camerainside camera

– Transfer digitized signal from camera to Transfer digitized signal from camera to computer throughcomputer through IEEE 1394IEEE 1394 interface ( interface (FireWireFireWire): 400 Mb/sec): 400 Mb/sec USBUSB: 12Mb/sec(version 1.1) ~ 480 : 12Mb/sec(version 1.1) ~ 480

Mb/sec(version 2.0)Mb/sec(version 2.0)

Page 13: Chapter10

Digitize in Computer v.s. CameraDigitize in Computer v.s. Camera

Digitize in cameraDigitize in camera– AdvantageAdvantage

Digital signals are resistant to corruption Digital signals are resistant to corruption when transmitted down cables and stored when transmitted down cables and stored on tapeon tape

– DisadvantageDisadvantage User has no control between picture User has no control between picture

quality and data rate (file size)quality and data rate (file size)

Page 14: Chapter10

Analog Video StandardAnalog Video Standard

Frame/FieldFrame/Field

Rate Rate (per sec)(per sec)

ScaScann

LineLiness

AspeAspectct

RatioRatio

Horizontal/Horizontal/VerticalVertical

frequencyfrequency

WorldwideWorldwide

StandardStandard

NTSCNTSC 29.97 / 59.9429.97 / 59.94525525

(480(480))

4 : 34 : 3 15.734 kHZ / 60 15.734 kHZ / 60 HZHZ

US, Japan,US, Japan,

Taiwan, …Taiwan, …

PALPAL 25 / 5025 / 50625625

(576(576))

4 : 34 : 3 15.652 kHZ / 50 15.652 kHZ / 50 HZHZ

Western Western Europe,Europe,

……

SECASECAMM

25 / 5025 / 50625625

(576(576))

4 : 34 : 3 15.652 kHZ / 50 15.652 kHZ / 50 HZHZ

France, …France, …

Page 15: Chapter10

Display Video on TVDisplay Video on TV

Cross-section of CRT Delta-delta shadow-mask CRT

(Scan From “Computer Graphics: Principles and Practice”)

Page 16: Chapter10

Field and InterlaceField and Interlace Transmitting many entire pictures in a Transmitting many entire pictures in a

second requires a lot of bandwidthsecond requires a lot of bandwidth FieldField

– Divide each frame into two fieldsDivide each frame into two fields One consisting of the odd-numbered lines of each One consisting of the odd-numbered lines of each

frame, the other of the even linesframe, the other of the even lines

InterlaceInterlace– Each frame is built up by interlacing the Each frame is built up by interlacing the

fieldsfields PALPAL

– 50 fields/sec => 25 frames/sec50 fields/sec => 25 frames/sec NTSCNTSC

– 59.94 fields/sec => 29.97 frames/sec59.94 fields/sec => 29.97 frames/sec

Page 17: Chapter10

Display Video on ComputerDisplay Video on Computer Progressive scanningProgressive scanning

– Write all lines of each frame to frame Write all lines of each frame to frame bufferbuffer

– Refresh whole screen from frame buffer Refresh whole screen from frame buffer at high rateat high rate

Page 18: Chapter10

Display VideoDisplay VideoTV

Computer Monitor

frame i frame i+1 frame i+2 frame i+3 …

Page 19: Chapter10

Field & Interlace ArtifactsField & Interlace Artifacts

Odd lines of frame i Even lines of frame i+1Combine previous two

analog video forprogressive display

A video clip of flash light on the water surface

Page 20: Chapter10

Field & Interlace ArtifactsField & Interlace Artifacts

Page 21: Chapter10

Prevent Interlace ArtifactsPrevent Interlace Artifacts Average two field to construct a single Average two field to construct a single

frameframe

Discard half fields and interpolate Discard half fields and interpolate remained fields to construct a full frameremained fields to construct a full frame

Convert each field into a single frame Convert each field into a single frame (reduce frame rate but much better !)(reduce frame rate but much better !)

Page 22: Chapter10

Types of Analog VideoTypes of Analog Video Component videoComponent video

– Three components: Y (luminance), U and V Three components: Y (luminance), U and V (color)(color)

– Often use in production and post-productionOften use in production and post-production Composite videoComposite video

– Combine three components into a signalCombine three components into a signal– Color component (U and V) is allocated half Color component (U and V) is allocated half

bandwidth as the luminance (Y)bandwidth as the luminance (Y)– Often use in transmissionOften use in transmission

S-videoS-video– Separates the luminance from the two color Separates the luminance from the two color

(total two signals)(total two signals)

Page 23: Chapter10

Digital Video StandardsDigital Video Standards CCIR 601CCIR 601 ( (Rec. ITU-R BT.601Rec. ITU-R BT.601))

– specifies the image format, and coding specifies the image format, and coding for digital television signalsfor digital television signals

Parameter Value

YUV encoding 4:2:2

Sampling frequency for Y (MHz) 13.5

Sampling frequency for U and V (MHz) 6.75

No of samples per line 720

No of levels for Y component 220

No of levels for U,V components 225

Page 24: Chapter10

PerplexingPerplexingNTSC System

1 2 3 …

480

1 2 3 … 640

Analog to digitalPixels are square

1 2 3 …480

1 2 3 … 720

CCIR 601 standardPixels are not square

Page 25: Chapter10

CCIR 601 SamplingCCIR 601 Sampling

Y samples

CB and CR samples

4 : 2 : 2 sampling(co-site)

4 : 2 : 0 sampling(not co-site)

4 : 1 : 1 sampling(co-site)

Page 26: Chapter10

Compression & Data Stream Compression & Data Stream StandardsStandards

Sampling produces a digital Sampling produces a digital representation of a video signalrepresentation of a video signal

This must be compressed and then This must be compressed and then formed into a data stream for formed into a data stream for transmissiontransmission

Further standards are needed to Further standards are needed to specify the compression algorithm specify the compression algorithm and the format of the data streamand the format of the data stream

Page 27: Chapter10

Compression & Data Stream Compression & Data Stream StandardsStandards

DV standardDV standard– For semi-professional & news-gatheringFor semi-professional & news-gathering

MPEG-2 standardMPEG-2 standard– For family useFor family use– Organized into different Organized into different profilesprofiles and and levelslevels

The most combination is The most combination is Main Profile at Main Main Profile at Main Level Level ((MP@MLMP@ML))

Used for digital television broadcasts & DVD Used for digital television broadcasts & DVD videovideo

Page 28: Chapter10

Introduction to Video Introduction to Video CompressionCompression

Adapted to consumers’ hardware, Adapted to consumers’ hardware, video data needs to be compressed video data needs to be compressed twicetwice– First during captureFirst during capture– Then again when it is prepared for Then again when it is prepared for

distributiondistribution

Page 29: Chapter10

Video CompressionVideo Compression Digital video compression algorithms Digital video compression algorithms

operate on a sequence of bit-mapped operate on a sequence of bit-mapped imagesimages– Spatial compressionSpatial compression ( (intra-frameintra-frame))

Compress each individual image in isolationCompress each individual image in isolation

– Temporal compressionTemporal compression ( (inter-frameinter-frame)) Store the differences between sub-Store the differences between sub-

sequences of framessequences of frames

Page 30: Chapter10

Spatial CompressionSpatial Compression Compress method is similar to image Compress method is similar to image

compressioncompression– LosslessLossless

No information lossNo information loss Compression ratios is lowerCompression ratios is lower

– LossyLossy Some information lossSome information loss Compression ratios is higherCompression ratios is higher

Why recompressing video is Why recompressing video is unavoidableunavoidable– The compressor used for capture are not The compressor used for capture are not

suitable for multimedia deliverysuitable for multimedia delivery– For post-productionFor post-production

Page 31: Chapter10

Temporal CompressionTemporal Compression Key framesKey frames

– Certain frames in a sequence are Certain frames in a sequence are designated as key framesdesignated as key frames

Difference frameDifference frame– Each of the frames between the key Each of the frames between the key

frames is replaced by a difference frameframes is replaced by a difference frame– Records only the differences between Records only the differences between

the frames the frames

Page 32: Chapter10

Time Required for Compression & Time Required for Compression & DecompressionDecompression

SymmetricalSymmetrical– Compression & decompression of a Compression & decompression of a

piece of video take the same timepiece of video take the same time

AsymmetricalAsymmetrical– Compression & decompression of a Compression & decompression of a

piece of video piece of video notnot take the same time take the same time– Generally Compression takes longer Generally Compression takes longer

timetime

Page 33: Chapter10

Motion JPEG (MJPEG)Motion JPEG (MJPEG) A popular approach to compressing A popular approach to compressing

video during capturevideo during capture

Applying JPEG compression to each Applying JPEG compression to each frame (No temporal compression)frame (No temporal compression)

Therefore it is called “Motion JPEG”Therefore it is called “Motion JPEG”

Page 34: Chapter10

DVDV Compression based on DCT Compression based on DCT

transformtransform

Perform temporal compression Perform temporal compression (motion compensation) between two (motion compensation) between two fields of each framefields of each frame

Quality is varied dynamically to Quality is varied dynamically to maintain constant data ratemaintain constant data rate

Page 35: Chapter10

MJPEG v.s DVMJPEG v.s DV

Date Date RatesRates

CompressionCompression

methodmethodCompressionCompression

ratioratio

MJPEGMJPEG 24 Mb/sec24 Mb/sec JPEG compressionJPEG compression 7:17:1

(mid-range capture (mid-range capture card)card)

DVDV 25 Mb/sec25 Mb/sec DCT based DCT based compressioncompression

Motion Motion compensationcompensation

5:15:1

(4:1:1 sample)(4:1:1 sample)

Page 36: Chapter10

Software Codecs for Software Codecs for MultimediaMultimedia

Popular software codecsPopular software codecs– MPEG-1MPEG-1– CinepakCinepak– Intel IndeoIntel Indeo– SorensonSorenson

Page 37: Chapter10

Vector QuantizationVector QuantizationSourceOutput

Groupinto

vectors

encoder decoder

Unblock

Reconstruction

Find closestcode-vector

code book index code bookindex

Tablelookup

Page 38: Chapter10

MPEGMPEG Stand for Stand for MMotion otion PPicture icture EExperts xperts

GGroup roup ((Joint of the ISO and the IEC)Joint of the ISO and the IEC)

Works on standards for the coding of Works on standards for the coding of moving pictures and associated moving pictures and associated audioaudio

Page 39: Chapter10

MPEG FamilyMPEG Family MPEG – 1MPEG – 1

– Coding of moving pictures and associated Coding of moving pictures and associated audio for digital storage media at up to about audio for digital storage media at up to about 1.5 Mb/s1.5 Mb/s

MPEG – 2MPEG – 2– Generic coding of moving pictures and Generic coding of moving pictures and

associated audioassociated audio– For broadcasting & studio workFor broadcasting & studio work

MPEG – 3MPEG – 3– no longer exists (has been merged into MPEG-no longer exists (has been merged into MPEG-

2)2) MPEG – 4MPEG – 4

– Very low bit rate audio-visual (integrated Very low bit rate audio-visual (integrated multimedia) codingmultimedia) coding

Page 40: Chapter10

MPEG FamilyMPEG Family MPEG – 7MPEG – 7

– Multimedia content description interfaceMultimedia content description interface MPEG – 21MPEG – 21

– Vision statementVision statement To enable transparent & augmented use of To enable transparent & augmented use of

multimedia resources across a wide range of multimedia resources across a wide range of networks and devicesnetworks and devices

– ObjectivesObjectives To understand how the elements fit togetherTo understand how the elements fit together To identify new standards which are required if To identify new standards which are required if

gaps in the infrastructure existgaps in the infrastructure exist To accomplish the integration of different To accomplish the integration of different

standardsstandards

Page 41: Chapter10

MPEG–1 StandardMPEG–1 Standard Defines a data stream syntax and a decoDefines a data stream syntax and a deco

mpressor, allowing manufacturers to dempressor, allowing manufacturers to develop different compressorsvelop different compressors

MPEG-1 compressionMPEG-1 compression– Temporal compression based on motion coTemporal compression based on motion co

mpensationmpensation– Spatial compression based on quantization Spatial compression based on quantization

& coding of frequency coefficients produced & coding of frequency coefficients produced by a DCT of the databy a DCT of the data

Page 42: Chapter10

MPEG –1 ObjectiveMPEG –1 Objective Medium quality video (VHS-like)Medium quality video (VHS-like) Bit rate < 1.5 Mb/sBit rate < 1.5 Mb/s

– 1.15 Mb/s for video1.15 Mb/s for video– 350 kb/s for audio & additional data350 kb/s for audio & additional data

Asymmetrical applicationAsymmetrical application– Store video & audio on CD-ROMStore video & audio on CD-ROM

Picture format : SIF (Source Input Picture format : SIF (Source Input Format)Format)– 4:2:0 sub-sampled4:2:0 sub-sampled– Frame size @ frequency rateFrame size @ frequency rate

352 352 288 @ 25 HZ 288 @ 25 HZ 352 352 240 @ 30 HZ 240 @ 30 HZ

Page 43: Chapter10

An object moving between frames

Area of potential change

Page 44: Chapter10

Motion CompensationMotion Compensation Divide each frame into macroblocks of 16 Divide each frame into macroblocks of 16

16 pixels 16 pixels Predict where the corresponding Predict where the corresponding

macroblock in next framemacroblock in next frame– Try all possible displacements within a limited Try all possible displacements within a limited

rangerange– Choose the best matchChoose the best match

Construct difference frame by subtracting Construct difference frame by subtracting each macroblock from its predicted each macroblock from its predicted counterpartcounterpart– Keep the motion vectors describing the Keep the motion vectors describing the

predicted displacement of macroblocks predicted displacement of macroblocks between framesbetween frames

Page 45: Chapter10

Picture TypePicture Type I (intra) picturesI (intra) pictures

– Code without reference to other picturesCode without reference to other pictures– Low compression rateLow compression rate

P (predicted) picturesP (predicted) pictures– Code using motion compensated prediction Code using motion compensated prediction

from a past I or P picturefrom a past I or P picture– Higher compression rate than I pictureHigher compression rate than I picture

B (bidirectional-predicted) picturesB (bidirectional-predicted) pictures– Code bidirectional interpolation between the Code bidirectional interpolation between the

I or P picture which preceded & followed I or P picture which preceded & followed themthem

– Highest compression rateHighest compression rateAll are compressed using the MPEG version of JPEG compression

Page 46: Chapter10

I

01

I

B

P I P

B B B B B B B

02 03 04 05 06 11 12 13 14 15 16 21

Group of Pictures (GOP)

An MPEG sequence in display order

01

I

B

P I P I

B BB BB BB

04 02 03 11 05 06 14 12 13 21 15 16

An MPEG sequence in bitstream order (decode order)

Page 47: Chapter10

MPEG-1 MPEG-1 視頻壓縮技術視頻壓縮技術 運動補償 運動補償 (Motion Compensation)(Motion Compensation) 頻率變換 頻率變換 (Frequency Transform)(Frequency Transform) 可變長度編碼 可變長度編碼 (Variable Length Coding)(Variable Length Coding) 彩色信號 彩色信號 subsamplingsubsampling 量化 量化 (Quantization)(Quantization) 預測編碼預測編碼 圖像插值圖像插值

Page 48: Chapter10

QuickTimeQuickTime Apple, 1991Apple, 1991 Time base, non-linear editingTime base, non-linear editing Component-based architectureComponent-based architecture

– Compressor componentsCompressor components Cinepak, Intel Indeo codecCinepak, Intel Indeo codec

– Sequence grabber components Sequence grabber components – Movie control componentMovie control component– TranscoderTranscoder

Translate data between different formatsTranslate data between different formats– Video digitizer componentVideo digitizer component

Support MPEG-1, DV, OMF, AVI, OpenDMLSupport MPEG-1, DV, OMF, AVI, OpenDML

Page 49: Chapter10

Digital Video Editing & Post-Digital Video Editing & Post-productionproduction

EditingEditing CompositingCompositing Reverse shotReverse shot

– Conversation between two peopleConversation between two people

Page 50: Chapter10

Film & Video EditingFilm & Video Editing Traditional Traditional In point and out pointIn point and out point TimecodeTimecode

– SMPTE timecodeSMPTE timecode– Hours, minutes, seconds, framesHours, minutes, seconds, frames

VHS VHS – Two copying operations is to produce serioTwo copying operations is to produce serio

us loss of qualityus loss of quality– Constructed linearlyConstructed linearly

Page 51: Chapter10

Digital Video EditingDigital Video Editing Random accessRandom access Non-destructiveNon-destructive PremierePremiere

– Three main windowsThree main windows Project, timeline, monitorProject, timeline, monitor Figs. 10.12-14Figs. 10.12-14

– TimelinesTimelines Have several video tracksHave several video tracks

– Transitions, Fig. 10.15Transitions, Fig. 10.15– Cuts and TransitionsCuts and Transitions

In a cut, two clips are buttedIn a cut, two clips are butted In transitions, two clips overlapIn transitions, two clips overlap

– Image processing is required to construct transitional Image processing is required to construct transitional framesframes

Page 52: Chapter10
Page 53: Chapter10

Digital Video Post-Digital Video Post-productionproduction

Over- or under-exposed, out of focus, color Over- or under-exposed, out of focus, color cast, digital artifactscast, digital artifacts– Provide image manipulation programsProvide image manipulation programs

Adjust level, sharpen, blurAdjust level, sharpen, blur

– The same correction may be needed for every The same correction may be needed for every frame, so the levels can be set for the first frame frame, so the levels can be set for the first frame and the adjustment will be applied to as many and the adjustment will be applied to as many frames as user specifies.frames as user specifies.

– If light fades during a sequence, it will necessary If light fades during a sequence, it will necessary to increase the brightness gradually to to increase the brightness gradually to compensate.compensate. Apply a suitable correction to each frame and allow Apply a suitable correction to each frame and allow

their values at intermediate frames to be interpolatedtheir values at intermediate frames to be interpolated Varying parameter values over timeVarying parameter values over time

Page 54: Chapter10

KeyingKeying Selecting transparent areasSelecting transparent areas Blue screeningBlue screening

– Chroma keying: any colorChroma keying: any color– Alpha channelAlpha channel– Luma keying: a brightness threshold is used to deteLuma keying: a brightness threshold is used to dete

rmine which areas are transparentrmine which areas are transparent Select explicitlySelect explicitly

– Create mask Create mask – In film and video, mask is called matteIn film and video, mask is called matte– Matte out: removing unwanted elementsMatte out: removing unwanted elements– Split-screen effectsSplit-screen effects

Alpha channel created in other applicationAlpha channel created in other application

Page 55: Chapter10

Track matteTrack matte Chroma keying and luma keyingChroma keying and luma keying

– Color and brightness changes between framColor and brightness changes between frameses

– Use a sequence of masks as matte Use a sequence of masks as matte Separate video track: track matteSeparate video track: track matte

– Track matteTrack matte Painstaking by handPainstaking by hand Generated from a single still imageGenerated from a single still image

applying simple geometrical transformations ovapplying simple geometrical transformations over time to create a varying sequence of matteser time to create a varying sequence of mattes

Page 56: Chapter10

Adobe After EffectsAdobe After Effects Apply a filter to a clip and vary it Apply a filter to a clip and vary it

over timeover time A wide range of controls for the A wide range of controls for the

filter’s parametersfilter’s parameters Premiere: parameter values are Premiere: parameter values are

interpolated linearly between key interpolated linearly between key framesframes

After effect: interpolation can use After effect: interpolation can use Bezier curves Bezier curves

Page 57: Chapter10

Preparing Video for Multimedia Preparing Video for Multimedia DeliveryDelivery

Frame size, frame rate, color depth, image quaFrame size, frame rate, color depth, image qualitylity

People sit close to monitors, so a large picture People sit close to monitors, so a large picture is not necessaryis not necessary

Higher frame rates are needed to eliminate flicHigher frame rates are needed to eliminate flicker only if display is refreshed at the same rate.ker only if display is refreshed at the same rate.– Computer monitors are refreshed at a much higher Computer monitors are refreshed at a much higher

rate from VRAM.rate from VRAM. Limiting colorsLimiting colors

– Not all codecs support Not all codecs support

Page 58: Chapter10

Streamed Video & Video Streamed Video & Video ConferenceConference

Streamed videoStreamed video– Delivering video data stream from a Delivering video data stream from a

remote server, to be displayed as it remote server, to be displayed as it arrivesarrives

– As against downloading an entire files to As against downloading an entire files to disk & playing it from theredisk & playing it from there

– Opens up the possibility of delivering Opens up the possibility of delivering live video on computerslive video on computers

Page 59: Chapter10

Streamed Video & Video Streamed Video & Video ConferenceConference

Video conferenceVideo conference– Streamed video doesn't restricted to a Streamed video doesn't restricted to a

single transmitter broadcasting to many single transmitter broadcasting to many consumers: Any suitably equipped consumers: Any suitably equipped computer can act both as receiver & computer can act both as receiver & transmittertransmitter

– Users on several machines can Users on several machines can communicate visually, taking part in communicate visually, taking part in what is usually called a what is usually called a video video conferenceconference

Page 60: Chapter10

Single transmitter Multiple receiver

All computer are receiver & transmitter

Page 61: Chapter10

Obstacle to Streamed VideoObstacle to Streamed Video BandwidthBandwidth

– SIF MPEG-1 video require a bandwidth of SIF MPEG-1 video require a bandwidth of 1.86 Mb/sec1.86 Mb/sec

– Decent quality streamed video is restricted Decent quality streamed video is restricted to LAN, T1 lines, ADSL & cable modems for to LAN, T1 lines, ADSL & cable modems for nownow

Delivering time over networkDelivering time over network– Deliver data with the minimum of delayDeliver data with the minimum of delay

Delay may cause independently delivered video & Delay may cause independently delivered video & audio stream to lose synchronizationaudio stream to lose synchronization

Page 62: Chapter10

Conventional Delivery of WWW Conventional Delivery of WWW VideoVideo

Embedded videoEmbedded video– Transfer movie files from a server to the user’s Transfer movie files from a server to the user’s

machinemachine– Playback from disk once the entire file has arrivedPlayback from disk once the entire file has arrived

Progressive download Progressive download ((HTTP streamingHTTP streaming))– Transfer movie files to user’s diskTransfer movie files to user’s disk– Start playing as soon as enough of it has arrivedStart playing as soon as enough of it has arrived– The file usually remains on the user’s disk after The file usually remains on the user’s disk after

playback is completedplayback is completed

start playing

100

0

downloadplay

time

Cannot be used for live video !

Page 63: Chapter10

True StreamingTrue Streaming Each stream frame is played as soon as it Each stream frame is played as soon as it

arrives over the networkarrives over the network Video files is never stored on the user’s Video files is never stored on the user’s

diskdisk– Length of streamed movie is limited only by the Length of streamed movie is limited only by the

storage size at the server, not by the user’s storage size at the server, not by the user’s machinemachine

Suit for live video & video on demand Suit for live video & video on demand (VOD)(VOD)– The network must be able to deliver the data The network must be able to deliver the data

stream fast enough for playbackstream fast enough for playback– Movie’s data rate & quality is restricted to what Movie’s data rate & quality is restricted to what

the network can deliverthe network can deliver

Page 64: Chapter10

State of the ArtState of the Art Leading technique over the InternetLeading technique over the Internet

– RealVideo (Real Networks)RealVideo (Real Networks)– Streaming QuickTime (Apple)Streaming QuickTime (Apple)– Media Player (Microsoft)Media Player (Microsoft)

ArchitectureArchitecture– RTSP (Real Time Streaming Protocol)RTSP (Real Time Streaming Protocol)

Control the playback of video streamsControl the playback of video streams– Providing several versions of a movie compressedProviding several versions of a movie compressed

Server chooses the appropriate one to fit the speed of the Server chooses the appropriate one to fit the speed of the user’s connectionuser’s connection

Page 65: Chapter10

Codecs for Video ConferencingCodecs for Video Conferencing

H.261H.261– Designed for two-way telecommunication Designed for two-way telecommunication

applications over ISDNapplications over ISDN– A precursor of MPEG-1A precursor of MPEG-1

DCT-based compression with motion DCT-based compression with motion compensationcompensation

It does not use B-picturesIt does not use B-pictures

H.263H.263– Very low bit rate videoVery low bit rate video

H.263+H.263+– An extension of H.263An extension of H.263

Page 66: Chapter10

H.261H.261 Real time constrainReal time constrain

– Video conference cannot tolerate longer delays Video conference cannot tolerate longer delays without becoming disjointedwithout becoming disjointed

– Maximum delay: 150 ms (about 7 frames/sec)Maximum delay: 150 ms (about 7 frames/sec) Bit rate: Bit rate: p p 64 kbps ( 64 kbps (pp = 1 ~ 30) = 1 ~ 30) Picture formatPicture format

– CIF (Common Intermediate Format)CIF (Common Intermediate Format) Component(size): Y(352 Component(size): Y(352 288), C 288), Cbb & C & Crr(176 (176 144) 144) Picture rate: 29.97 frames/secPicture rate: 29.97 frames/sec

– QCIF (Quarter CIF)QCIF (Quarter CIF) Component(size): Y(176 Component(size): Y(176 144), C 144), Cbb & C & Crr(88 (88 72) 72)

Page 67: Chapter10

H.263H.263 Very low bit rate video (< 64 kbps)Very low bit rate video (< 64 kbps) Primary target rate is about 27 kbps (V.34 modPrimary target rate is about 27 kbps (V.34 mod

em)em) Compression techniquesCompression techniques

– Chroma sub-sampling 4:2:0Chroma sub-sampling 4:2:0– DCT compression with quantizationDCT compression with quantization– Run-length and variable-length encoding of coefficiRun-length and variable-length encoding of coeffici

entsents– Motion compensation with forward & backward preMotion compensation with forward & backward pre

dictiondiction– Compress a QCIF picture as low as 3.5 frames/secCompress a QCIF picture as low as 3.5 frames/sec