Multimedia Services: Video

Multimedia Services:Video

Sep-2015

Dani Gutiérrez PorsetAssociate Professor

Communications Engineering

Eman ta zabal zazu

2 2Multimedia Services:

Video

Thanks, Licences and Tools

● Thanks to people and organizations who took or take part in free software and free knowledge projects, specially Wikimedia Foundation and KDE

● This presentation is licensed as CC BY-SA 3.0 EShttp://creativecommons.org/licenses/by-sa/3.0/es/

● This presentation has been made with KDE, LibreOffice, Inkscape, Gimp, Chromium, Firefox

3 3Multimedia Services:

Video

Sources and References

● Images from Wikimedia Foundation, if not referenced other source. Logos and trademarks belong to respective organizations

● Texts:

– Wikipedia pages and referenced articles and material– “Guide to Voice and Video over IP” - Sun, Mkwawa, Jammeh,

Ifeachor– “Video over IP” - Wes Simpson– “Computer Networking, a top-down approach” - Kurose, Ross

4 4Multimedia Services:

Video

Index

● Introduction● Codecs and Containers● MPEG● Wires and Connectors

5 5Multimedia Services:

Video

Parameters

● Image + Audio parameters:– Raster Image parameters: size in pixels, colors,...– Audio parameters: sampling rate, bitrate,...

● Frame Rate (fps): Constant or Variable (VFR)● Bitrate: CBR or VBR● Compression: spatial and/or time domains

Introduction

6 6Multimedia Services:

Video

Video scanning:Progressive vs Interlaced. Fields

● Progressive video:

– In all frames each line appears next to previous line

– Used in computer graphics. Easier for software processing.

– e.g. 720p

Introduction

● Interlaced:

– A frame is divided in two fields: odd field (for odd lines) and even field (for even lines). Each time one field is showed

– Used in broadcasting. Permits lower frame rate.

– E.g. 525i, 625i, 1080i

7 7Multimedia Services:

Video

Frame rates and Flicker effect

● Flicker effect: an image needs to be updated >= 50 times/sec. Solutions:– Progressive: repeat the same frame– Interlaced: use frame field

● Typical frame rates:– PAL 50i– NTSC 60i (exactly 29,97 fps interlaced)– Others: 24p for video adquisition, 48p for high quality,...

● i and p letters usually appear near the number of vertical lines, e.g. 720p

Introduction

8 8Multimedia Services:

Video

Raw video

● Also called Uncompressed video and Clean HDMI (no overlaid text or graphics)

● Useful for High quality edition● Very high bitrate and space

e.g. 24bit @ 1080i @ 60fps :24 x 1920x1080 x 60 / 2 = 1.49 Gbps

● If recording to computer, need of RTOS and fast hard disks (e.g. SSD)

Introduction

9 9Multimedia Services:

Video

Video compression

● Types of compression:– Intraframe– Interframe: near frames are correlated. Make a

combination of actual and predicted frame contents

● Techniques:– Spatial correlation– Motion estimation– Motion compensation

Introduction

10 10Multimedia Services:

Video

Motion compensation andMotion estimation

● Describe a picture in terms of the transformation of a reference picture to the current picture

● Supposition: the only difference between one frame and another is the result of either the camera moving or an object in the frame moving

● Aim: determine motion vectors that describe the transformation from a 2D image (whole picture or zones) to another one

Introduction

11 11Multimedia Services:

Video

Codecs and Containers

● Containers:

– Main streams: Video + Audio + Subtitles

– Others: Chapters, Captions, Menus,...● Classifications:

– Codecs: highest supported Bitrate & Resolution, Streaming support,...

– Containers: Video and Audio supported formats,...

http://en.wikipedia.org/wiki/Comparison_of_container_formats

http://en.wikipedia.org/wiki/Comparison_of_video_codecs

http://en.wikipedia.org/wiki/Comparison_of_video_encoders

http://en.wikipedia.org/wiki/Comparison_of_video_converters

http://en.wikipedia.org/wiki/Video_codec

Codecs and Containers

12 12Multimedia Services:

Video

Organizations for standarization

● ISO/IEC MPEG: Moving Picture Experts Group– MPEG* codecs and containers

● ITU VCEG: Video Coding Experts Group: H.26* codecs

● SMPTE: Society of Motion Picture and Television Engineers

● (ISO/IEC JPEG: Joint Photographic Experts Group, from image)

Codecs and Containers

13 13Multimedia Services:

Video

Examples of Standard codecs

● H.261 and H.263: for lower bit rates videoconference. Low resolutions (CIF, QCIF,...)

● MPEG* (see later)● Motion JPEG, Motion JPEG-2000: each video

frame encoded as separate image. Adequate for editing, but higher bitrates than MPEG-2, MPEG-4

Codecs and Containers

14 14Multimedia Services:

Video

Correspondences betweenMPEG and VCEG codecs

● H.261 <-----> MPEG-1 Part 2● H.262 <-----> MPEG-2 Part 2● H.263 <-----> MPEG-4 Part 2 = MPEG-4 Visual

(implemented in DivX, Xvid codecs)● H.264 <-----> MPEG-4 Part 10 = MPEG-4 AVC● H.265 <-----> MPEG-H Part 2

Codecs and Containers

15 15Multimedia Services:

Video

Other Codecs

● Open and free:– Theora, from Xiph.org foundation, initially proposed for HTML5

● Proprietary codecs:– Microsoft WMV codecs: WMV7, WMV8, WMV9 = standard

SMTPE VC1– Real Networks– Sorenson (used in Adobe Flash and in Apple Quicktime

multimedia framework)– On2 Technologies: VP5, VP6, VP8 (used in Adobe Flash), VP9.

Google purchased On2 in 2010

Codecs and Containers

16 16Multimedia Services:

Video

Examples of Standard Containers

● MPEG Program Stream (MPEG PS): MPEG-1 Part 1 and MPEG-2 Part 1– VCD (MPEG-1 PS), SVCD (MPEG-2 PS)– VOB container:

● Used as container format in DVD-Video● Contents: Video, Audio, Subtitles, Menus● Format: Subset of MPEG PS

● MPEG Transport Stream (MPEG TS): MPEG-2 Part 1● MP4: MPEG-4 Part 14

Codecs and Containers

17 17Multimedia Services:

Video

Other Containers

● Open, free:– Ogg from Xiph.org foundation, initially proposed in HTML5– Matroska: admits lot of contents– WebM, derived from Matroska, for use in HTML5

● Proprietary:– Real Media– Apple Quicktime (.mov)– Microsoft AVI, ASF– Adobe Flash FLV– (GIF animation)

Codecs and Containers

18 18Multimedia Services:

Video

DV format

● Specification (codecs, containers,...) to save video and audio, mainly over a FireWire cable (IEE 1394)

● Frequently used to store digital video, e.g. video camera recorders

● Video:– Interlaced, 720x576@50 4:1:1 or 720x480@60 4:2:0– Lossy compression with intraframe DCT

● Audio: no compression

Codecs and Containers

19 19Multimedia Services:

Video

MPEG

● Specification history:– 1990: MPEG-1. Legacy. Uses: VCD– 1996: MPEG-2. Uses: DVD Video, Digital TV, first

HDTVs and Blu-ray– (No MPEG-3)– 2000: MPEG-4– 2003: H.264 = MPEG-4 Part 10 = MPEG-4 AVC

● MPEG LA manages licence fees e.g. for devices

MPEG

20 20Multimedia Services:

Video

MPEG StructureMPEG

21 21Multimedia Services:

Video

Macroblocks, Blocks and Slices

● Macroblocks of 16px x 16px● Block size depending on chroma subsampling, e.g.

– 4:2:0: 4 for luma, 1 for Cr and 1 for Cb block. Used in DVDs– 4:2:2: 4 for luma, 2 for Cr and 2 for Cb

● Type of Blocks:– Transform blocks: to apply DCT calculations– Prediction blocks: to apply motion compensation

● In MPEG-4 AVC macroblocks can be grouped horizontally into slices

MPEG

22 22Multimedia Services:

Video

Picture or Frame types

● I-frames (Intra-coded): coded without references to other frames; same as a static full image. Least compressed. Equivalent to JPEG image

● P-frames (Predicted) or delta-frames: encoded frames with reference to the previous ones. Medium compressed

● B-frames (Bi-predictive): encoded with reference to previous and forward frames. Maximum compression

● D-frames: Old: in MPEG-1 low quality, not referenced by other frames. Used for fast previews

MPEG

23 23Multimedia Services:

Video

Group of Pictures

● A GOP is a Group of a single intraframe (I) and zero or more interframes (P and B). It always begin with an I frame and ends with the last frame before next I frame

● Types: Open GOP if it uses I frame of the next GOP, Closed if it's self-contained

● GOP structure referenced by two numbers:– N = GOP length: distance between two full images (I)– M: distance between two anchor frames (I or P)– Example: N=12, M=3: IBBPBBPBBPBB I

MPEG

24 24Multimedia Services:

Video

Comparison of GOP lengths

● Long, better to reduce bitrate and bandwidth● Short:

– Better Quality, mainly if fast image changes– Less delay– Easier editing

MPEG

25 25Multimedia Services:

Video

MPEG-2

● MPEG-2 = H.262● Commonly used in professional video recording and broadcasting.

Huge installed base of devices (DVD Video, Digital TV)● Video Features:

– Video: interlacing (PAL, NTSC) and HD video (720p, 1080i)– 12 combinations of 6 Profiles (compression techniques) and 4 Levels

(resolutions), that require distinct bandwidth and complexitye.g: DVDs: main level (720x576 in PAL or 720x480 in NTSC) and main profile (color 4:2:0)

● Audio features: 5.1 MPEG-1 and AAC

MPEG

26 26Multimedia Services:

Video

MPEG-4

● Improvements in video compression (specially for IP networks), ~ 50% bandwidth of MPEG-2

● Techniques:– New video objects, created from synthetic sources (raster or

vectorial graphics, text) that use less bandwidth and can be controlled by receptor user

– Variable Macroblock size: smaller for accuracy in some image areas, bigger for less bandwidth

– Fractal compression– Allows B frames based on other B frames

MPEG

27 27Multimedia Services:

Video

MPEG Audio compression

● Lossy compression, from 4:1 in MPEG-1 layer I to 32:1 (AAC+)

● Audio streams of 32 kHz, 44.1 kHz, 48 kHz● Main families:

– MPEG-1 audio Layers: 3 versions: I, II, III (“mp3”)– Advanced Audio Coding (AAC ), from MPEG-2. There

are 3 versions, e. g. AAC: up to 48 channels, lossless compression mode, audio synthetic objects (e. g. MIDI)

MPEG

28 28Multimedia Services:

Video

Video hardware:wires and connectors

● Each specification defines the protocols, signals, electrical interfaces and mechanical structure

● For video and/or audio● For analog and/or digital signals, e.g.

– Analog: VGA– Digital: HDMI, IEEE 1394=Firewire (similar to USB but faster)

● Each one supports a maximum spatial resolution and frequency, e.g.– VGA: QXGA (2048 × 1536) @ 85– HDMI: 4K (4096 × 2160) @60

Wires and Connectors

http://en.wikipedia.org/wiki/List_of_video_connectors

29 29Multimedia Services:

Video

Video hardware:Analog wires and connectors

● Composite Video: 1 conductor with all info● S-Video (Separate Video): 2 conductors separating luma

and chroma● Component Video: 3 conductors, for RGB or YUV/YPbPr.

For sync signal 3 choices: on green (Y), 1 conductor for both V and H sync, 2 conductors for separate V and H sync

● VGA: carries R, G, B and H, V syncs● SCART: audio (stereo) + video (composite and S-Video)

Wires and Connectors

30 30Multimedia Services:

Video

Video hardware:HDMI wire and connector

● High-Definition Multimedia Interface● Features:

– Digital– Uncompressed video in RGB or YCbCr– Digital Content Protection: before sending info verifies the remote device is

authorized, and encrypts the info– No subtitles support

● Distinct supported A/V capabilities depending on HDMI version:– For Video: resolution (up to 4K), fps, color spaces, chroma subsamplings, color

depth, aspect ratios, multi streaming (up to 2 streams)– For Audio: no. of channels (up to 32), sample sizes and rates, formats (Dolby

Digital, DTS,...), multi streaming (up to 4 streams)http://en.wikipedia.org/wiki/HDMI

Wires and Connectors