Digital Media Lecture 10: Video & Compression Georgia Gwinnett College School of Science and Technology Dr. Jim Rowan.

Digital Media

Lecture 10: Video & Compression

Georgia Gwinnett CollegeSchool of Science and Technology

Dr. Jim Rowan

Coping with Video Size

Consider human vision limitations1) Chrominance sub-sampling

Compression - two versions2) Spatial3) Temporal

differencingvectoring

Coping with Video Size

Consider human vision limitations1) Chrominance sub-sampling

Compression - two versions3) spatial4) Temporal

differencingvectoring

Chrominance sub-sampling

Humans can’t distinguish changes in color as well as they can distinguish luminance changes– http://en.wikipedia.org/wiki/Chroma_subsam

pling In our cameras…

– Of every 4 frames– store the luminance for each frame– only store a proportion of the color info– 4:2:0

Chrominance sub-samplinghttp://dougkerr.net/pumpkin/articles/Subsampling.pdf

http://en.wikipedia.org/wiki/Chroma_subsampling#Sampling_systems_and_ratios

Chrominance sub-samplingLuminance, Cr, Cb

Coping with Video Size

Aside from screen size and frame rate...

Consider human vision limitations1) Chrominance sub-sampling

Compression - two versions2) spatial3) Temporal

differencingvectoring

Coping with Video Size

Spatial compression • Individual images can be compressed

using the techniques discussed in the bitmapped section

• Doesn’t result in very much compression for video

• Doesn’t take into consideration the other frames that come before or after it

• Our video cameras do this, compressing each frame to jpeg

Coping with Video Size

Aside from screen size and frame rate...

Consider human vision limitations1) Chrominance sub-sampling

Compression - two versions2) spatial3) Temporal

differencingvectoring

Temporal Compression differencing

Use the Difference in two frames– A naive approach can result in good

compression– Works well for a small amount of movement

• Security cameras spend most of their time “seeing” the same thing all night long

– A Tarantino film? not so much…• Most pixels change with nearly every frame

Image Differencing

To subtract one image from the next Do it one pixel at a time

– red minus red– green minus green– blue minus blue

Store the difference To play it back

– Play frame one 1– Add frame 2 to frame 1

Next, an example in black & white

Temporal Compression differencing

Example 1, the difference of two identical images

Example 1, the difference of two identical images

The result ===>

Example 2, the difference of two similar images

Example 2, the difference of two similar images

The result ===>

Temporal Compressionvectoring

• When an OBJECT moves – compute its trajectory– fill in the resulting exposed background

– BUT there’s a problem...– why isn’t this an easy thing to do?

CapturedFrame 1

CapturedFrame 2

StoredObject

MovementVector

StoredBackground

More on differencing…

The differencing can happen in a forward manner and a backward manner

It might be more economical (in data size) to create a frame from a frame that follows it...

MPEG-2iFrame (interframe prediction)pFrame (forward predicted)bFrame (backward predicted)GOP (group of pictures)

http://en.wikipedia.org/wiki/Inter_frame

http://en.wikipedia.org/wiki/I-frames#Intra_coded_frames_.28or_slices_or_I-frames_or_Key_frames.29

MPEG terminology

iFrame: -a keyframe-spatially compressed(a fully specified image)

pFrame-predicted frame-contains only the differencebetween the current frame andthe previous iFrame(smaller in size than iFrame)

bFrame-bi-predicted frame-contains difference betweencurrent and both the precedingand following iFrames-even smaller in size than iFrame

MPEG terminology

MPEG terminology

All spatially (intra-frame) compressed

Spatially compressedand predictive (difference)

Spatially compressed forward and backward predictive(difference)

The largest

Smaller…but morecomputation

Smallest…but more computationand it is transported out of order!

Play sequence: 1 2 3 4 5 6 7 8 9 10

Transmit sequence: 1 4 2 3 7 5 6 10 8 9

Group(s) Of Pictures (GOPs)

So…

How does this ===> happen?

http://en.wikipedia.org/wiki/Inter_frame

Video CompressionWhat does it?

Coder/Decoder - Codec– encodes and decodes video

Can be symmetric• it takes as long to compress as decompress

Can be asymmetric • it takes longer to compress or decompress than it

does to decompress to compress

Video Compression:Which codec to choose?

It’s a tradeoff… compression technique

• sorenson• DV• Cinepak• Intel Indeo• MPEG4

the compression result• asymmetric or symmetric

– satellite provider’s choice?• larger or smaller final size• computational complexity• the artifacts generated

http://www.100fps.com/codec_quality_comparison.htm

Which is the original?Notice the artifacts?

http://www.100fps.com/codec_quality_comparison.htm

Center one is the originalLeft is “sharpened” Right is “blurred”

Retrograde motion

http://wiki.ggc.edu/images/8/85/JrowanSpring2012CroppedRetrogradeMotionClip.mov

http://wiki.ggc.edu/images/3/35/RetrogradeGMCarcadiaBroadbandHigh.mov

Two ways to make Moving Pictures:Video & Animation

In this class:– Video

• shot with a camera • captures images from the world• then play them back

– Animation • create frames individually• using inkscape and blender• play them back

Reading in the supplemental text:

Moving images: Video