Encoding For the Web. Encoding for the Web Target bit rate – knowing the connection speeds Balancing audio vs. video bit rate Image size Frame rate I.

Encoding For the Web

Encoding for the Web

Target bit rate – knowing the connection speeds

Balancing audio vs. video bit rate Image size Frame rate I - frames

Firstly what is video?

A sequence of still images displayed many times a second

The brain interprets the rapid changes and moving images are created.

15 frames per second and above is meant to create smooth motion

PAL (UK) video uses 25 interlaced frames per second NTSC (USA) video uses 30 interlaced frames per

second

Interlaced Scanning The image is made up of 625 individual lines and therefore the TV

set draws 15,625 lines per second (625x25)

The term horizontal retrace is used to refer to the beam moving back to the left at the end of each line, while the term vertical retrace refers to its movement from bottom to top.

Interlaced Scanning

Half of the 625 lines are drawn first. This is called field 1.

The first field draws is called the odd field as it draws the odd numbers 1,3,5,7, etc

The second half of the lines are called field 2 and this draws the even numbered lines 2,4,6,8 etc

This is also called 50i recording, for 50 interlaced images per second.

Field 1 and Field 2 = 1 Frame

Progressive Scan – what computers use and therefore streaming files to! A progressive scan

draws each of the scan lines one after another in numerical order e.g. 1,2,3….625

This gives a sharper image that an interlaced image.

This is called 25p recording.

There are no fields in a progressive scan just a single frame.

What are you seeing on a TV? PAL and NTSC

25fps and 29.97 fps

Interlaced and Progressive Pictures

How is a streaming file structured?

Normal video is made up from 25 frames complete frames per second.

When encoded to a streaming format it is made up from I-frames (key frames) and difference frames (delta).

The only real frames are the I-frames. The difference frames are predicted based upon complex algorithms which analysed the I frame for change.

Basic Encoding of a Streaming Media file

Encoder Input - video

Encoder Output

Some I frames, lots of difference frames

A Normal Video File Encoded to MPEG

MPEG-2 uses 2 or 3 I frames every second.

Streaming uses 1 every 300 frames (12 seconds!)

A Streaming File

Note the distance between the I frames. 300 frames is quite common between the two.

Decisions to make for Unicast Streaming to Prepare a file for professional encoding

Know your target audience Understand the client connection speed this will

determine what Image size will you select What frame rate What codec How much bandwidth will you give to the entire file

so how will you spilt this between video and audio content?

What pre-processing techniques could be deployed?

Allocating BandwidthMaking a decision on how much space you give to each component is very important over all connection speeds.

Low bandwidth connections like 56Kb modems usually give the audio around 30% to 40% of the total bandwidth. Consumer tests have shown that as long as the audio is constant and clear the viewer is satisfied with the experience even if the picture keeps breaking up.

It is possible during the encoding to specify which is most important, the video or the audio.

Cropping and High Quality Scaling Video has edges that you cannot see on a TV but a computer sees the

entire frame. Video runs at 720x576. Scale it to 320x240 and you have already removed

half the pixels without any filtering. Crop and Scale source video to eliminate the black edges and "garbled"

edge pixels. If the view is just a talking head you can crop out all the unimportant

information.

Pre-processing

Generally only used when encoding down not have to be real time.

Why pre-process? Filtering further reduces bandwidth Reducing/adding keyframes?

Why? What do they do? How many in a sequence?

Structure of a file in reference to streaming Information frames and delta frames.

Pre-Processing Techniques

De-interlacing Field Dominance change Inverse Telecine 2D Filters

BlursAdaptive noise Reduction

De-Interlacing – Various Techniques 1. Process of removing a field of video – prepares the

video for a progressive viewing and sharpens the image up.

2. Blending the two fields together – what issues could this raise?

3. Adaptive deinterlace• Analyses the video to selectively interlace the parts of the

video which are moving and leaves the still shots. This results in higher quality video stream in areas of motion.

• This also helps to average the noise in the picture.

3:2 Pull Down and Inverse Telecine

3:2 pulldown converts NTSC into PAL or into film frame rates. 5 or 6 frames are dropped every second.

Inverse TelecineUsed to increase a frame rate from film

(24fps) or Pal (25) to NTSC (30). It works by adding frames to compensate for

the difference.

Blurs Use to clean up a noisy picture by blurring the noise into the image. A low pass filter is applied to the entire image. This colours are reduced but detail is lost. You alter the radius of the blur the amount can be altered. Not effective to use unless on a progressive image. The amount of colours have been reduced slightly which will makes encoding easier. Why? As

the images lends its self to thing like RLE encoding better.

Run Length Encoding

Run-length encoding Bitmapped graphics files are typically larger than they need to be.

Run-length encoding is a compression technique that reduces file sizes, especially for black-and-white or cartoon-style line art graphics.

It works by replacing "runs" of three or more with the same color with a single character. The more runs there are, and the longer the run sequence, the greater the compression.

Run Length Encoding

Consider a screen containing plain black text on a solid white background. There will be many long runs of white pixels in the blank space, and many short runs of black pixels within the text. Let us take a hypothetical single scan line, with B representing a black pixel and W representing white: WWWWWWWWWWWWBWWWWWWWWWWWWBBBWWWWWWWWWWW

WWWWWWWWWWWWWBWWWWWWWWWWWWWW If we apply a simple run-length code to the above hypothetical scan line, we

get the following: 12WB12W3B24WB14W

Interpret this as twelve W's, one B, twelve W's, three B's, etc. The run-length code represents the original 67 characters in only 16.

It is this theory that is applied after the Zig Zag scan in the encoding process.

Therefore, run-length encoding replaces strings of repeated characters (or other units of data) with a single character and a count

%4020

1220

input

ouputinput

L

LL

Run Length Encoding

Simplistic example: Given the 20 character data string: oooeiiiippuuuutddddd This could be re-coded as: o3ei4ppu4td5 which has only 12 characters

Compression rate:

Run Length Encoding (contd.)

Give a simple RLE representation for the following data string: ggggguujjiiiQQQn

What should the answer be? g5uujji3q3n

Compression ratio is (16-11)/16=0.31 or 31% Why does it look only replace runs or 3 or more

with a letter and a count?

Sharpening This improves the image by subjectively refocusing the image. The edges of objects are amplified by an unsharp mask algorithm. Whereby a high-pass filter is applied and added back onto the original. This can be very important when streaming over low bandwidth connections. Offering the client an acceptable image quality is the key thing. Streaming over low

bandwidth is all about making the images the best you can whilst applying heavily compression.

The control is via two parts. A radius sharper that controls the sharpening pixel to pixel and then also the percentage of the sharpened image that is overlaid on the original image.

Before After

Adaptive Noise Reduction

Noise wastes precious bits. But it is also not present throughout the entire image. This is where this filter comes in.

Blur applies the low pass filter across the entire image, while Adaptive Noise Reduction only applies it to flat areas, so edges aren't affected. It's better to apply the latter unless you're looking for a softer focus effect.

It also selectively blurs some pixels while sharpening others, resulting in output that compresses well but retains sharp edges.

You can remove single stray pixels either alone in, or in combination with, a flat field filter.

You can also customize the filter characteristics as shown in the figure. Here you can specify how flat an area needs to be

If you have serious noise artefacts in your video, experiment with these settings -- otherwise it's faster to use the preset filters.

VBR v CBR Live streaming uses CBR only at present. DVD mainly uses VBR (in pro work)

- 3 to 9Mbps is average What is CBR (constant bit rate)? What is VBR (variable bit rate)? VBR also used for multiplexing digital

satellites. e.g. Borrowing bandwidth from other channels.

Practical

Show how to edit the codec choice in Windows Media Encoder

Cropping Show Video on Encoding

Digital rights management

Digital rights management (DRM) is the umbrella term referring to any of several technologies used to enforce pre-defined policies controlling access to software, music, movies, or other digital data.

In more technical terms, DRM handles the description, layering, analysis, valuation, trading and monitoring of the rights held over a digital work.

Digital rights management

The term is often confused with copy protection and technical protection measures (TPM).

These two terms refer to technologies that control and/or restrict the use and access of digital media content on electronic devices with such technologies installed.

There are technical measures that could be used not to restrict use or access, such as to monitor use in order to record rights of a content consumer

Some digital media content publishers claim DRM technologies are necessary to prevent revenue loss due to illegal duplication of their copyrighted works. However, others argue that transferring control of the use of media from consumers to a consolidated media industry will lead to loss of existing user rights and stifle innovation in software and cultural productions.

DRM

As the name implies, it applies only to digital media (and analog media that was at one point digital).

Digital media have gained in popularity over analog media both because of technical advantages associated with their production, reproduction, and manipulation, and also because they are sometimes of higher perceptual quality than their analog counterparts.

Since the advent of personal computers, digital media files have become easy to copy an unlimited number of times without any degradation in the quality of subsequent copies.

Many analog media lose quality with each copy generation, and often even during normal use.

The popularity of the Internet and file sharing tools have made the distribution of copyrighted digital media files simple.

DRM

The availability of multiple perfect copies of copyrighted materials is perceived by much of the media industry as a threat to its viability and profitability, particularly within the music and movie industries.

Digital media publishers typically have business models that rely on their ability to collect a fee for each copy made of a digital work, and sometimes even for each performance of said work.

DRM was designed for digital media publishers as a means to allow them to control any duplication and dissemination of their content.

DRM

DRM is vulnerable to an additional class of attacks due to its need to be run on tamper-resistant hardware (DRM systems that do not run on tamper-resistant hardware cannot ever be theoretically secure since digital content can be copied on a hardware level).

The case for DRM is that without a strong system in place to ensure only paying consumers can access media, piracy will run rampant and cut drastically into profits for producers and distributors. With declining sales, so the argument goes, creative input will also drop and the overall quality of media produced will decline.

CSS DRM

One of the first and most widely contested DRM systems was the Content Scrambling System (CSS) used to encode DVD movie files.

This system was developed by the DVD Consortium as a tool to influence hardware manufacturers to produce only systems which didn’t include certain features.

By releasing the encryption key for CSS only to hardware manufacturers who agreed not to include features such as digital-out, which would allow a movie to be copied easily, the DVD Consortium was essentially able to dictate hardware policy for the DVD industry.

DRM

While DRM is most frequently used for movies, it is gaining more widespread use in other media as well. Audio files purchased through many online stores, such as Apple’s iTunes Store, have various DRM schemes built in to limit the number of devices they may be played on.

Many producers of eBooks are using a similar implementation of DRM to limit how many computers a book may be viewed on, and even how many times it may be viewed.

Flaws of DRM Physical protection: using external dongles DIVX: Required a phone line, inhibiting mobile use. CSS: Restricts fair use and first purchaser rights, such as the creation of compilations

or full quality reproductions for the use of children or in cars. It also prevents the user from playing CSS-encrypted DVDs on any computer platform (although this restriction can be easily circumvented). CSS is an example of certificate-based encryption.

Product activation: Invalidates or severely restricts a product's functionality until the product is registered with a publisher by means of a special identification (activation) code. The process often uses information about the specific configuration of the hardware on which the software runs, hashing it with the identification number specific to the product's license.

Digital watermarking: Allows hidden data, such as a unique disc ID, to be placed on the media. Then, the name and address of the purchaser would be taken at the location of sale, and entered into a database along with the unique media ID.

This does not prevent copying, but it ensures that any copies made of the media will bear the same hidden information—so if the content appeared on (for example) P2P networks, the ID number could be easily extracted and the purchaser prosecuted. This scheme is flawed primarily because authenticating the buyer as the infringing party is nearly impossible: