Top Banner
Allyn Romanow ([email protected] ) Mark Duckworth ( [email protected] ) Andy Pepperell ([email protected] ) Brian Baldino ([email protected] ) CLUE Framework First Draft IETF - 81 July, 2011
35
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Allyn Romanow ([email protected]) Mark Duckworth ([email protected] ) Andy Pepperell ([email protected]) Brian Baldino ([email protected] )

CLUE Framework

First Draft

IETF - 81

July, 2011

Page 2: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

R

Multiple Media Streams

C C

LL

R R

London

Dallas

Paris

Video and Audio

Video and Audio

R

L

Video and Audio

L

Page 3: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Challenges

Usable now

• Current functionality

Simple

• Practical to implement

Extensible

• Future functionality

Page 4: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

What’s Needed?

MEDIA CAPTURE DESCRIPTION

CHOOSING STREAMS

Page 5: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Process Consumer sends hints to provider

Provider sends capabilities

Consumer chooses streams (Not negotiated in the strict sense, 2 one-way)

Page 6: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Structure of Information

Media CaptureAudio or Video

Attributes

EncodeGroup

Media CaptureAudio or Video

Media CaptureAudio or Video

Simultaneous Transmission Set

Capture Sets

Page 7: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Media Capture Description

Mark Duckworth

Page 8: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Media Capture & Attributes

Capture Sets

Media CaptureAudio or Video

AttributesEncodingGroup

Media CaptureAudio or Video

Media CaptureAudio or Video

Simultaneous Transmission Set

Page 9: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Attributes

EXTENSIBILITY

Audio attributes• Purpose (role)

Main Presentation

• Mixed – true/false• Channel Format

Linear array Stereo Mono

• Linear position 0 to 100

Video attributes• Purpose (role)

Main Presentation

• Composed – true/false• Auto switched

True/false

• Spatial scale Image width

Page 10: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Capture Scene

VC0 VC2VC1

VC3 VC4Cameras

People VC1

VC2

VC0

Capture Scene

Three cameras

Two cameras, moved & zoomed out

Switched (based on voice) with composed PiP

VC5

Page 11: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Capture Set

Each alternative representation of a Capture Scene is a row in a Capture Set

Three cameras

Two cameras, moved and zoomed out

Switched (based on voice), composed PiP

(VC0, VC1, VC2)

(VC3, VC4)

(VC5)

(AC0)

Capture Set Rows VC0 VC2VC1

VC3 VC4

VC5

Page 12: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Video Capture Adjacency

cameraspeople

right

leftVC0

VC1

right

left

VC0

VC1

Capture Set:

(VC0, VC1)Other capture set rows

Page 13: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Matching Audio with Video

Same capture scene Video adjacency matches audio sound stage

Page 14: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Linear Array

Stereo

Matching Audio with VideoSpatial extent of video

Spatial extent of audio

Left Right

0 10050

VC0 VC2VC1

Page 15: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Choosing Streams

Andy Pepperell

Page 16: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Basic message flow

Media Stream

Consumer

Media Stream

Provider

Consumer capability advertisement

Media capture advertisement

Consumer configurationof provider’s streams

Page 17: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Capabilities Sent by Consumer

Media Stream

Consumer

Consumer capability advertisement

Physical factors

User preferences

e.g. number of screens

Software limitations

e.g. media capture attributes known

Page 18: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Advertisement Sent by Provider

Media Stream

Provider

Media capture advertisement

Consumer capability advertisement

Provider fixed characteristics

Dynamic factors

e.g. number of cameras

e.g. whether presentation source present

Page 19: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Configure Msg Sent by Consumer

Media Stream

Consumer

Stream configure message

Provider capture advertisement

Consumer’s fixed characteristics

Dynamic factors

e.g. number of screens

e.g. change of user preferences

simultaneous transmission set + encoding groups

Page 20: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Provider Capture Advertisement

Captures and attributes

Simultaneous transmission sets

Capture sets

Encoding groups

Page 21: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Simultaneous Transmission Sets

Center camera can do either regular or zoomedPeople

Right

CenterVC1

VC2

LeftVC0(VC0, VC1, VC2)(VC0, VC3, VC2)

VC3

Page 22: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Encoding Groups

Media Stream

Provider

Encoding group

Encoding group

Encoding Group

Attribute Name Description

maxBandwidth Maximum number of bits per second relating to all encodes combined

maxVideoMbps Maximum number of macroblocks per second relating to all video encodes combined:((width + 15) / 16) * ((height + 15) / 16) * framesPerSecond

videoEncodes[] Set of potential video encodes can be generated

audioEncodes[] Set of potential audio encodes that can be generated

Page 23: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Media stream provider

Encoding groupEncoding group

Encoding Group Structure

Encoding group

Encode 1 Encode 3Encode 2

Page 24: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Video Encode Attributes

Name DescriptionmaxBandwidth Maximum number of bits per second relating to the video encode

maxMbps Maximum number of macroblocks per second relating to the video encode:((width + 15) / 16) * ((height + 15) / 16) * framesPerSecond

maxWidth Video resolution’s maximum width, expressed in pixels

maxHeight Video resolution’s maximum height, expressed in pixels

maxFrameRate Maximum frame rate

Page 25: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Sample Encoding Group

<=2 encodes, <= 1080p30

Bandwidth trade-off between encodes & group as a whole

EG0: maxMbps = 489600, maxBandwidth=6000000 ENC0: maxWidth=1920, maxHeight=1080,

maxFrameRate=60, maxMbps=244800, maxBandwidth=4000000

ENC1: maxWidth=1920, maxHeight=1080, maxFrameRate=60, maxMbps=244800, maxBandwidth=4000000

Page 26: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Examples

Brian Baldino

Page 27: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Single Camera Endpoint

Page 28: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Single Camera Endpoint

Page 29: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Single Camera Endpoint

Page 30: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Three Camera Endpoint

Page 31: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Three Camera Endpoint

Page 32: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Three Camera Endpoint

Page 33: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

MCU Scenarios

Page 34: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

Three Camera Endpoint with Presentation

Page 35: Allyn Romanow (allyn@cisco.com) Mark Duckworth (mark.duckworth@polycom.com ) Andy Pepperell (apeppere@cisco.com) Brian Baldino (bbaldino@cisco.com )allyn@cisco.commark.duckworth@polycom.comapeppere@cisco.combbaldino@cisco.com.

QUESTIONS