AES67 audio over IP within SMPTE 2110 - We are SMPTE | …€¦ · · 2018-01-03to IP networking. SMPTE ST 2110 ... campuses, stadiums, event venues and local area IP networks.

AES67 audio over IP within SMPTE 2110

Peter Stevens

Date of Presentation: 16th November 2017

Topics

• Introduction to SMPTE Digital Media Standards

• SMPTE 2110 Relationship to AES67

• AES67 History

• ASE67 Technology Components

• Interop Plugfests

• Broadcasting Examples

• What Next?

• 2110-30 Summary


Acknowledgments to:

Andreas Hildebrand of ALC NetworX for adapting some of his ideas and

Swedish Radio for their slides and video

SMPTE ST 2022 SeriesSending digital video over an IP network – the Early Years


• Video formats supported include MPEG-2 and serial digital interface (SDI)

• Transportation based on RTP (Realtime Streaming Protocol) RFC3550

• SMPTE ST 2022-1 – FEC for Real-Time Video/Audio Transport Over IP Networks

• Defines row/column FEC (Forward Error Correction) for IP video streams

• SMPTE ST 2022-2 – Unidirectional Transport of CBR MPEG-2 Transport Streams on

IP Networks

• Encapsulation of MPEG-2 transport streams into IP packets

• SMPTE ST 2022-3 – Unidirectional Transport of VBR MPEG-2 Transport Streams on

IP Networks

• Defines IP packets for variable bit-rate MPEG-2 TS - constrained to have a CBR between PCR

messages

SMPTE ST 2022 SeriesSending digital video over an IP network – the Early Years


• SMPTE ST 2022-4 – Unidirectional Transport of Non-Piecewise Constant VBR

MPEG-2 Streams on IP Networks

• Similar to 2022-3, except removes bit rates constraints

• SMPTE ST 2022-5 – High Bit Rate Media Transport Over IP Networks

• Expansion of 2022-1 allows larger row/column FEC combinations - support signals > 3 Gbps

and beyond

• SMPTE ST 2022-6 – Transport of High Bit Rate Media Signals over IP Networks

(HBRMT)

• Transportation of high bit-rate signals not encapsulated in MPEG-2 transport streams

• SMPTE ST 2022-7 – Seamless Protection Switching of SMPTE ST 2022 IP Datagrams

• Automatic resilience switching by receiver of identical streams over different routes

• SMPTE 2022 is an important technology enabling the transition of broadcast systems

to IP networking

SMPTE ST 2110 Standard SuiteProfessional Media over managed IP Networks


• SMPTE ST 2110-10 – System Timing and Definitions

• Defines transport layer & synchronisation (SMPTE 2059, clocks, RTP, SDP, etc)

• SMPTE ST 2110-20 – Uncompressed Active Video

• Defines payload format for raw video (RFC4175, RTP, SDP, constraints)

• SMPTE ST 2110-21– Traffic Shaping and Delivery Timing for Video

• SMPTE ST 2110-30 – PCM Digital Audio

• SMPTE ST2110-31– AES3 Transparent Transport

• SMPTE ST 2110–40 – Payload definition for SMPTE ST 291-1 ANC Data

• https://www.smpte.org/st-2110

https://www.smpte.org/st-2110

SMPTE ST 2059 Standard Suite


• SMPTE ST 2059-1 – Generation and Alignment of Interface Signals to the

SMPTE Epoch

• Alignment points for interface signals (that exist today)

• Formulae for direct calculation of signals from PTP time

• Formulae and algorithms for deterministically calculating ST12 Time-address and ST309

date

• SMPTE ST 2059-2 – SMPTE Profile for Use of IEEE-1588 Precision Time

Protocol in Professional Broadcast Applications

• Specific PTP rules required by SMPTE application

• SMPTE-specific helper metadata

• Network and SMPTE parameters

SMPTE 2110 Relationship


Video

Ancillary

Audio

Video

Ancillary

Audio

Elemental

RTP Streams

• 2110 allows SDI packaged media to be sent over network as groups of

individual streams; individual streams not derived from SDI may also be used

• Received streams can be put back together into SDI, if required, or left as

synchronised individual streams

• Benefit also comes with audio only devices…….

Encapsulate,

Packetise &

Timestamp

Video

Ancillary

Audio

SMPTE

2110-x

SenderSource IP Network

SDI or non-SDI signals

SMPTE

2110-x

Receiver

Video

Ancillary

Audio

Accumulate,

Decapsulate,

&

Synchronise



Accumulate,

Decapsulate,

&

Synchronise

Encapsulate,

Packetise &

Timestamp

Video

Ancillary

Audio

IP Network

SMPTE

2110-x

Sender

SMPTE

2110-x

Receiver

Video

Ancillary

Audio

Video

Ancillary

Audio

Video

Ancillary

Audio

Elemental

RTP Streams

• Receiver can take just the audio elements (as 2110-30) from the network

• Or add external IP media sources…. All streams can be fully synchronised

Source

Accumulate,

Decapsulate,

&

Synchronise

SMPTE

2110-30

Receiver

AudioAudio

SDI or non-SDI signals


• SMPTE 2110-30 – PCM Digital Audio

• Digital audio streams shall conform to AES67• An RTP-based transportation specification of PCM digital

audio streams over IP

• Includes SDP (RFC4566) metadata for stream reception and

interpretation

• Mandatory audio requirements for all devices:• 48kHz sampling (Media & RTP clocks same as sampling rate)

• 1ms packet time

• 1..8 channels per stream

• 16 & 24 bit depth (L16 - RFC3551 & L24 - RFC3190)

• Specification includes further provisions

• Outside mandatory requirements – read spec carefully



• Some differences between how AES67 works within SMPTE

2110-10 and 2110-30 and operation within the pure audio

domain.

• Synchronisation and Timing

• PTP:• Support of SMPTE 2059-2 required

• Message rate according to AES-R16-2016 (AES Media profile)• defaultDS.slaveOnly=true for devices not intended of entering

the PTP master state (under consideration at the moment)• a=ts-refclk:ptp=traceable and a=tsrefclkts-

refclk:localmac=<mac_addr> allowed state

• RTP clock: offset=0 with respect to media clock/network

clock• a=mediaclk:direct=0



• Protocols

• Support of RTCP – RFC3551 not required by 2110 (must be

tolerated)

• Support of SIP - RFC3261 (or other connection

management protocol) not required

• Redundancy: SMPTE 2022-7• Identical IP source and destination addresses not allowed

• Optional channel assignment map (SDP)• a=fmtp:<payload type> channel-

order=<convention>.<order>

• E.g: a=fmtp:101 channel-order=SMPTE2110.(51.ST)

• No differences in SDP between 2110-30 and AES67 (apart

from channel assignment)



• More Protocols

• Conformance level support by 2110-30 receivers with

respect to AES67 stream formatting

• Other values are already recommendations in AES67


Level

48 kHz, 1..8,

1ms

48 kHz, 1..8,

125 µs

48 kHz,

1..64, 125 µs

96 kHz, 1..4,

1ms

96 kHz, 1..8,

125 µs

96 kHz,

1..32, 125 µs

A Y

Ax Y Y

B Y OR Y

Bx Y OR Y Y OR Y

C Y OR Y

Cx Y OR Y Y OR Y

Pure

AE

S67

AES67 History

EBU Audio Contribution over IP (ACIP) used existing

standards from the world of VoIP (Voice over IP) along with

broadcast quality codecs to produce a manufacture

interoperable standard, usable over the Internet, wide and

local area networks.

AES67 has used the same model for the development of an

interoperable standard for existing and competing

professional low latency audio solutions used within

campuses, stadiums, event venues and local area IP networks.

Use existing standards where possible -

don’t reinvent the wheel


AES67 History

AES67-2015 Standard for Audio Applications of

Networks -

High-performance Streaming Audio-over-IP

Interoperability

Originally published on September 11th 2013

Second Edition published September 21st 2015


AES67 History

• Interoperable guidelines for professional audio• IP Based

• Low latency

• Campus and LANs

• Excludes:• Other network types

• Low-bandwidth media

• Data compression – codecs (ACIP)

• Low performance WANs and general Internet

• Methodology could be used for video

• “Mother audio networking protocol” installed in products

to make use of AES67


AES67 History

• Intended Applications

• Commercial Audio• Installed sound: theatres, stadiums, theme parks, cruise ships

• Live sound – fixed and touring

• Professional broadcast• In-house distribution

• Inter-facility links on corporate networks

• BBC R&D used similar RTP/audio for Commonwealth

Games in 2014 as an experimental trial

• OB vehicles

• Music production

• Post production


AES67 HistoryCandidate AoIP Solutions

Date Technology - Manufacturer Transport Synchronisation

2003 Livewire – Telos/Axia RTP Proprietary

2005 Wheatnet-IP - Wheatstone RTP Proprietary

2006 Dante - Audinate UDP IEEE 1588-2002

2007 N/ACIP – EBU RTP Adaptive per stream

2009 Q-LAN – QSC UDP IEEE 1588-2002

2010 RAVENNA – ALC NetworX RTP IEEE 1588-2008

2011 AVB - IEEE AVnu Ethernet/RTP IEEE 802.1AS


AES67 History


OSI Layer A-Net EtherSound Cobranet Livewire,

Dante, …

AVB AES67 &

RAVENNA

Application (7)

Presentation (6)

Session (5) RTP RTP

Transport (4) UDP UDP

Network (3) IP IP

Data Link (2) Ethernet

Physical (1) Copper Copper/fibre

IP

AES67 History


Well we do!

We can’t talk to one another!Now we can all talk to one another

“O” Negative of Audio Networking – Roland Hemming, Independent Audio Consultant


AES67

RAVENNA

AES67

AES67AES

67

AES67 Outline

• High-performance Streaming Audio-over-IP Interoperability Specification

• Minimal common set of parameters defined for audio in realms of:

• Audio Coding

• Payload Format and Sampling rates

• Packet Time

• Other requirements:

• Media Clocks

• Network Synchronisation

• Transport

• Connection Management

• Network Quality of Service

• Discovery

•AES67 audio over IP within SMPTE 2110

Core AES67 Technology Components


Default Profile

Media ProfilePTP IEEE 1588-2008Synchronisation

AES67

Technology

Components

Synchronisation & Media Clocks


Wall Clock

PTP GM

Local

ClockLocal

Clock

Media

Clock

Media

Clock

Stream

Clock

Sender Receiver

RTP

Clock

RTP

Clock

GPS

PTP copyPTP copy

RoffToff

Soff

SDP

Stream Data

Local Clocks receive PTP copy

Sender Toff established

and Receiver Roff established

Convey SDP to Receiver

Relationship Soff established on

stream start-up – may be

random

Offset constant throughout

stream lifetime – conveyed via

SDP

(a=mediaclk:direct=<offset>)

but “0” in ST2110


• Phase accuracy of AES 11 (± 5% of sample period)

• Deploy PTP-aware switches (BC or TC)

• Essence data (audio samples or video frames) is

related to the media clock upon intake – essentially

receiving a generation “time stamp” with respect

to the media clock

• Fixed / determinable latency by configuring a

suitable link offset (“playout delay”)

• Independent stream alignment by comparing and

relating the RTP time stamps of individual essence

data


BC BC BC

GM

TC TC

Node Node Node

Node


• Link offset describes the latency through a

media network

• Time difference between ingress at the

sender and egress at the receiver

• Determined by receiver setting a buffer

value greater than the network latency

• Receiver buffering to absorb jitter - not

too short or long

• Sender buffer – take into account packet

time and networks stack/controller




Default Profile


From PTP

Local

Media Clock

Generation

AES67

Technology

Components

Example multicast SDP

8 channel, 24-bit, 48kHz, 1ms packet timev=0

o=- 1311738121 1311738121 IN IP4 192.168.1.1

s=Stage left I/O

c=IN IP4 239.0.0.1/32

t=0 0

m=audio 5004 RTP/AVP 96

i=Channels 1-8

a=rtpmap:96 L24/48000/8

a=recvonly

a=ptime:1

a=ts-refclk:ptp=IEEE1588-2008:39-A7-94-FF-FE-07-CB-D0:0

a=mediaclk:direct=0


Example unicast SDP

8 channel, 24-bit, 48kHz, 250μs packet timev=0

o=audio 1311738121 1311738121 IN IP4 192.168.1.1

s=Stage left I/O

c=IN IP4 192.168.1.1

t=0 0

m=audio 5004 RTP/AVP 96

i=Channels 1-8

a=rtpmap:96 L24/48000/8

a=sendonly

a=ptime:0.250

a=ts-refclk:ptp=IEEE1588-2008:39-A7-94-FF-FE-07-CB-D0:0

a=mediaclk:direct=2216659908




1..8 channels

48 (44.1/96) kHz

16/24 bit linear

IGMPv2

unicast/multicast

IPv4 (IPv6)

Default Profile


Network

Encoding48 samples

(6/12/16/192)

Max payload

1440 bytes

125/250/333μs

1 or 4ms

Packet Setup

RTP/AVP

UDP

IP

Transport

From PTP

Local

Media Clock

Generation

AES67

Technology

Components

Encoding & Packet Setup

• Payload format defines audio sample encodings

• Both L16 & L24 bit linear format at 48 kHz sampling mandatory

• Payload formats from RFC3551 (L16) & RFC3190 (L24)

• Receivers both, senders – either or both

• At 96 kHz sampling, L24 supported by both senders and receivers

• At 44.1 kHz sampling, L16 supported by both senders and receivers

• Packet time – real-time duration of media in packet

• Determined by sender and sent via SDP

• 1 ms is the mandatory requirement

• 48 samples at 48 or 44.1 kHz - 96 samples at 96 kHz

• Support by both senders and receivers



• Recommended and required packet times


Packet time Packet samples

(48 kHz)

Packet samples

(96 kHz)

Packet samples

(44,1 kHz)

Notes

“125

microseconds”6 12 6 Compatible with class A AVB transport

“250

microseconds”12 24 12

High-performance, low-latency operation. Interoperable

with class A and compatible with class B AVB transport.

“333 microseconds

”16 32 16 Efficient low-latency operation

“1 millisecond” 48 96 48Required common packet time for all devices adhering to

this standard

“4 milliseconds” 192 n.a. 192

For applications desiring interoperability with EBU Tech

3326 or transport over wider areas or on networks with

limited QoS capability


• Stream channel count

• maximum number per stream limited by the packet time, encoding format and

network MTU


Format, sampling rate Packet time Maximum channels per stream

L24, 48 kHz 125 microseconds 80



L24, 48 kHz 333-1/3 microseconds 30


L24, 48 kHz 1 millisecond 10

L24, 48 kHz 4 milliseconds 2



Best Effort DF (0)

Media AF31 (34)

Clock EF (46)1..8 channels

48 (44.1/96) kHz

16/24 bit linear

IGMPv2

unicast/multicast

IPv4 (IPv6)

Default Profile


Network

Encoding

DiffServ/DSCPQuality of Service

Connection

Management

SIP/SDP (unicast)

Multicast (IGMPv2/SDP)

48 samples

(6/12/16/192)

Max payload

1440 bytes

125/250/333μs

1 or 4ms

Packet Setup

RTP/AVP

UDP

IP

Transport

From PTP

Local

Media Clock

Generation

AES67

Technology

Components

Connection Management - 1

• Exchange of information describing stream characteristics & connection

information – SDP

• For multicast connections AES67 doesn’t specify transport method of SDP – has

to be done manually between different systems

• Different methods are used:

• RAVENNA – RTSP – automatic within own domain

• Axia Livewire – RTSP – automatic within own domain

• Dante – SAP – no manual means for SDP read in or out

• For unicast connections AES67 uses SIP for connection management and to

transport SDP

• AMWA IS-05 potentially offers connection management for both multicast and

unicast


Connection Management - 2

• RAV2SAP© tool – a temporary windows application solution for connection

between nodes from other manufactures/systems

• AES70 – architecture for system control and connection management - OCA

• Addresses device control and monitoring only – streaming media standards

• Network Classes are foundation for many media transport networks

• Specific adjustments may be required to support particular media transport types –

AES70 Adaptation – configuration rules for the network classes

• AES67 Adaptation specification - in progress - provide media connection

management services for devices that implement AES67 media transport.




Best Effort DF (0)

Media AF31 (34)

Clock EF (46)

NMOS IS-04

WheatNetIP Discovery Protocol

Axia Discovery Protocol

Bonjour

SAP

1..8 channels

48 (44.1/96) kHz

16/24 bit linear

IGMPv2

unicast/multicast

IPv4 (IPv6)

Default Profile


Network

Encoding

DiffServ/DSCP

PossibilitiesDiscovery

Quality of Service

Connection

Management

SIP/SDP (unicast)

Multicast (IGMPv2/SDP)

48 samples

(6/12/16/192)

Max payload

1440 bytes

125/250/333μs

1 or 4ms

Packet Setup

RTP/AVP

UDP

IP

Transport

From PTP

Local

Media Clock

Generation

AES67

Technology

Components

AMWA Discovery/Connection Management

• AMWA – Advanced Media Workflow Association Networked Media Open

Specifications (NMOS) – a set of APIs

• NMOS IS-04 Discovery - discover and register devices as they are connected to

the network, enabling their subsequent connection through the ……

• NMOS IS-05 Connection Management - allows the configuration of connections

between Senders and Receivers

• Both of the above have been tested for discovery and connection management

of AES67/ST 2110-30 devices

• IS-05 was also tested with some audio manufacturers at a recent AMWA workshop,

also part of IBC IP showcase of IS-05

• NMOS IS-06 Network Control – in progress - viewable network topology,

allows creation/retrieval/update/deletion of flows in the network between

endpoints. Includes related monitoring and diagnostics.


AES67 Interop PlugFests

• Plugfests organised to confirm interoperability in

commercially neutral environment

• Users benefit from compliant equipment that will connect

together regardless of manufacture

• Three held so far - 2014, 2015 & 2017

• Increasing numbers with each one

• Additional new manufacturers appear

• AES Reports from each

• Fourth in planning for 2018



Munich Oct 2014

Institut für Rundunkunktechnik


Munich Oct 2014

Institut für Rundunkunktechnik

• 10 Manufacturers

• 16 variable product types - software on a PC to hardware-based FPGA solutions – all

based on extensions of existing networked-audio products

• Mark Younge, AES Standards Manager acting as chair

• providing technical assistance and independent observers


Washington DC Nov 2015

• 11 companies - same as Munich plus these:

• Meinberg Radio Clocks GmbH & Co. KG

• QSC LLC

• Wheatstone Corporation

• Yamaha Corporation

• 13 products

• In Munich – a number of products based on prefabricated

hardware and firmware sub-systems

• In Washington products were largely-independent

implementations of AES67.


Washington DC Nov 2015


London Feb 2017

New Broadcasting House

• 24 companies - same as Washington plus these:

• A.R.G ElectroDesign Ltd Audinate

• Audio-Technica U.S., Inc. Bosch Security Systems

• Calrec Coveloz

• Genelec Imagine Communications

• Riedel Communications Shure Inc.

• Sonifex Tektronix

• Thum+ Mahr+ Gmbh

• 36 products

• Software implementations on PC to hardware-based FPGA solutions.

• Products - prefabricated hardware, prefabricated firmware sub-systems

and bespoke implementations of AES67

• Four NMOS 1S-04 implementations tested in AES67 devices – 75%

success rateAES67 audio over IP withinYamaha SMPTE 2110

London Feb 2017



London Feb 2017


Broadcasting Examples – BBC Internet Fit Studio


Broadcasting Examples – BBC Internet Fit Studio

DIRA playout system

VM backend and frontend

processes

Sound card Interface feeding Axia

sound card driver process

Uses just the mandatory common

AES67 settings of L16/L24, 48Khz,

1ms


Broadcasting Examples – BBC Wales/Cymru

Cardiff Central Square

• New Build in planning process – predominantly IP based

• Extensive use of• AES67 in combination with 2110 for video/audio

• AES67 (and Dante) within audio domains

• Hence PTP (locked to GPS) across network

Although some legacy SDI & AES3 I/O and Black & Burst

• 5012 audio sources, 6097 audio destinations (mono)• 855 video sources, 1169 video destinations

• Live IP for core routing of all video and audio

• SMPTE 2110 elemental streams (2110-10, -20, -30 & -40)

• NMOS IS-04 discovery and NMOS IS-05 connection management


Broadcasting Examples – Swedish Radio – NXG Project

Simple to produce radio and create ability to broadcast anywhere, anytime

• Production• Content fast from anywhere

• Contribute live from/to studio anywhere

• Broadcast live, self op from anywhere

• Workflow• Intuitive on-air equipment

• Modern studio environment

• Create conditions & support mobile workflow

• Engineering

• Regional & centralised data centre

• Create scalable end user system

• System integration – ACIP & cloud computing


http://sverigesradio.se/sida/artikel.aspx?programid=4042&artikel=6648515


• Public Tender• Non-technical

• Based on production scenarios

• Mutual contract with LAWO in 2014

• The project combines AES67 and ACIP together with an in-

house developed front-end mixer user interface, based on

touch technology and IT services

• Deployment• 1st station Q4 2016

• 2nd station Q4 2017




• Regional Data Centre• Network audio hub in Gothenburg

• No local data centres

• Cost, maintenance access, shared

resources, optimise unit storage

• Mix core engines

• Network/routers, PTP GM (based on

AES67)

• Playout and recording services

• IP audio codec pool

• TX on air, STL & processing

• Centralised Data Centres in Stockholm• Private Cloud, VMWare

• Back end services

• SIP/CCM environment

• PTP GM Cluster – buys PTP over Ethernet

• Network audio hub in Gothenburg

• In development 2017 - ?• Playout & recording

• Software based IP audio codec pool



• Networking• With industry – AES, EBU, MNA - shows IBC, ISE, NAB

• IP Any-to-Any!! – Have to have this!

• Connectivity• LAN/WAN

• EWAN (Leased Line)

• Satellite

• 3G/4G diversity

• Public Internet

• QoS Scheme

• AES67 in office VRF/VLAN – replace internal sound card

with virtual card (AES67)

• PTP – Precision Time Protocol

• Directory Services



• Interoperability• AES67 – AES70 – EBU ACIP II (3326/3368)

• System Integration & UI via LAWO

• Mixing/DSP• Control onto touchscreens – with buttons

• Group audio sources to increase no. of sources

• On-air functions on panel

AES67 What Next?

Latest version AES67-2017 expected to be published very

soon

Includes

Protocol Implementation Conformance Statement (PICS)


AES67 What Next?

• AES70 for device control, network connections and

discovery?• Archwave are combining AES67 and AES70 as a solution -

AudioLAN2.0. Audio and control streams plus remote

control

• AMWA NMOS APIs:

• NMOS IS-04 Discovery – about to be included as one of

the (informative) discovery mechanisms for AES67

• Include NMOS IS-05 for connection management?

• Include NMOS IS-06 (in development) for network control?


SMPTE 2110-30 Summary

• Streams shall conform to AES67 – RTP based PCM digital audio only +

SDP metadata (RFC4566)

• Mandatory requirements for all devices (senders and receivers):• 48kHz sampling (Media & RTP clocks same as sampling rate)

• Compliance of media & RTP clocks with sections of 2110-10

• Should support 44.1 kHz and/or 96 kHz sampling

• 1ms packet time

• 1..8 channels per stream

• 16 & 24 bit depth (RFC3551 & RFC3190)

• Timing & buffering provisions of AES67

• Compliance of RTP timestamp with 2110-10

• Outside mandatory requirements – read 2110-30 & AES67 specs carefully

• Be aware of differences within 2110-30 and pure AES67 domain


Thank you

bbc.co.uk/rd

[email protected]

Email:

AES67 audio over IP within SMPTE 2110 - We are SMPTE | …€¦ · · 2018-01-03to IP networking. SMPTE ST 2110 ... campuses, stadiums, event venues and local area IP networks.

Documents