Audio Video Bridging on i.MX Applications Processorsread.pudn.com/downloads699/doc/comm/2814014/FTF-SHB-F1211.… · 1722a – AVTP (Draft) ... − Represents 125 µs worth of audio
Post on 28-Sep-2020
1 Views
Preview:
Transcript
External Use
TM
Audio Video Bridging on i.MX Applications ProcessorsFTF-SHB-F1211
J U N E . 2 0 1 5
Rui Sousa | Software EngineerDavid Babin | Hardware Engineer
TM
External Use 1 #FTF2015
Agenda
• AVB Overview
• Hardware Requirements
• AVB Implementation Challenges
• AVB Software Stack
• AVB Hardware Platforms
• AVB Hardware Media Clock Capture/Recovery
• Demo Platform
• Q&A
TM
External Use 2 #FTF2015
AVB Overview
TM
External Use 3 #FTF2015
Audio Video Bridging (AVB)
• Definition: Set of standards, protocols and technologies for the transport of time sensitive streams (audio, video and control) in bridged LANs
• Purpose: Replace existing analog and multi-standard digitized stream transport with a commonly deployed technology, offering:− Bounded and low transport latency
− Reliable delivery (with guaranteed bandwidth and network resources)
− Time synchronization
• Advantages− Can replace complex and costly wiring with a single and existing technology:
Ethernet
− Based on open Standards
− Coexists with legacy IP data traffic
− Self-managed
TM
External Use 4 #FTF2015
Audio Video Bridging Standards
IEEE Standards (https://www.ieee.org)
• 802.1BA-2011 AVB (Audio Video Bridging)
• 802.1AS-2011 gPTP (Generalized Time Precision Protocol)
• 802.1Qat-2010 SRP (Stream Reservation Protocol)
• 802.1Qav-2009 FQTSS (Forwarding and Queuing of Time Sensitive Streams)
• 1722-2011 AVTP + MAAP (Audio Video Transport Protocol, MAC Address
Acquisition Protocol)
• 1722.1-2013 AVDECC (Audio Video Discovery, Enumeration, Connection
Management and Control)
• 1722a D13 AVTP
TM
External Use 5 #FTF2015
Audio Video Bridging Standards
AVnu Alliance (http://www.avnu.org)
• Community promoting AVB/TSN technologies in an interoperableecosystem
• Certification program
• Market specific workgroups− Automotive
− Professional Audio/Video
− Consumer Electronics
− Industrial
• Automotive profile
TM
External Use 6 #FTF2015
802.1AS gPTP
• Provides time synchronization across bridged LAN networks− Within 1us peak to peak for 7 hops
• Based on IEEE 1588-2008 PTPv2
• gPTP domain with several interconnected time aware systems (end stations and bridges) synchronized to a grandmaster clock
• Best Master Clock Algorithm (BMCA)− Network is able to select best source of timing (based on ANNOUNCE
messages)
• Propagation delay measurement between peers of a point-to-point link− Media dependent (802.3 -> Pdelay Requests/Responses)
• Propagation of Synchronization messages− Bridges update SYNC messages based on peer delay measurements and
own delay (residence time)
TM
External Use 7 #FTF2015
802.1Qat - SRP
• Provides reservation of bandwidth and resources along the network path between Talker and Listener(s)− Bounded maximum latency− Reliable delivery (if reservation is successful)
• Two traffic classes− Identified by VLAN priority and ID− Class A (2ms transit time, 125us observation interval)
− Class B (50ms transit time, 250us observation interval)
• Talkers declare available streams and their properties• Listeners declare their intention to receive a stream• Bridges adjust/merge/propagate declarations
− Possibly changing Talker/Listener declarations if reservation fails
TM
External Use 8 #FTF2015
1722 - AVTP
• Defines transport protocol for Audio & Video− Layer 2 protocol over 802.3, 802.11, …− Data, control and timing
• Several media formats supported (based on IEC 61883)− Raw audio (IEC-61883-6)− MPEG2-TS (IEC-61883-4)
− …
• AVTP presentation time− Based on gPTP
− Used to synchronize presentation time (across several Listeners)
− Used to recover media clock
TM
External Use 9 #FTF2015
1722a – AVTP (Draft)
• New backwards compatible revision• Several new/native media formats introduced
− AVTP audio format (AAF, Raw audio, Int, Float, 16/24/32bit, 8-192 KHz, …)
− Compressed Video format (H264, MJPEG, JPEG2000)− SDI Video format− Raw Video format
• Improved distribution of clock timing (not limited to media clock)− Dedicated clock reference format (CRF) stream− Several timestamps per packet
• AVTP control format• Encrypted data transport encapsulation• Encrypted/Authenticated control transport encapsulation
TM
External Use 10 #FTF2015
1722.1 - AVDECC
• Provides the ability to discover devices on the LAN, enumerate their properties, manage their connections and control them
• Talker/Listener/Controller Entities• Formal/Full description of Entity capabilities (Entity model):
− Descriptor based
− Stream inputs/outputs
− Analog/Digital Audio/Video inputs/outputs
− Media formats (sample rates, channels, aspect ratio, frame sizes, codecs, …)
− Clock domains
− Signal routing
− …
• Talker/Listeners Advertise their presence and answer to commands• Controllers Discover/enumerate Talker/Listeners and issue commands
TM
External Use 11 #FTF2015
AVnu Automotive Profile
• Objectives− Reduced start up time
− Reduced transport overhead
− Reduced end station load
• gPTP− No BMCA, fixed port role, fixed grandmaster, stored delay measurements
• SRP− Fixed configuration
− New traffic classes: C (15ms transit time, 1.333us observation interval), D (15ms transit time, 1.451us)
• AVTP− Reduced number of supported formats (AAF, Compressed Video, 61883-6)
• AVDECC− Not used
TM
External Use 12 #FTF2015
Audio Video Bridging Network
• Talker end stations
• Listener end stations• Talker & Listener end stations
• AV Bridges
• gPTP domain− gPTP Master
− gPTP Slave(s)
• SRP domain(s)• AVB domain(s) (gPTP ∩ SRP)
• Media clock domain(s)− Media clock master
− Media clock slave(s)
TM
External Use 13 #FTF2015
Audio Video Bridging Network Streams
Talker end station AV BridgeListener end
station
AV Bridge AV Bridge Talker end stationTalker end station
Listener end station
Listener end station
Talker & Listener end station
TM
External Use 14 #FTF2015
Audio Video Bridging Network gPTP
gPTP Master AV Bridge gPTP Slave
AV Bridge AV Bridge gPTP SlavegPTP Slave
gPTP Slave gPTP SlavegPTP Slave
TM
External Use 15 #FTF2015
Audio Video Bridging Network Media clock
Media clock slave
AV BridgeMedia clock
slave
AV Bridge AV BridgeMedia clock
slaveMedia clock
Master
Media clock slave
Media clock slave
Media clock slave
TM
External Use 16 #FTF2015
Audio Video Bridging Network Automotive
Talker Left Mirror Camera AV Bridge
Talker Right Mirror Camera
Listener RSEListener RSE
Talker Rear View Camera
Talker & Listener Head Unit
TM
External Use 17 #FTF2015
Audio Video Bridging Network Pro Audio
AV BridgeListener
LoudspeakerListener
Loudspeaker
Talker & Listener Mixing table
Talker & ListenerMic & Monitor
AV Bridge
AV Bridge
AV Bridge
Listener Loudspeaker
Listener Loudspeaker
Talker & ListenerMic & Monitor
Talker & ListenerMic & Monitor
Listener Recording
TM
External Use 18 #FTF2015
Hardware Requirements
TM
External Use 19 #FTF2015
Hardware Requirements
• End stations− 802.3 Ethernet� Full duplex 100Mbps
� 1588 (PTPv2) MAC support
− 802.11 Wireless� 802.11n + 802.11v
• Local clock source (+-100ppm, 40ns granularity)• Media clock recovery (highly recommended)• Ethernet Rx classification/multiqueue (optional, Listener)• Tx credit based shaper (per stream)/multiqueue (optional, Talker)
TM
External Use 20 #FTF2015
Hardware Requirements
• AV Bridges− gPTP support− FQTSS support
− SRP support
• Automotive profile− Static SRP configuration
TM
External Use 21 #FTF2015
AVB Implementation Challenges
TM
External Use 22 #FTF2015
AVB Timing Challenges
• AVB defines low latency Class A traffic with a maximum latency of 2 msover 7 hops
• AVB defines higher latency Class B traffic with a maximum latency of 50 ms over 7 hops
• AVB 61883-6 Audio Format suggests “A Class A isochronous packet is generated every 8 kHz”− Represents 125 µs worth of audio− For an audio CD type (16 bit sample, stereo) stream @ 48 kHz, this means 6
samples � 24 bytes of payload
• The talker must send media data at 125 or 250 µs interval (extra slide)• The listener must provide 2 or 50 ms buffer capacity (worst cases of “must
wait” before presenting samples), and deliver on time at the Class interval rate
• Consider video playback scenario (extra slide)
TM
External Use 23 #FTF2015
Shaping for Multiple Streams (IEEE 802.1Qav)
Class A Credit-based
Shaper
Class B Credit-based
Shaper
Static P
riority
Ethernet M
AC
Best Effort Traffic
Software Scheduler
Streaming Source #A
Streaming Source #B
Streaming Source #N
…
Typical AVB Ethernet IPwith Class-based Shaper
• Class A/B traffic needs to be shaped/scheduled with a 125/250 µs interval rate
• Class queuing requires that only MaxIntervalFrames of each stream may be scheduled within one scheduling cycle (125 µs for class A, 250 µs for class B)
⇒ The software scheduler must run at the class observation interval of 125 or 250 µs
“stream-based” Ethernet AVB hardware IP, e.g. on i.MX 6SLX or MPC5748G, reduces the shaping processing load (support hw queues per traffic class with „Launch Time“ capability)
TM
External Use 24 #FTF2015
Video Streaming Scenario Challenge
• Scenario: Head unit streams movie to the Rear Seat Entertainment unit:
− Video data is received by the RSE, audio data by the Ethernet amplifier
− Consumer video (e.g. H.264 main profile) decompression has some very significant latency (typically 50-100 ms)
− Audio and video need to be “lip synced” with ~ 10 msaccuracy
⇒Some significant amount of audio data needs to be stored temporarily
� Option A: In the Ethernet amp � larger amount of RAM required
� Option B: Streaming source (head unit) needs to buffer audio sufficiently and send audio delayed compared to video
Amplifier
HeadUnit
Rear Seat Entertainment
Ethernet Switch
Video D
ata
Audio D
ata
TM
External Use 25 #FTF2015
Media Clock Challenges
• Cross Time stamping at the Talker side (outgoing stream)
Source: Dave Olsen
Timestamp generation must be made without latency (matching the
exact sampling operation)
TM
External Use 26 #FTF2015
Media Clock Challenges
• Reconstruction at the Listener side (incoming stream)
Source: Dave Olsen
Clock regeneration must be made without latency (at the perfect
matching time between timestamps
and gPTP time)
TM
External Use 27 #FTF2015
AVB Software Stack
TM
External Use 28 #FTF2015
AVB Software Stack
• Multi-platform− Runs on several Freescale platforms: i.MX, VFxxx− OS/Hardware independent core code, greatly reducing re-development
cost− Platform abstraction layer
• Scalable− Allows usage of subsets of the AVB stack to meet specific market needs− Offers variety of performance/cost levels according to the platform
• AVB Hardware Support− Able to run both on platforms with standard Ethernet IP or with Ethernet
AVB IP
TM
External Use 29 #FTF2015
AVB Software Stack
• All AVB components are provided (as defined by IEEE)− GPTP− MSRP, MVRP
− FQTSS
− AVTP− MAAP
− AVDECC
• Media Application Support− AVB library provides API for Media Applications
− Demo integration with ALSA & GStreamer, implementing Media Server, Audio Amplifier, Video Player…
TM
External Use 30 #FTF2015
AVB Stack Diagram
Control application
IEEE 1722
AVTPMedia
IEEE802.1QatSRP
Bandwidth Reservation
Ethernet IEEE 802.3 MAC
IEEE802.1Qav FQTSSQueuing & Shaping
IEEE 1722 MAAP
IEEE 1722.1AVDECC
AVB Management & Control
IEEE 802.1ASgPTP
Timing & SynchronizationTCP/IP
stack
Alsa/Gstreamer/…
DMA
I/O
Audio/Video Driver
Control API
Media API
Operating System
Media Clock
Recovery
Productized component
IEEE 1722 AVTP ControlOS
Abs
trac
t
HW
Abs
trac
t
TM
External Use 31 #FTF2015
AVB Stack Diagram
• Automotive Profile
Control application
IEEE 1722
AVTPMedia
IEEE802.1QatSRP
Bandwidth Reservation
Ethernet IEEE 802.3 MAC
IEEE802.1Qav FQTSSQueuing & Shaping
IEEE 1722 MAAP
IEEE 1722.1AVDECC
AVB Management & Control
IEEE 802.1ASgPTP
Timing & SynchronizationTCP/IP
stack
Alsa/Gstreamer/…
DMA
I/O
Audio/Video Driver
Control API
Media API
Operating System
Media Clock
Recovery
Productized component
IEEE 1722 AVTP ControlOS
Abs
trac
t
HW
Abs
trac
t
TM
External Use 32 #FTF2015
Linux AVB Stack (i.MX 6)
• Linux 3.10.17 with real-time extensions• Kernel patch for deep integration with ENET driver• AVB kernel module for system services
− Shared memory− Network tx/rx− Rx classification− Tx QoS scheduler− Media clock capture/recovery
• AVB userspace task/thread for each stack component− Linux specific event loop− Platform independent stack implementation
• Media interface driver + user library/API• Example media applications
− Audio (Alsa), Video & Audio (GStreamer)
TM
External Use 33 #FTF2015
Linux AVB Stack (i.MX 6)
gPTP
Linux PHC subsystem AVB Tx Shaping
ENET 1588 timer
ENET Low-level driver
Regular network stack
MAAP
PKT RX/TX
CLK control
Timer
HAL SRP AVTP
ENET PTP driver
ENET MAC
Media stack /
applicationOther
processes
Kernel
User-space
Media clock driver
AVB Rx bypassqueues
AVDECC
OS abstraction layer
Audio interfaces
SDMAMedia clock
Audio/VPU/GPU drivers
KernelMedia interface driver
AVB library
Videointerfaces
TM
External Use 34 #FTF2015
Platforms
• Primary target is i.MX 6Quad/6Dual, running Linux OS
• For secondary targets (SoCs and OSs)− i.MX 6SoloX, Linux OS Cortex-A5
i.MX 6 Series i.MX 6SoloX VFxxx Controller
ARM
Cortex-A9
ARM
Cortex-A9
ARM
Cortex-M4
ARM
Cortex-A5
ARM
Cortex-M4
Linux
OSX X X
Priorities• i.MX 6Quad/6Dual (ARM Cortex-A9) on Linux
OS for Automotive• i.MX 6SoloX (ARM Cortex-A9) on Linux OS for
Automotive/Prosumer Audio• VF6xx (ARM Cortex-A5)
TM
External Use 35 #FTF2015
AVB Stack Roadmap
Color Legend
Released
Committed
Roadmap Date
Original Release
Major Release
Version/Beta
Service Pack
2015 2Q 3Q 4Q1Q
Release Date: Apr. 2015
• Device/OS support
− i.MX 6, VF6xx
− Linux RT
• Features
− AVTP A/V packetizer
− FQTSS Class A, B, BE
− Media Clock Recovery
& Capture
− gPTP, SRP, AVDECC
− AVB API
• Evaluation Package
− Audio Sampler, Server, Amplifier with GUI control
− Video Server, multi-Players
− Audio/Video Sync
− ALSA & GStreamer
AVB Beta3
• New Features
− Multi-stream synchronization
− MJPEG video transport
− AVB API extended
• New Evaluation Package
− A/V Server, multi-Players with GUI control
− Camera source
• Evaluation Package
− Extended Media Applications, integrated withGStreamer and ALSA.
AVB Beta (Final)
• New Device/OS− i.MX 6 SoloX
• New Features− FSL gPTP− Completion &
Stabilization� AVTP, FQTSS, Media
Clock
� SRP, AVDECC
� AVB API
• New Evaluation Package− Extended A/V
application examples
• Development Package
− Yocto based
• New Device/OS
− Kinetis: K6x, K7x
(2016)
− MQX RTOS (2016)
• New Features
- Additional AVTP
formats for A/V, Control
streams
- MAAP
- Preparation for Avnu
Certifications
(Automotive, ProAudio)
• New Demo Application
− Surround Cameras
− Audio amplifier on Kinetis(Cortex M4/ MQX)
AVB
AVB 1.x
Target Date: June. 2015 Target Date: Sep. 2015 Target Date: Q4 2015, Q1 2016
AVB 1.0
TM
External Use 36 #FTF2015
AVB Hardware Platforms
TM
External Use 37 #FTF2015
AVB i.MX 6Quad/6Dual/6Solo
• MAC support for IEEE-1588• EPIT hardware timer for Tx scheduling• External media clock recovery PLL (Cirrus CS2000)
TM
External Use 38 #FTF2015
AVB i.MX 6SoloX
• MAC support for IEEE-1588• Tx credit based scheduler
− Three transmit queues (typically SR Class A, B and best effort)
− Timestamp launch time allows per stream scheduling
• Rx classification− Three receive queues (typically SR Class A, B and best effort)
− VLAN priority based
• EPIT hardware timer for Tx scheduling (lower rate)• Internal adjustable media clock PLL• GPT for media clock cross time stamping
TM
External Use 39 #FTF2015
i.MX 6 AVB Development Hardware
• SABRE Automotive Infotainment Rev2 CPU Card with i.MX 6Quad or i.MX 6DualLite
• New “Flexible Ethernet Expansion” Card with options for:
− Atheros Gb PHY (same PHY as on current SABRE hardware for software compatibility)
− 10/100 Broadcom BroadR-Reach ®
PHY
− 10/100 Broadcom BroadR-Reach ® 4-port switch
• First hardware available – 1st Half 2014
• Final boards available – January 2015
TM
External Use 40 #FTF2015
AVB Hardware Media Clock Capture/Recovery
TM
External Use 41 #FTF2015
i.MX 6Q/6DL Talker or Head Unit, Audio Master Clock = Local
L.O.24.576 MHz
CS2000Clk Sync
i.MX 6Q/6DL
Audio ClkInput
ESAI CODEC on Base Board
Audio ClkInput
Selector
Head UnitMaster Mode
ListenerSlave ModeRef Clk
I2C
Timing Ref(Clk In)
i.MX 6Q/6DL
1588_ Event_0_In
1588_ Event_0_Out
SDMA_Event_1 (input)
1588_Event2_Out
MCLK
MLE145170Programmable
Divider
SPI
750 Hz – 8 kHz
Test point scope
connections for
Events & MCLK
Selector
ListenerHead
Pulse Stretc
h
TM
External Use 42 #FTF2015
i.MX 6Q/6DL Listener, Audio Master Clock = Synthesi zed
L.O.24.576 MHz
CS2000Clk Sync
i.MX 6Q/6DL
Audio ClkInput
ESAI CODEC on Base Board
Audio ClkInput
Selector
Head UnitMaster Mode
ListenerSlave ModeRef Clk
I2C
Timing Ref
(Clk In)
i.MX 6Q/6DL
1588_ Event_0_In
1588_ Event_0_Out
SDMA_Event_1 (input)
1588_Event2_Out
MCLK
MLE145170Programmable
Divider
SPI
750 Hz – 8 kHz
Selector
ListenerHead
Pulse Stretc
h
Test point scope
connections for
Events & MCLK
TM
External Use 43 #FTF2015
• AVB Signal Assignment
i.MX 6Quad/6Dual
Ref AVB Associated Name MX6Q Ball & ALT Mode Comment
TP43 ENET_1588_EVENT0_IN ENET_TXD1 (ALT4) Legacy system function = MLB CLK
TP40 ENET_1588_EVENT0_OUT GPIO_19 (ALT1) Legacy system function = ENET INT_B
TP41 ENET_1588_EVENT2_OUT ENET_RX_ER (ALT4) Legacy system function = USB_OTG_ID
TP40 SDMA Event 1 DISP0_DAT17(ALT4) Legacy system function = RGB Disp (cluster)
TP42 MCLK, 24.576 MHz CLK2_P/N config’d as input Audio master clock
TM
External Use 44 #FTF2015
i.MX 6SX, External Chips Eliminated Due to On-chip AVB MCR
L.O.24.576 MHz
CS2000Clk Sync
i.MX 6SX
Audio ClkInput
ESAI CODEC on Base Board
Audio ClkInput
Selector
Head UnitMaster Mode
ListenerSlave ModeRef Clk
I2C
Timing Ref(Clk In)
i.MX 6SX
1588_ Event_0_In
1588_ Event_0_Out
SDMA_Event_1 (input)
1588_Event2_Out
MCLK
MLE145170Programmable
Divider
SPI
750 Hz – 8 kHz
Selector
ListenerHead
Pulse Stretch
Test point scope
connections for
Events & MCLK
TM
External Use 45 #FTF2015
Demo Platform
TM
External Use 46 #FTF2015
AVB Demo Platform
SABRE AI CPU board
SABRE AI CPU2 + Base
board
SABRE AI CPU2 + Base
board
SABRE AI CPU2 board
AVB Switch
0
1
2
3
Media Server
A/V Unit
Video Display
Audio Amplifier
Ethernet AVB
SD Card
LVDS
LVDS
Display
Display
Display
LVDS
TM
External Use 47 #FTF2015
Q&A
TM
© 2015 Freescale Semiconductor, Inc. | External Use
www.Freescale.com
top related