An Introduction to Freescale - NXP Semiconductors...Low footprint, high performance package of integrated voice technologies optimized for automotive applications. HD voice support.

External Use

TM

An Introduction to Freescale

Audio Solution Framework, Part 1

FTF-SDS-F0130

A P R . 2 0 1 4

Jason Martin | Software Manager

Rudan Bettelheim | Business Development Manager

TM

External Use 1

Agenda

• Introduction to the Freescale Audio Solution

• Freescale Audio Solution Architecture

• Multimedia Framework

• System Tasks

• Bluetooth and Acoustic Echo Cancellation

• HMI Framework

• Third Party Software Integration

• Startup Timing and Fast Boot

• Platform Configuration and Building

• Platform Support and Professional Services

TM

External Use 2

Freescale Audio

Solution Introduction

TM

External Use 3

Freescale Audio Solution Platform

The Freescale Audio Solution platform is being developed for the

Kinetis, Vybrid, and i.MX processors. The platform is made up of

functional blocks to deliver media player functionality.

• Multimedia Framework with Radio Tuner and CD Support

− Support for industry standard audio CODECs

• CE Device Connectivity (MFi, USB, SD, AOAP)

• External device management and control

• Input management and control

• HMI Framework

• Bluetooth Stack and Profiles with AEC/NS (HFP, A2DP, AVRCP)

TM

External Use 4

Freescale Audio Solution OS Support

Which Operating Systems will be supported?

− The Freescale Audio Solution platform leverages the Freescale

Operating System Abstraction layer provide platform support on MQX

and Linux*

− Through extensions to the OSA, FAS can be modified to work with any

modern operating system that supports common OS primitives (memory

management, signals, mutexes, etc)

Operating System (MQX, Linux, QNX, etc)

Operating System Abstraction (OSAL)

Freescale Audio Solution Platform

Customer Application

TM

External Use 5

Freescale Audio Solution Support

Which hardware will the Freescale Audio Solution platform support?

− Production level code will be available on the Vybrid automotive EVB,

Kinetis Tower System module, and i.MX6 SABRE ARD development

boards.

Which Compilers will the FAS platform support?

− FAS platform will provide support for the following compilers

GNU GCC

IAR

− Support for porting of the FAS platform to a new compiler can be

provided by the Freescale Professional Services Team.

TM

External Use 6

Freescale Audio Solution

Architecture

TM

External Use 7

Freescale Audio Solutions Platform Architecture A

ud

io S

olu

tio

n

So

ftw

are

Hardware

Low Level

OS

T

M

MCU /MPU

USB I2S Ethernet SD

DSP

UART

Memory DMA

I2S Analog Audio

HW Codec/Class-D/FM Modulator

USB

Stack TCP/IP

Stack

SD

Driver

Audio

Drivers Serial

Drivers

I/O

Drivers

MQX Linux

Media

Player

Media

Browser

Media

Indexing

Playlist /

Play

Queue

Customer Applications

FAS Framework

Common Audio Framework API

Media

Device

Support

OS Abstraction Layer

Additional 3rd Party OS

Support (uItron)

Refe

rence iO

S / A

nd

roid

Ap

ps

Platform

Teams AA Team 3rd Party

Devic

e M

an

age

r

IIn

pu

t M

an

age

r

TM

External Use 8

Freescale Audio Solution Framework

What is Maestro?

− Maestro is the Multimedia Audio Framework in the FAS platform

− Maestro does not provide video playback support, today

Which Multimedia CODECs will Maestro Support?

− Maestro will support the following multimedia formats

MP3

AAC

WMA

OGG Vorbis

WAV

Which Consumer Electronics Devices will the Freescale Audio Solution platform support?

− Full support for Apple devices (USB only)

− Android AOAP (USB)

− FAT16 / FAT32 / NTFS* File system support on the following devices

Support for USB mass storage class

Support for SDHC devices

− Bluetooth A2DP with AVRCP

TM

External Use 9

Architecture Diagram Legend

TM

External Use 10

Maestro Audio Framework Architecture

Maestro

Metadata Manager

External FS Devices

SD Card

USB MSC

NAND Mem

Audio

Processor

Object

Audio Player Application Input Manager Device

Manager

MFi

Stack

Browse

Object

AOA

Stack

Play Queue

Object

Player Audio

Object Play Queue

Settings

HW Codec

MediaDev Android Apple Filesystem

PCM Manager (MQX) Or ALSA

(Linux)

DSP/Codec Ctl

External Device(USB, I2S)

AudioOut data Streamer

CD

TM

External Use 11

Player Object

• Player Object will create the streamer for all playback objects

• For file system based devices, the streamer will also control playback functionality

• For other devices, mediadev provides abstraction to allow the same control interface across all devices

• Player Objects can be associated with play queue objects for track selection without interacting with the application, or directly by the application

• Streamer provides audio post processing hooks.

TM

External Use 12

Media Browse Object

• Media Player Application will create the browser object.

• The browse object (via mediadev) abstracts the browsing functionality from the player. The same API is used regardless of the device type and if it’s indexed or not.

• Filesystem based devices can either be browsed by metadata stored in the database, via indexed file system tree, or directly from the filesystem

• Apple devices will only support metadata browsing.

• Mediadev_create() makes the decision of how to browse the device when populating the virtual function table.

TM

External Use 13

Play Queue Object

• The Play Queue can store a list of tracks to be played.

• The Media Player Application is responsible for all “writes” to the play queue, which represents the active playlist.

• The application must also associate a play queue with a player object in order for player object to automatically navigate the playlist without on-demand interaction from the application.

• Low memory play all possible through use of internal play queue management.

• Also provides support for industry standard playlist formats: pls, m3u, asx, and wpl.

TM

External Use 14

Media Device Layer

• Mediadev is the key module that

provides abstraction across all external

devices for playback control and

browsing.

• The tuner use case is considered

unique enough that it will not be

abstracted through the mediadev. The

tuner object will operate directly on the

tuner device.

TM

External Use 15

Mediadev Architecture

• Mediadev session based on the device type will be created by the application.

• The virtual table is populated during session creation.

• The application will provide the mediadev session to the player and browse object.

TM

External Use 16

Playback/Browse Architecture – File System devices

TM

External Use 17

Playback/Browse Architecture – Apple devices

TM

External Use 18

Device Manager

• The device manager is an asynchronous task / process that detects

the presence and removal of consumer electronic devices.

• Detects the presence and removal of USB mass storage devices,

SD flash memory cards and Apple USB devices

• Reports the presence of an external device to the application for

use by the media player

• Responsible for determining if a device has been indexed by the

metadata manager for a richer file system browsing experience

TM

External Use 19

Metadata Manager

TM

External Use 20

Metadata Management

• Indexing of metadata initiated on device insertion

− FAS will recognize previously-used devices and retain metadata

information

• Device browse by file system available while indexing

• Metadata information stored in SQLite database

• API provided to query metadata information from database engine

for browse and display

• Database queries optimized for fast retrieval time and low heap

memory usage by utilizing table indices

• Metadata strings stored as UTF-8 for i18n support

− Custom data collation available for specific language support

TM

External Use 21

Tuner Object

TM

External Use 22

Audio Pipelines

External Memory

USB Flash Drive

SD Card

Apple Device

Android Device

USB Device

Analog Input

USB MSC FAT

SDIO FAT

NFC FAT

File Source

File Source

File Source

MP3

WAV

WMA

FLAC

AAC

HE AAC

APE

SDIO FAT

NFC FAT

File Destination

File Destination

File Destination

MP3

WAV

Audio Source

PCM Mgr

Audio Source PCM Mgr

Audio Sink

PCM Mgr

USB Audio-In

PCM Mgr

SAI Rx Driver

PCM Mgr

External Memory

USB Flash Drive

SD Card

Codec SGTL5000 DA7321

Codec SGTL5000 DA7321

USB

MSC

FAT

ASP Unit

Volume Ctrl

Tone Ctrl

Balance Ctrl

EQ

Loudness

Stereo Widening

Dolby

DTS/SRS

Audio Sink

PCM Mgr

USB Device USB

Audio-Out

PCM

Mgr

Queue FSRC

Queue FSRC

SAI Tx

Driver

PCM

Mgr

Driver or none Audio

Code

PCM Mgr or ALSA

External Device

Maestro

TM

External Use 23

System Tasks

TM

External Use 24

Startup Timing and Fast Boot

TM

External Use 25

Startup Timing and Fast Boot

Requirement Timing (mS)

Response to CAN message 50

Display Splash Screen 500

HMI Available1 1000

Radio Tuner / CD Playback 1000

MFi Playback 5000

USB Playback2 5000

1 HMI available playback based on previously indexed or file system based

playbackty is based on the complexity (size) of the HMI 2 USB, based on USB 2.0 device type

TM

External Use 26

Startup Timing and Fast Boot

MQX RTOS Based platform

• Low overhead boot sequence

• Ability to run directly from flash

• Minimal bootloader required

• Ability to initialize and execute CAN handling early in the boot sequence from “userspace”

Linux Based platform

• Requires heavier bootloader for loading of Linux Kernel

• Ability to run directly from flash

• Partition and transfer essential portions of code to SRAM

• Removal of non-essential Kernel functionality

Fast Boot Optimizations

• Ensure highest possible throughput on flash memory devices

• Optimized startup sequence

TM

External Use 27

Third Party Software Integration

TM

External Use 28

Freescale Audio Solution Software Partners

Software Package Partner

HMI Framework Altia

Bluetooth Cybercom / iAnywhere

AEC / NS Alango

TM

External Use 29

Develop Concept

• Altia PhotoProto and FlowProto

• Make graphics interactive, navigable, and ready to turn into a complete user interface in Altia Design

• Verify the behavior of your model

• These offer the first “partial model” usability studies with users

TM

External Use 30

Build GUI

• Easily organize hundreds of screens, buttons, lists and images

• Produce exciting animations easily

• One complete simulation model

• Integrate your GUI with other tools to simulate your whole application

TM

External Use 31

Generate Code

• The artist’s concept is

deployed on the final model

• No loss of concept

• Performance expectations can be verified

TM

External Use 32

A robust, generic and

portable multi-profile

Bluetooth SW application

framework targeting the

automotive market

A framework encapsulating

Bluetooth

complexity with focus on two

things:

=> Simplicity

=> Interoperability

”We Love Making Bluetooth Easy”

blueGO – What is It?

TM

External Use 33

• Built on top of market

leading iAnywhere Blue

SDK

• Tested with hundreds

of phones

• Close cooperation

with phone

manufacturers

• blueGO used in phone

manufacturers testing

• PTS used for design

testing

blueGO – ”Interoperability-off-the-shelf”

TM

External Use 34

Sybase iAnywhere Bluetooth® Stack

Top Benefits

• 12 years maturity and evolution

• Seventh generation stack

• Customers and market feedback for continuous reliability

improvements

• Continuous participation and leadership in Bluetooth SIG and

various working groups

• Ported to QNX, Android, Linux, Windows, MQX

• Unique experience of the Automotive Market

TM

External Use 35

Advantages Using Sybase iAnywhere Products

• Small code base enables memory-limited implementation

• Portable C ANSI Source Code

• O/S, Processor and Radio/Transceiver/Transport Independent

• Sample Applications

• Comprehensive & full documentation

• Software Maturity & Reliability

• Certified compliant by IrDA, Bluetooth® SIG, OMA

• Short time-to-market by reducing development costs

• Enables developers to focus on applications, data transports and

system integration

TM

External Use 36

Bluetooth Stack SDK

Architecture

TM

External Use 37

Acoustic Echo canceller : Eliminates linear and non-linear acoustic echoes enabling full-duplex communication

Dynamic Range Compressor : Improves speech intelligibility, decreases loudspeaker distortions

Noise Suppressor : Suppresses stationary and transient noises (traffic, engine, passing cars, etc)

Automatic Gain Control : Compensates for possible changes in signal levels

Automatic Volume and Equalization : (Optional) Amplifies and equalizes loudspeaker signal according to the cabin noise.

Adaptive Dual Microphone : (Optional) Attenuates all types of noises for near field and far field cases

EasyListenTM : (Optional) Improves intelligibility by slowing down incoming speech in real time

Frequency equalizers : 125Hz bandwidth equalizers for precise microphone and speaker equalization VCP

Ba

seb

an

d, p

roto

col

sta

ck a

nd

RF

Acoustic Echo

Canceller

Automatic Volume and Equalization

AGC

DRC

Dual microphone (optional)

Noise

Suppressor

Voice Communication Package

Noise

AGC

DRC

Basic VCP VCP add-ons

Easy

Listen

Acoustic

echo

Voice

Noise

Suppressor EQ

EQ

VCP add-ons

Acoustic profile Log signals and data

Voice Communication Package (VCP) Low footprint, high performance package of integrated voice technologies optimized for automotive applications. HD voice

support. VDA 1.6 compliance. Fast echo/noise convergence . Signal logging capabilities.

TM

External Use 38

Freescale Audio Solution

Support and Services

TM

External Use 39

So

ftw

are

OS Kernel

FSL BSP

Drivers

H

ard

ware

MS

S P

rim

ary

Fo

cu

s A

rea

UI &

Applications

Vybrid

Solutions

i.MX

Solutions

MSS Architecture Focus Area

Middleware

Kinetis

Solutions

Professional Services Focus

Software Services

• Linux, Android, MQX BSP’s, Audio Framework − Customization

− Integration

− Development

− Porting

− Testing

− Optimization

− Issue analysis, Debug & Fix

• Long-term support on Customer Board

Hardware Services

• Schematic and layout optimization & review

• Simulation

• On-site board bring-up

TM

External Use 40

define design develop integration

and test deploy

• Requirements

Gathering

• Project

Assessments

• IP Selection

• Design

Consulting

• System

Consulting

• Architecture

• Custom

Development

• Porting /

Migration

• Integration

• Testing

• Documentation

• Performance

Testing

• Performance

Optimization

• Knowledge

Transfer

• Project

Specific

Support

(3rd Level)

• Maintenance

• Performance

Tuning

Quality Assurance (ISO, CMMI & SPICE Level III)

Services Development Methodologies & Quality:

Engineering Discipline

TM

© 2014 Freescale Semiconductor, Inc. | External Use

www.Freescale.com