1
Android™ User-Space Tests Multimedia codec tests, status and open discussion
Ashok Bhat
System & Software, ARM
LCU14 – 502
September 2014
2
Use-case - Android Media Codec - AArch64 Port - Validation
Android user-space tests and benchmarks - Discussion
Do we need more?
How to create?
How to share?
How to deploy and maintain?
Agenda
3
Android Core Media Support
Type Format / Codec Encoder Decoder
Audio/ Speech
AAC LC • •
HE-AACv1 (AAC+) • •
HE-AACv2 (enhanced AAC+) •
AAC ELD (enhanced low delay AAC) • •
AMR-NB • •
AMR-WB • •
FLAC • •
MP3 •
Vorbis •
PCM/WAVE •
Video
H.263 • •
H.264 AVC • •
MPEG-4 SP •
VP8 • •
4
Application and Native Media Framework Interaction
Media Codecs
libstagefright.so
Application Framework
android.media.*
Binder IPC Proxies (frameworks/av/libmedia)
Media Player Binder MediaPlayerService.cpp
Media Player Service
(frameworks/av/media/libmediaplayerservice)
OMX Core
MediaSource.cpp
Stagefright Engine
(frameworks/av/libstragefright)
Native Multimedia Framework
(frameworks/av)
OpenMax Integration Layer
OMX Plug-in
Codec Implementation
OpenMAX IL Component
SOURCE : https://source.android.com/devices/media.html
5
Android Media Codecs
6
Codecs external to Android
Active upstream projects
Projects reside in external directory
Audio – AAC, FLAC, Vorbis
Video – VP8, VP9
Typically have validation suite outside Android
Codecs present only in Android
No upstream project
Projects reside in frameworks/av/media/libstagefright/codecs
Speech – AMR-NB, AMR-WB
Audio – MP3
Video – H.263, H.264, MPEG4
Rely on CTS for validation
Android Media Codecs
7
Decoder and Encoder need to conform to a standard
For example, MP3 decoder conforms to ISO/IEC 13818-3 and ISO/IEC 11172-3
For a decoder, standard provides a set of conformance clips and a reference decoder
For an encoder, standard may provide a set of input clips and a test criteria
Decoder test mechanism
Decode conformance test clips
Compare output with the output generated by the reference decoder
Output should be either bit-exact or within a specified range when compared to reference output
Encoder test mechanism
Encode input for different combination of encoding parameters
Check if encoded output conforms to the standard
Typical codec validation
8
Compatibility Test Suite
Part of Android compatibility program
Runs on a desktop machine and executes test cases directly on attached devices or an emulator
Does not claim to be comprehensive
Media codec validation
Part of android.media.cts package
Present in cts/tests/tests/media/src/android/media/cts directory
Tests are Java programs that run on top of the Android software stack
Android CTS
9
Need a method to validate AArch64 port
CTS is a good starting point. But...
Not meant to be comprehensive
Runs on top of Android stack – tricky to use for benchmarking component
Need something in addition to CTS that would
Allow validation/benchmarking/optimization at the component level.
Allow validation of conformance test vectors – preferably file I/O based.
Allow tests to be part of larger testing framework like LAVA.
Media Codec - AArch64 Port - Validation
10
New component level tests developed
By ARM and Linaro
For Video, Audio and Speech components
For encoder and decoder
Mostly file based tests – tests use file input and generate file output
Tests can run independent of Android – Does not require Android Software Stack
Allows porting/optimization/testing of components outside Android
Available in AOSP Gerrit and Linaro Android user-space git repo
Patches pushed to AOSP Gerrit – Not completely merged yet
Part of Linaro Android user-space git repo
Run as part of LAVA for each Linaro Android release
Media Codec - AArch64 Port – Validation (Contd...)
11
Media Codec Test Patches in AOSP Gerrit
12
Tests in Linaro Android user-space git
Repo for Android user-space tests and programs
Link
https://android.git.linaro.org/gitweb/platform/external/linaro-android-userspace-test.git
Contents
13
Android supports extensive list of media codecs
A significant set of codecs exist only in Android and do not have an upstream project
Codec validation requires exhaustive conformance tests as specified by standard
CTS is a good starting point but not the right place for comprehensive component level
validation
Codec component level tests were developed by ARM and Linaro
Test patches are available in AOSP Gerrit
Test code, media files and test script are also available in Linaro Android user-space git
repo
Tests are run as part of each Linaro Android release
Summary
14
Do we need more such component/subsystem level tests?
How to create them?
How do we share them?
Will Android Linaro git repo meet your requirement? Do you need anything different?
How to maintain them?
Is running part of Linaro Android release sufficient?
Anything else that can be done differently?
Discussion
15
Thank you!