1 打造智能电视的应用生态系统 Leon Zhang – 章立 Strategic Software Alliances ARM <<[email protected]>> SinaWeibo: Leon章立
1
打造智能电视的应用生态系统
Leon Zhang – 章立
Strategic Software AlliancesARM
<<[email protected]>>SinaWeibo: Leon章立
4
The Challenge is about ApplicationThe existing flourishing application ecosystem Google Play : ~ 500,000+ Apple App Store : ~ 590,000+ Windows Phone App : ~ 100,000+
However, can it benefit SmartTV user directly? When you install an application from mobile/tablet application market
to your SmartTV, what do you expect to happen? Often happen: exception,and can’t run; display issue; layout
issue; user interactive issue; not full features workable Rarely happen: run and play well Even it runs and plays well, do you think it is a real application
you want to play on the TV screen?
5
Application Ecosystem for SmartTV
Applications
OS
Hardware
Video Streamer AppManager
Security
Open Middleware
CPU & GPU
OS & S/W
Tools
7
Home Consumer Entertainment TrendsConnected home drivers Convergence Intuitive, 3D video and
graphics UI Connectivity and
social networking Energy efficiency Content Security Reducing cost
Enabling with ARM 2GHz Cortex™-A9MP and
Cortex-A15 TrustZone ® Virtualization
Mali ™ -3D graphics /GPGPU Physical IP, system
interconnect, compute systems Software ecosystem OS, browser
optimizations for core components Plug-ins and CODECs Development toolsMobile investments to be re-used in connected home
8
Drivers for A15 and Mali T6xx big-LITTLE Fully positive feedback by the industry
Ultra high performance Esp for single thread app
Virtualization SW flexibility Convergence
GPGPU OpenCL/Renderscript Video enhancement Gesture/voice/face recognition Augmented Reality
TV OS ( Guest )
App2App1
Tablet OS ( Guest )
App4App3
Virtual Machine Monitor (VMM) orHypervisor
Cortex-A15
11
Open Platform Scenarios
Cross Platform
Cloud Service
Local Resource Sharing Internet Resource Sharing Value-Added Services
Open PlatformWeb Application(HTML5/CSS/JavaScript/Web GL)
SDM DCM yWidgetLibrary
ExtendAPI g
DataMgmt …
Application Framework
12
The Open Platform and the FoundationsOpen Platform
Web Application(HTML5/CSS/JavaScript/Web GL)
SDM DCM WidgetLibrary
ExtendAPI
DataMgmt …
Application Framework •Android Davlik•QT Framework•Adoble AIR Framework•iOS Framework•Bada Framework•Android•MeeGo/Tizen/……•RIM OS•WebOS•Bada•iOS
•Linux + Open or Proprietary M/W stacks
•Proprietary OS + Open or Proprietary M/W stacks
• HTML5/CSS/JavaScript/WebGL
13
Tomorrow’s Converged Home Network
In-Home Coax Network
Gateway
MOCA->Ethernet
QAM+CAS
DOCSIS
EoC(e.g MoCA)
ETHERNET
Gateway is suited to act as hub for all services in the home
Gateway can talk to CE devices
Operator Managed Device
Power Efficient & Compliant
CE Devices all use standard Middleware
Fiber
EoC(e.g MoCA)
e.g. North America Cable Market
15
ARM Enablement Focus for the Foundation SW
Linux Kernel
LinuxMiddleware
WebBrowser
Webkit
JavaScriptCore
RIM OS
AIRFlash
AIRRuntime
Webkit
AIR for Mobile
JavaScriptCore
Flash
ProprietaryGraphics
QtRuntime
Webkit
Widget Toolkit
V8 JavaScriptEngine
Linux Kernel
AndroidMiddleware
MobileBrowser
Webkit
V8 JavaScriptEngine
Mobile
AIRFlash
Dalvik
LLV
M
Linux Kernel
LinuxMiddleware
SKIA 2DGraphics
ChromeBrowser
Webkit
V8 JavaScriptEngine
Chrome OS
X Windows
Flash
VP
8
Linux Kernel
AndroidMiddleware
TabletBrowser
Webkit
V8 JavaScriptEngine
Tablet
Flash
Dalvik
Linux Kernel
AndroidMiddleware
ChromeBrowser
Webkit
V8 JavaScriptEngine
Google TV
Flash
Dalvik
LLV
M
LLV
MLL
VM
LLV
M
LLV
M
VP
8
VP
8V
P8
VP
8
VP
8
LLV
M
VP
8
SKIA 2D SKIA 2D SKIA 2D
17
JavaScript Acceleration on ARM JavaScript accelerated by compiling to native code using JIT ARM performance increased 5X in 1 year ARM works on Multiple JITs Google V8 TraceMonkey JaegerMonkey IonMonkey Squirrel Fish Nitro Tamarin
ARM contributions to upstream projects results in performance optimizations across the ecosystem Example: ARM and Google continually optimizing V8 together Optimizations released into public: http://code.google.com/p/v8 More info at: http://bit.ly/v8ARMTurbo
Google V8
Android Chrome Chrome OS webOS
18
Optimization in HTML5 Optimizing DOM constructs for HTML5 tags <canvas> 2D bitmap graphics NEON optimized- libpixman, SKIA
<svg> 2D vector graphics NEON optimized- libpixman,SKIA feFilters optimized using SMP and NEON
Example: SVG 1.1 defines filters now sped up by ParallelJobs Turbulance 60-80% ConvolveMatrix 20-30% Morphonlogy 20-25% Lighting 10-20% [NEONized version takes this to 3X] Gaussian Blur 15% [NEONized version takes this to 4X]
<video><audio> NEON optimized : TheorARM, WebM VP8 and ffmpeg
CSS Parsing Redesigned CSS parser giving 5% speedup+
Added new RGB color parsing giving 30% speedup
19
Improving HTML5 with
CSS3 Animation (Accelerated Compositing) HTML5 2D Canvas (Accelerated 2D Canvas)
Evidence of improvement in Webkit using a GPU
20
NEON in open source Google WebM – 11,000 lines NEON assembler! Bluez – official Linux Bluetooth protocol stack Pixman (part of cairo 2D graphics library) ffmpeg (libav) – libavcodec
LGPL media player used in many Linux distros and products Extensive NEON optimizations
x264 – Google Summer Of Code 2009 GPL H.264 encoder – e.g. for video conferencing
Android – NEON optimizations Skia library, S32A_D565_Opaque 5x faster using NEON Available in Google Skia tree from 03-Aug-2009
LLVM – code generation backend used by Android Renderscript Eigen2 – C++ vector math / linear algebra template library Theorarm – libtheora NEON version (optimized by Google) libjpeg / libjpeg-turbo – optimized JPEG decode libpng – optimized PNG decode FFTW – NEON enabled FFT library Liboil / liborc – runtime compiler for SIMD processing DirectFB – Neon enabled s/w fallback
22
Anatomy of an Android Application
Marmalade System Libraries
Marmalade System Libraries
Native Activity App
Native Activity App
Custom Native Library
Custom Native Library Dalvik
EEDalvik
EE
Android System LibrariesAndroid System Libraries
Dalvik Native LibrariesDalvik Native Libraries
Flash+AIREE
Flash+AIREE
Gaya3D EE
Gaya3D EE
Mono EE
Mono EE
C/C++/AsmC/C++/Asm JavaJava ActionScript, C#, GayaScriptActionScript, C#, GayaScript
Dalvik Native LibrariesDalvik Native Libraries
Languages
Execution Environments
Native code Intermediary Byte CodeIntermediary Byte Code
Android Application – Run as a Dalvik process
Android Platform
ARM
23
ARM Native Code Advantage
Up to 4x faster performance when using NEON
Fast reuse of existing and cross platform software IP, including C/C++/Asm code
ARM native code example: Significant impact on a range of
applications Games Media Players Browsers VoIP clients
All 3D OpenGL ES content leverages ARM code
24
DS-5™CE for Android App Developers
Android ADT
NDK
SDK
adb
CDT(C/C++)
JDT(Java)
Streamline DS-5 Debugger
Native code debug,Performance analysis
App package generation,Debug bridge,
Android Virtual DeviceC/C++ and Java editors,
Java debug GUI
Ethernet, WiFi*
USB
*Available in Q2 2012
25
Timeline: The Big Picture Find hotspots, system glitches, critical conditions at a glance
Select from 40+ CPU counters, OS level and custom metrics
eAccumulate counters, measure timeand find instant hotspots
Select one or more processes tovisualize their instant load on CPU
Combined task switch trace andsampled profile for all threads
26
Mali GPU Graphics AnalysisOpenGL® API eventsCPU, and GPU fragment and
vertex processing activity
Frame buffer filmstrip
Hardware and Software counters
Visualize application activity per processoror processor activity per application
27
Enabling Developers on OEM Devices
ARM DS-5CommunityEdition
OEMs SDK
Develop
Debug
Optimize
Google AndroidSDK + NDK
Enables… Better development
platform Faster development
cycle Better quality content
29
Ne10 Introduction Ne10 is designed to provide a set of common, useful
functions which have been optimised for the ARM architecture and NEON provide consistent well tested behaviour and that can be easily incorporated into applications
C and JNI interfaces are provided (JNI Not publicly available yet)
The library is modular; therefore the functionality that is notrequired within an App can be discarded
30
Ne10 - Routines
Images and Video•Image processing•Color conversion•JPEG/JPEG2000•Video Coding•Ray-Tracing/Rendering
Communications& Signal Processing
•DSP•Audio Coding•Speech Coding
Data Processing•Data Compression•Cryptography•String Processing•Matrix/Vector Algebra•Data Integrity
Targeting three main categories of functions
31
Ne10Droid – The App in actionNe10Droid is a benchmarking Android App that uses Ne10Routines are written using VFP in C, VFP in Assembly and NEON.Example routines:arm_result_t normalize_vec2f(arm_vec2f_t * dst, arm_vec2f_t * src, unsigned int count);arm_result_t normalize_vec3f(arm_vec3f_t * dst, arm_vec3f_t * src, unsigned int count);arm_result_t normalize_vec4f(arm_vec4f_t * dst, arm_vec4f_t * src, unsigned int count);
32
Ne10 - Future plans
Dual licensing under both Apache 2.0 and MIT as to be compatible with GPLv2.
Ne10 will include DSP functions (Think Zephyr and OpenMAX)
working on using Ne10 to optimize Box2D and the popular game engine
We want your input on future functionality and participation!