Ubuntu Linux on the Qualcomm Snapdragon 600 Processor

1

Ubuntu Linux on the Qualcomm® Snapdragon™ 600 Processor

Mark Charlebois, Director of Open Source SW Strategy, Qualcomm Innovation Center, Inc. Nicolas Dechesne, Landing Team, Tech Lead, Linaro Krunal Patil, Technical Lead, eInfochips

Qualcomm Snapdragon is a product of Qualcomm Technologies, Inc.

3

Building a Community

Developer Enablement

• Mydragonboard.org • IFC6410 from Inforce • Linaro Ubuntu Based Developer Releases Solutions • eInfoChips

4

Linaro

Nicolas Dechesne, Linaro Landing Team Tech Lead

5

Linaro Overview

Linaro is working to accelerate and improve the open source ecosystem for the ARM architecture, helping to deliver industry leading innovation

Linaro is a collaborative software engineering company funded by members

Instead of duplicating effort for common open source software, companies collaborate to share development costs and software is built once

The work is carried out in the open, tested and then upstreamed into the relevant open source projects: kernel.org, AOSP …

6

Linaro Members Overview

Founded in 2010 with 6 members, now 28

CORE

CLUB

GROUP

COMMUNITY

7

Linaro Organization

Technical Steering Committee (TSC) O

ffic

e of

the

CTO

(OC

TO)

Enterprise Group (LEG) 16 companies

Digital Home Group (LHG) 8 companies

Mobile Group (LMG) 10 companies

Networking Group (LNG) 12 companies

Core Development

Kernel

Power Management

Security

Virtualization

Product Technology

LAVA Software Linaro Stable Kernel (LSK) Toolchain

Platform Engineering

Builds & Baselines

Systems

LAVA Lab

QA Services

Member Services

NDA Projects

Linaro Developer Technical Support (LDTS)

Field Engineering

>200 OSS engineers distributed globally

8

Linux for Broader Embedded Computing

9

Problem Statement

• Android kernel and user space stack not necessarily applicable in some broader embedded markets

• Lack of standard components support: Gstreamer, X11, Wayland, (embedded) Qt

• One-off Linux kernel drop not helping long development and maintenance cycle

10

Improvements and Solutions

• Linux Platform releases for Snapdragon • Multimedia and Graphics hardware acceleration for non-

Android Linux • Linux upstream kernel.org development

11

Linux for Snapdragon

12

Ubuntu Based Linaro Member Build

Based on standard Ubuntu for ARM (14.04), using Gnome Shell desktop

Access to Ubuntu repositories (apt-get) and updates.

Modified packages: Linux kernel, Mesa, X11 driver, Gstreamer

H/w acceleration enabled when installing proprietary “firmware”

Always based on “current” Ubuntu version

Monthly releases, names <YY>.<MM>, latest release: 14.08 − http://www.linaro.org/downloads/

− http://releases.linaro.org/14.08/ubuntu/ifc6410/

− http://bugs.linaro.org/, against “Qualcomm LT” product

− IRC: #linaro on irc.freenode.net

13

Release Content

The following pre-built images are released: − boot image that includes prebuilt kernel and initrd

− developer image that includes Ubuntu core packages as well as typical development packages

− gnome image that includes a minimal Gnome Shell environment GUI

All images have a pre-configured user called « linaro »

The images can be flashed on eMMC or installed any other external media such as USB, SD card, SATA.

Support for h/w video codecs and camera is “work in progress”

Linux kernel derived from Code Aurora msm-3.4 kernel. Upgrade to 3.14 is “work in progress”.

14

Managing Proprietary Firmware Blobs

Proprietary firmware blobs are required − GPU

− DSP

− WLAN

− Video hardware coprocessor (codec)

Need to be acquired legally from board vendor

Simple instructions to pack the firmware files into a separate image and flash into eMMC mkdir image && mount -o loop firmware-ifc6410-20140722-23.img image <extract firmware files in ./image> umount image

15

Install and Run Linaro Ubuntu Images

Download the release files

Flash the boot and rootfs partition

Optionally flash the firmware partition

Ensure serial connection or USB keyboard/mouse + HDMI display

Optionally (recommended!): use external storage e.g. SSD drive in SATA enclosure

fastboot flash boot boot-ifc6410-20140722-23.img fastboot flash cache firmware-ifc6410-20140722-23.img fastboot flash -S 768M userdata linaro-trusty-gnome-ifc6410-20140722-23.img fastboot continue

16

A Desktop OS, Using Gnome Shell Environment

17

You Can Get Some Work Done

18

Yes, That Too!

19

… And Relax!

20

“bare” X-server For Embedded Use Cases And Maximum Performance

21

IFC6410 Desktop setup

22

Graphics Stack

Linaro Ubuntu builds are currently based on Open Source “freedreno” graphics driver from freedesktop.org “mesa” project

Using Mesa 10.2.x stable releases

OpenGL 2.1

OpenGLES1.0, 2.0

Additional support to Qualcomm Technologies proprietary driver in future releases

23

Gstreamer

gst-launch-1.0 filesrc location=<file>.mov ! qtdemux ! avdec_h264 ! glimagesink

Software (CPU) video decoders

GPU rendering (rescale and color conversion)

Gstreamer

plugin with h/w video decoder support

qcom-vidc-dec

24

Developer Workflow

Ubuntu is a binary based distribution − Installable images are released

− Target devices can be used as development host

Nobody “rebuilds” Ubuntu, images are assembled from binary packages

Each user space package or component builds independently, “natively”

Most development packages (.h files) are available in the Ubuntu archives

Linaro uses Ubuntu Launchpad PPAs − ppa:linaro-maintainers/overlay

− ppa:linaro-maintainers/qcom-overlay

25

If You Prefer “DIY”

Embedded often requires to build your own GNU/Linux product/distro

The Yocto Project is a handy tool to create customized images − Initial BSP layer available for IFC6410 that brings

accelerated Xserver support − Initial patches accepted upstream in OpenEmbedded

− https://wiki.linaro.org/Boards/IFC6410/OpenEmbedded

− https://git.linaro.org/people/nicolas.dechesne/meta-qualcomm.git

26

Linux Kernel Upstreaming

Overall goal to contribute full platform enablement in kernel.org

“linux-msm” is the growing open source community dedicated to Qualcomm Technologies’ SoC kernel upstreaming ##linux-msm on irc.freenode.net, [email protected]

Linaro maintains a “tracking” branch − Rebased on each “-rc” kernel.org tag release

− Integrates all outstanding (public) development patches for APQ8064, APQ8074 and APQ8084

− Continuous Integration loop using Linaro LAVA framework and board farm

The “tracking” branch is a solid baseline to get started with upstream kernel development for Qualcomm Technologies’ SoC.

On IFC6410: Wlan, Ethernet, SD, eMMC, USB, SATA, HDMI Display.

27

Video Conferencing on the Snapdragon 600 Processor

Krunal Patil, Technical Lead eInfochips

28

eInfochips Solutions For Broad Embedded Market

Product Engineering Services

Re-engineering Sustenance Enhancement Production QA and

Certification Development Concept and Design

Application SW

BSP + System SW

Mechanicals

Hardware

29

www.einfochips.com

Peer-to peer and Multi-party Conference

Linux 3.4.0

eInfochips RTP Stack

QT5 based GUI

Sofia SIP and Freeswitch

Open Embedded

V4L2 based Video

Low latency SIP conferencing with Audio, Video and IM support

Solution Features

30 Qualcomm Technologies, Inc. All rights reserved.

Setup

31

System Diagram VOIP Conferencing UI

(QT5 on EGLFS)

Video capture module

V4L2 video capture driver for USB

camera

Video Encode module

Gstreamer based video encoding

ALSA sub system

Audio Capture module

G.711 encoding

Audio Encode module

A-V capture-encode-stream <- - -> A-V receive-decode-display

Sofia SIP for VOIP session mgmt

RTP Sender Module

eInfochips RTP stack for sending A-V data

RTP Receiver Module

eInfochips RTP stack for sending A-V data

Video Decode module

Gstreamer based video decoding

Video Display module

V4L2 video display driver

G.711 decoding

Audio Decode module

ALSA sub system

Audio Playback module

Media encoding Pipeline

Media Transmission Pipeline

Media Consumption Pipeline

32

Step by Step Process

• Build oe-core, bitbake and meta-qualcomm • V4L2 video capture support

OE – BSP layer

• Build core-image-minimal File system • Boot the IFC6410 board Filesystem

• Build core-image-x11 with OpenGL X11

Support

• Build meta-qt5 layer & make it part of file system

• Modify Qt layer to use xcb instead of default eglfs

QT5

• Customized layer to integrate software components Qt5, Sofia SIP, VOIP stack and Gstreamer

• .bbappend file for qtbase custom • voip-demo.bb for custom rootfs • Develop VC Application

Meta-eInfochips

• Modify QT SDK to bypass Qt-Gstreamer plugin and directly use Gstreamer native library APIs

• Use glimagesink to gain advantage of GPU acceleration

Performance Optimization

33

Video Conference Application

35

[email protected]

36

For more information on Qualcomm, visit us at: www.qualcomm.com & www.qualcomm.com/blog

©2013-2014 Qualcomm Technologies, Inc. and/or its affiliated companies. All Rights Reserved. Qualcomm and Snapdragon are trademarks of Qualcomm Incorporated, registered in the United States and other countries,. Uplinq is a trademark of Qualcomm Incorporated. Alll Qualcomm Incorporated trademarks are used with permission.Other products and brand names may be trademarks or registered trademarks of their respective owners.of their respective owners. References in this presentation to “Qualcomm” may mean Qualcomm Incorporated, Qualcomm Technologies, Inc., and/or other subsidiaries or business units within the Qualcomm corporate structure, as applicable. Qualcomm Incorporated includes Qualcomm’s licensing business, QTL, and the vast majority of its patent portfolio. Qualcomm Technologies, Inc., a wholly-owned subsidiary of Qualcomm Incorporated, operates, along with its subsidiaries, substantially all of Qualcomm’s engineering, research and development functions, and substantially all of its product and services businesses, including its semiconductor business, QCT.

Thank you FOLLOW US ON: