1 Mobile Platform: Technical Perspective Justin (Joon Sung) Hong Software Center, Samsung Electronics.

1

Mobile Platform:Technical PerspectiveJustin (Joon Sung) HongSoftware Center, Samsung Electronics

2

Contents

Agenda Introduction Technical Trend Case Studies Standardization Conclusion Appendix. Subjective Recommendation

Audience Researchers/Developers of embedded mobile software

solutions Students majoring in electrical/computer/software engineering

3

Introduction

4

Computers

Products in Samsung Electronics

Quiz Which programming

languages are used for devices?

How many lines of codes are inside a device?

How many mobile phones are sold for a year? How many mobile subscribers in the world?

5

A Secret Life of a Cellphone

Samsung SGH-D600

6

MCPMCP

Reference Hardware Architecture

RFRF

MCPMCP

AudioAudioApplicationProcessor

ApplicationProcessorModemModem

MemoryCard

MemoryCard

LCDLCD

LCDLCD

NANDNAND

KeypadKeypad

NORNOR

SDRAMSDRAM

CameraCamera

NORNOR

SDRAMSDRAM

FIFO or DPRAM MUX

Single or Dual CPU

7

Reference Software Architecture

ProcessorProcessor

OS,Security

OS,Security

Audio-Visual &Multimedia

Audio-Visual &Multimedia

Application Framework (Optional)Application Framework (Optional)

DataProtocol &

LocalConnectivity

DataProtocol &

LocalConnectivity

Graphics,Window

Graphics,Window

HW

Pla

tfo

rm

Native ApplicationNative Application

3rd Party and VMApplication

3rd Party and VMApplication

Platform Interface

Application Service Extension (Optional)Application Service Extension (Optional)Ap

pli

ca

tio

n

ShellShell

PeripheralsPeripherals

NetAccess

NetAccessMultimediaMultimediaConnectivityConnectivityMMIMMI

TelephonyTelephonySW

Pla

tfo

rm

8

What is Mobile Platform?

A mobile platform describes some sort of framework, either in hardware or software, which allows mobile applications to run.

Typical platforms include a mobile device's architecture, operating system or middleware, and their runtime libraries.

Related terms {Mobile|Wireless} {software|system|application|internet|}

platform Examples

Mobile software platform Wireless internet platform Mobile system platform Mobile application platform

9

Why Mobile Platform?

Device manufacturers Common interfaces Better development environment Productive application framework Digital convergence Reliability (minimise crashes and bugs)

Chipset vendors System engineering Fast time-to-market

Mobile operators Reduced content and services management Support of push-pull information delivery [ARPU] Multimedia, interactivity and location based

features [ARPU] Multitasking [ARPU] Security

Application providers Open APIs Enhanced libraries and tools for easier and

faster development Support of a variety of programming languages

10

Mobile Platform War

Platform by platform/application software companies Symbian OS Microsoft Windows for Mobile MontaVista Mobilinux PalmOS Openwave PhoneSuite Teleca Obigo RIM BlackBerry Nokia Symbian OS + Series 40/60/80/UIQ/90

Platform by manufacturers Nokia Symbian OS + Series 40/60/80/UIQ/90 Motorola Synergy-2000, Linux Samsung Mocha, Ocean

11

Mobile Platform War (Cont.)

Platform by chipset vendors Ericsson Mobile Platform QUALCOMM BREW TTPCom Ajar TI OMAP

Platform by operators Java2 Mobile Edition WIPI QUALCOMM BREW SavaJe GVM, MAP, etc.

12

Te

chn

olo

gy R

equ

irem

ents

& B

OM

Low-end Mid-range High-end/richFeature

Low-end Smartphone

Connected PDA/High-end

Smartphone

Technology Range

200445% 32% 18% 2.4% 2.6%200357% 28% 12% 1.5% 1.5%

Source: Future Mobile Handsets 2004

Mobile Phone Sales 2004

13

Market Share Estimates for Smart Phones

0.00%

5.00%

10.00%

15.00%

20.00%

25.00%

30.00%

35.00%

2003 2004 2005 2006 2007

Ovum

ARC-Group

ARC-Chart

IDC

Gartner

Source: Public industry reports

14

Technical Trend

15

1.Processor

CPU Core: ARM9 with MMU

ARM11~12Multi-core

Configuration: Single chip (CPU core, baseband processing, DSP) or dual chip (modem and application processor)

Single CPU or MCP with dual-core DSP

Core: TI TMS, Ceva, StarCore, Samsung CalmRISC, ARM DSP, QUALCOMM QDSP, etc.

DSP array Software: Remote execution/wait,

emerging DSP platformMore DSP platforms, DSP virtualization

16

2.System Software

OS Kernel: RTOS for modem (e.g. Nucleus, REX, AMX, OSE), RTOS or GPOS

for application processor (e.g. Linux, Windows Mobile), multi-tasking, single address space, no memory protection

Multi-threading, virtual memory, memory protection for applicationsHybrid/Nano kernel for running multiple OS (e.g. Jaluna Vmware)Energy-awareness

Storage File system: Proprietary NAND/NOR file system

Performance/security enhancementeXecution-In-Place (XIP), shared library

Database: Indexed file system or proprietary table data management

Public database with connectivity (e.g. JDBC), SQL-like query

17

2.System Software (Cont.)

Security Anti-threats: Not yet covered but on going…

Code protection, secure boot, RTC, memoryTrusted computing

Acceleration: SW ECC and HW RSA acceleration

+ Package engine (e.g. IPSec) Reliability

Bug patch: Manual w/ USB, OTA firmware upgrade

OTA firmware/component upgradeZero administration

18

TI Applications Processing Solutions

19

OMAP1710 Architecture

90 nanometer CMOS technology

ARM926EJ CPU core (~220MHz) Secure boot-loader Hardware acceleration for

security standards and random number generator

Memory controller for I/F to 128MB SDRAM, 256MB Flash, Mobile DDR

CompactFlash, SD/MMC/SDIO ports

TMS320C55X DSP (~220MHz) One/two instr. executed per

cycle Video HW accelerator for DCT,

iDCT, pixel interpolation, and motion estimation for video compression

Connectivity USB OTG, UARTS, I2C, Fast IrDA

(FIR), SPI LCD Controller, camera ports

20

Models of Trusted Computing

kernel

userland

kernel VMM kernel

userlandkernel

userland

kernel

userland

kernel

userland

trustedkernel

trusteduserland

Normal OS Virtual Machine (VMWare, Java VM)

Virtual Machine Monitor

CPU with trusted mode (MS NGSCB,

ARM TrustZone)

virtualizationBetter!

21

Trusted Computing Architecture

22

Microsoft Next-Generation Secure Computing Base

TPM 1.2TPM 1.2

User

Kernel

Hardware Secure InputSecure Input ChipsetChipsetCPUCPUSecure VideoSecure Video

Main OSMain OS

USBUSBDriverDriver

Nexus-Mode (RHS)

NexusNexus

NexusMgr.sysNexusMgr.sys

HALHAL

NALNAL

User Apps.User Apps.

AgentAgent

NCA Runtime LibraryNCA Runtime Library

Trusted UITrusted UIEngine (TUE)Engine (TUE)

TSPTSP TSPTSP TSPTSP

AgentAgentAgentAgent

Standard-Mode (LHS)

23

ARM TrustZone

TrustZone adds a “parallel world” to allow trusted programs and data to be safely separated from the operating system and applications

Not TPM-based TCG, but resolves the security requirements

Similar to Intel’s.

24

3.Graphics

3D graphics Engine: Proprietary or standard-based solutions (e.g. Hi

MascotCapsule, Superscape Swerve, Fathammer)OpenGL-ES, Mobile 3D Graphics (JSR-184)

Acceleration: SW or 1~2M triangles/sec acceleration, OpenGL-ES/EGL 1.0(e.g. ARM MBX, nVIDIA GeForce)

50M < triangles/sec acceleration, OpenGL-ES 2.0 Vector graphics

Engine: Standard-based solutions (e.g. Sharp SVG-T, BitBoys)

SVG Basic Acceleration: 100~200M pixels/sec acceleration

OpenVG Font

Font: bitmap and vector trials

Downloadable/upgradeable fontsVector, 3D font

25

ARM MBX 3D Graphics Accelerator

Easily integrated with ARM926EJ-S, ARM1136J-S using AMBA

Features Deferred texturing Tile-based rendering Scene manager PVR-TC : texture compression YUV to RGB conversion Flat & Gouraud shading Full-screen anti-aliasing Optimized UMA (Unified

Memory Architecture) ITC (Internal True Color

operations) Performed on chip at 32-bpp

for superior color precision

Two kinds of cores MBX HR-S

Fill rate : 320-480M pixels/s Rendering rate : 2.5M tri/s Screen resolution : up to VGA

MBX R-S Fill rate : 120-180M pixels/sec Rendering rate : 1M tri/s Screen resolution : up to

QVGA

26

4.Multimedia

A/V CODEC Video: H.263/MPEG4, RTP/RTCP streaming (e.g. RealNetworks,

PacketVideo)H.264/WMV10, OpenMAX

Sound/Speech/Audio: Yamaha 40~64 poly, NB-AMR, MP3

MP3/AAC+, OpenMAX, WB-AMR Image: BMP, WBMP, GIF, JPEG, PNG

+ JPEG2000, Motion JPEG

Broadcasting Trials on Satellite-DMB, Terrestrial-DMB, DVB-Handheld/IPDC,

QUALCOMM MediaFLORelated to policy / regulationInteractive

27

5.Network

WAN GSM: 3GPP release 4, WCDMA/EDGE commercialization

3GPP release 6, HSDPA CDMA: IS-856, 1xEVDO

1xEVDV (cdma2000 Rev.D)

WAN+WiFi (seamless handover)

28

5.Network (Cont.)

MAN/LAN/PAN 802.11a/b/g commercialization

802.11n, Mobile WiMax (802.11e) vs. WiBro IrDA, Bluetooth 1.0

More Bluetooth profiles, Bluetooth 2.0, UWB

MobilityHigh

Low

4 G4 G

802.11b

CDMA2000 EV-DO/DV

WCDMA/HSDPA

2.4GHzWLAN

802.11a/g

CDMA/GSM/TDMA

UWB802.11nMANET

High speedWLAN

WiBroMobile WiMax

(802.16e)

5GHz WLAN

Bluetooth

1995 2000 2005

<100 Mbps~ 14.4 kbps <50 Mbps384 kbps144 kbps

AMPSETACSJTACSNMT

PANData Rate

2010+

1G( Analog )

2G( Digital )

3G( IMT2000 )

3G+

29

5.Network (Cont.)

Upper layers Wireless-profiled TCP/IPv4, mobile IPv4

IPv6/v4Ad-hoc networking

Emerging SIP/SDPIP Multimedia Subsystem, VoIP, IMPS

30

QUALCOMM MSM Roadmap

31

MSM6250 Architecture

32

MSM6250 Features

Processor ARM926EJ-S up to 150MHz

with MMU Java HW acceleration MMC/SD interface

NetAccess Dual-mode WCDMA plus

GSM/GPRS ZIF RF supports WCDMA,

quad-band GSM (GSM 850, 900, 1800, 1900)

gpsOne solution RUIM/SIM Controller

Connectivity Support Bluetooth baseband

Multimedia 2D/3D graphics acceleration 2 Megapixel camera interface QDSP4000 up to 75MHz (Data

transmit rate 153Kbps) Supports the advanced

feature set of the Launchpad suite, including streaming video, audio, still image and video encoding and decoding

33

6.Application

Browsing Presentation (e.g. Access, Openwave, etc.): XHTML MP, WML1.3

XHTML, CSS Protocol: WSP/HTTP dual stack

HTTP

Messaging Multimedia messaging: OMA MMS 1.1

OMA MMS 2.0, E-mail Instant messaging: OMA IMPS 1.1 (WirelessVillage)

IOP w/ wired IM clients Push-To-Talk

34

6.Application (Cont.)

Security Digital rights management: OMA DRM 1.0 (FL/CD/SD)

OMA DRM 2.0, Microsoft Janus, Apple Java

Virtual machine: SW – Sun CLDC HI, HotSpot, HW - ARM EJ, Nazomi

Similar tech. but more hardwired Class libraries: MIDP/CLDC, Wireless Messaging, Mobile Multimedia

API+ CDC, Mobile 3D/Vector Graphics, New Architecture

Remote service Web service

OMA Mobile WS

35

Java2 Micro EditionJava2 Micro Edition

KVM / CVM

Applet

36

ARM Jazelle Java Acceleration

Jazelle bytecode partition 139 bytecodes executed directly in hardware (constant loads,

variable loads/stores, array load/stores, etc.) 88 bytecodes executed in software (floating point, integer

division, switch, invoke, etc.) ARM9EJ-S pipeline (6 stage - Java state)

InstructionFetch

RegisterWrite

Sum/Accumulate & Saturation

Memory Access

ComputePartial Products

Shift + ALU

Thumb Decode

ARM Decode

RegisterDecode

RegisterDecode

RegisterRead

RegisterRead

Jazelle logic turns on

FETCH DECODE EXECUTE MEMORY WRITEBACK

Java DecodeStack

Management

Java DecodeRegister

Read

ALUControlSignals

BytecodeInstructionStream

37

7.User Interaction

Man-Machine Interface IME: Too many solutions (e.g. T9, ez, CJI, etc.)

Similar but better idea (ex. Fast tap, Q12, Virtual keypad, etc.) Voice recognition: Used for voice dialing/phonebook lookup but not

complete (e.g. VoiceSignal, Conversay)Voice command/browsing

Personalization Context awareness: Trial

Preference, location-based consulting, etc. UI customization: Color, theme, menu tree

Whole look&feel with Macromedia Flash, XML, or COM technologies

38

8.Device

Camera1M~2M pixels, flash, optical zoom

5M < pixels, auto focus, etc. LCD

TFT/TFD, 65K colors, 176x220 or QVGA sizeOLED/3D, <= 16M color, <= SVGA size

Storage32~64MB NAND internal flash memory

8GB < NAND flash memory card, 40GB < HDD

39

Case Studies

40

Symbian OS (+Series 60)

Symbian OS Owned by Nokia, Ericsson, SonyEricsson, Panasonic, Siemens,

and Samsung. Series 60

Developed and licensed by Nokia

41

v9.1 Features

Real-time A real-time, multithreaded

kernel Hardware support

Latest CPU architectures, peripherals and internal and external memory types

Security Application capabilities

through certification, private protected data stores. full encryption and certificate management, secure protocols (HTTPS, SSL and TLS) and WIM framework

Graphics Direct access to screen and

keyboard, graphics accelerator API, increased UI flexibility

Multimedia Recording, playback and

streaming, image conversion Mobile telephony

WCDMA (3GPP R4 and R5 IMS), GSM CSD and EDGE CSD, GPRS and EDGE GPRS, IS-95 and 1xRTT, SIM, RUIM, UICC Toolkit, CDMA specific features

Communications protocols TCP/IP (dual mode IPv4/v6)

and WAP 2.0 (Connectionless WSP and WAP Push), IrDA, Bluetooth and USB

International support Unicode Standard version 3.0

42

v9.1 Features (Cont.)

Device Management/OTA provisioning

OMA DM 1.1.2 compliant, OMA Client provisioning v1.1

Data synchronization PC-based/OTA

synchronization, a PC Connectivity framework

PIMS Contacts, scheduling, and

messaging, OBEX for exchanging appointments (vCalendar) and business cards (vCard)

Messaging Enhanced messaging (EMS)

and SMS, internet mail using POP3, IMAP4, SMTP and MHTML, attachments

Java MIDP 2.0, CLDC 1.1, JTWI

(JSR185), Mobile Media API (JSR135), Java API for Bluetooth (JSR082), Wireless Messaging (JSR120), Mobile 3D Graphics API (JSR184) and Personal Information Management and FileGCF APIs (JSR075)

43

Device Family Reference Design

Crystal Feature-rich, Communicator-

type devices Full (typically qwerty)

keyboard Landscape-oriented

(horizontal) display Nokia 9290 Communicator

Peal Standard smart phones Small display & limited

keyboards Nokia 7650

Quartz Pen-based phone similar to the

Palm PDA Portrait-oriented (Vertical)

display SonyEricsson P802

44

Standardization

45

Standardization Bodies

OMA (Open Mobile Alliance) www.openmobilealliance.org All mobile application

middleware and services including browsing, synchronization, device management, messaging, location, security, etc.

Multimedia ITU-T (International

Telecommunication Union) www.itu.int/itudoc/itu-t/ MPEG4, H.324M, JPEG2000,

etc. Khronos

www.khronos.org Media authoring and playback

including OpenGL-ES, OpenVG, OpenMAX, etc.

46

Standardization Bodies (Cont.)

Operating System CELF (CE Linux Forum)

www.celinuxforum.org Embedded Linux

Internet W3C (World Wide Web

Consortium) www.w3c.org XML, DOM, PNG, SMIL, SVG

IETF (Internet Engineering Task Force)

www.ietf.org SIP, TCP/IP, MIPv6, DNS,

IPSec, RTP/RTSP ECMA (European Computing

Machinery Association) www.ecma-international.org C#, ECMAScript, etc.

47

Standardization Bodies (Cont.)

System MIPI (Mobile Industry

Processor Interface) www.mipi.org Mobile application processor

TCG (Trusted Computing Group)

www.trustedcomputinggroup.org

Secure computing Telephony

3GPP www.3gpp.org GSM/GPRS/WCDMA

3GPP2 www.3gpp2.org IS95, IS2000

CDG (CDMA Development Group)

www.cdg.org

48

Standardization Bodies (Cont.)

Wireless internet JCP (Java Community Process)

www.jcp.org J2ME, MMAPI, WMA, etc.

KWISF (Korea Wireless Internet Standardization Forum)

www.kwisforum.org WIPI, LBS, MMS, etc.

Operator OMTP (Open Mobile Terminal

Platform) www.omtp.org

GSMA (GSM Association) www.gsmworld.com

49

Conclusion

50

Conclusion

The mobile world is moving very fast. Digital convergence comes true and mobile phones are at

the center of the trend. Digital still camera, MP3 player, PDA, GPS receiver, game

console, digital video recorder, and TV have been converged. What will be the next?

The mobile platform is the key technology. Currently, ARM, QUALCOMM, Symbian, Nokia, TI, Openwave,

Real, etc. are winners. Who will be the next?

51

Appendix. Subjective RecommendationTo be a good software architect and/or software project manager in embedded worlds

52

Common Fallacies of Software Lovers

Software development is art. Computer/Software engineering is NOT computer science.

Scientists develop for learning, but engineers learn for development.

Design and coding are most valuable skills. Maintenance is the longest (> 40%~80%) phase of software

lifecycle, i.e. Maintenance > Testing > Design > Coding/Requirements.

Read good codes before writing your own.

Development process is the only need to know. Learn various software engineering disciplines.

The knowledge of hardware and system software is none of my business.

The development without the knowledge is to create an empire built on sand.

53

Software Engineering Body of Knowledge

RequirementsRequirements DesignDesign ConstructionConstruction TestingTesting MaintenanceMaintenance

ConfigurationManagement

ConfigurationManagement

EngineeringManagementEngineeringManagement

EngineeringProcess

EngineeringProcess

Tools andMethods

Tools andMethodsQualityQuality

RelatedDisciplines

RelatedDisciplines

• Requirements Fundamentals• Requirements Process• Requirements Elicitation• Requirements Specification• Requirements Validation• Requirements Analysis• Practical Considerations

• Design Fundamentals• Key Issues in Design• Structure and Architecture• Design• Quality Analysis and Evaluation• Design Notations• Design Strategies and Methods

• Construction Fundamentals• Managing Construction• Practical Considerations

• Testing Fundamentals• Test Levels• Test Techniques• Test Related Measures• Test Process

• Maintenance Fundamentals• Key Issues in Maintenance• Maintenance Process• Techniques for Maintenance

• Management of the SCM Process• Configuration Identification• Configuration Control• Configuration Status Accounting• Configuration Auditing• Release Management and Delivery

• Quality Fundamentals• Quality Management• Processes• Practical Considerations • Computer

Engineering• Computer Science• Mathematics• Software Ergonomics• Systems Engineering

• Initiation and Scope Definition• Project Planning• Project Enactment• Review and Evaluation• Closure• Engineering Measurement

• Process Implementation and Change• Process Definition• Process Assessment• Process and Product Measurement