© Copyright Khronos Group, 2007 - Page 1 Khronos Overview Neil Trevett Vice President Mobile Content, NVIDIA President, Khronos Group.

© Copyright Khronos Group, 2007 - Page 1

Khronos OverviewKhronos Overview

Neil TrevettVice President Mobile Content, NVIDIA

President, Khronos Group


100Ms / year

Pervasive Mobile Media ComputingPervasive Mobile Media Computing• Handsets are becoming personal computing devices - not “just” phones- Your most personal computer – mobility, connectedness and numerous sensors

• Sophisticated media processing will be central to handheld revolution- Graphics and media will become as pervasive as it is on the PC

• Diverse applications will drive the need for handheld accelerated media- User Interface, 2D maps and 3D navigation, TV/Video, games …

80’sWorkstations

100Ks / year

90’sPCs

10Ms / year

00’sHandhelds

MediaPlatforms


Khronos Open StandardsKhronos Open Standards

• Khronos develops state-of-the-art open standards for the Authoring and Acceleration of graphics and media content- 3D and Vector 2D Graphics,

Video, Imaging, Audio

• Khronos API standards provide “Foundation-Level” acceleration- “Close-to-the-silicon” access to media

processors for efficiency and generality

• Khronos standards are royalty-free- For wide adoption on multiple platforms –

reducing application source fragmentationKhronos APIs enable mobile software to be PORTABLE and to EFFICIENTLY use

graphics and media acceleration

Khronos APIs enable mobile software to be PORTABLE and to EFFICIENTLY use

graphics and media acceleration


Around 100 companies creating media authoring and acceleration standards


Conformance Tests

Conformance Tests

Khronos Participation ModelKhronos Participation Model

PromotersPromoters

AdoptersAdopters

RatifiedSpecifications

RatifiedSpecifications

Conformance Tests and Conformance Test Process.Typically $10K per API fee

Conforming products can use API

trademark and logo

Openly and publicly distributed – free of charge,

royalty free

Board decides strategy – approves working groups, controls budget,

ratifies specifications. $25,000 annual membership dues

SDKsSDKs

Free libraries, utilities, examples,

open source

ContributorsContributors

Any company can join Khronos to participate in any number of working

groups to produce specifications.$7,500 annual membership dues

A Working Group for each API standard –

one company one vote


“DirectX-like” API set with OS Abstraction and API Interoperability

Khronos Standards EcosystemKhronos Standards Ecosystem

Embedded Media APIs

Vector 2D

Streaming Media Enhanced Audio

Dynamic Media Authoring

Dynamic Media Authoring Standards

Cross platform desktop 3D

3DAuthoring

Open StandardsEffects Framework

AL

Safety Critical 3D Embedded 3D

ILStreaming Media

System IntegrationOpen Standards

for window systems

DLCODEC and media

component portability

System IntegrationStandards

ApplicationAcceleration APIs

Family of Market-focused 3D APIs

Bridging 3D Authoring and Acceleration for advanced visual effects


We Are Here - OpenGL ES 1.1 is widespread- OpenVG is in adoption ramp

- OpenMAX is in adoption ramp

Mid-2005OpenVG 1.0 Spec release

Market adoption in media-accelerated handsets

Mid-2004OpenGL ES 1.1

Spec release

Beginning-2006OpenMax IL 1.0

Spec release

THIS MONTHOpenSL ES 1.0 and OpenMAX AL 1.0

Spec release

100%3D

Gra

phic

s

Vect

or 2

D

Stre

amin

g M

edia

Enha

nced

Aud

io

Adoption of Embedded Khronos Adoption of Embedded Khronos APIsAPIs

News Flash!


Penetration of Native Media APIsPenetration of Native Media APIs

Millions of unitsJon Peddie Research

Handheld Multimedia Devices report

0

100

200

300

400

500

600

700

800

900

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

MM phones

Non 3D MM phones

Native API based phones


OpenGL ES Applications and OpenGL ES Applications and DevicesDevices• OpenGL ES 1.1 has become the most widely deployed 3D API- Used in diverse applications, devices and markets

• Advanced User Interfaces- iPhone uses OpenGL ES

• 3D Navigation- Google Earth and 3D city data

• Mobile Games- $7.2Bn market in 2010 (Informa) Nokia N93, iPhone, Sony PS3

All use OpenGL ES 1.1Nokia N93, iPhone, Sony PS3

All use OpenGL ES 1.1


3DSmall footprint 3D for embedded systems

Complete Khronos Media StackComplete Khronos Media Stack

Vector 2DLow-level vector acceleration API

Media Engines – CPUs, DSP, Hardware Accelerators etc.

Platform MediaFrameworks

IL

SOUNDEnhanced

audio

Image Libraries, Video Codecs, Sound Libraries

Component interfaces for codec integration

AL

Playback and recording interfaces

Khronos defines low-level, FOUNDATION-level APIs.“Close to the hardware” abstraction provides portability AND flexibility

The Khronos API family provides a complete ROYALTY-FREE, cross-platform media acceleration platform

Applications or middleware libraries (JSR 184 engines, Flash players, media players etc.)

EGL Graphics surface management for efficient mixed mode 2D/3D/video rendering

Accelerated media primitives for codec

developmentDL


A SET of native APIs to de-fragment mobile application development.Like DirectX – but an open standard for mobile platforms

SOURCE PORTABILITY for rich media applications

Operating System Abstraction APIsDefine APIs to isolate applications from operating system differences

Media APIs with defined and tested InteroperabilityInclude Khronos media APIs and define how they work together


Media AccelerationGPU / DSPs / CPUs

Kernel Operating System (e.g. Rex, Symbian OS, Windows Mobile, Linux, Nucleus)

Native Applications

OpenKODE API SetOpenKODE API Set

OpenKODECore

OS Abstraction

Enhanced Audio3D Vector 2D Streaming Media

AL

EGLSurface abstraction and Trans-API Communication

3D Game Engines

TV/Video/Audio Players

User Interface

NativeApplications

Flash/SVG Players

Software Platform (e.g. Brew, Symbian UIQ, S60, WIPI)

JavaApplications

Java JVM

Bindings


OpenKODE ComponentsOpenKODE Components

Trans-API Conformance Tests

3D

EnhancedAudio

Vector 2D

OpenKODE CoreEvents, IO, files, memory, threads,

strings, math, locale, time, networking, windowing, crypto, debug

EGLTrans-API

Communication StreamingMedia

POSIX-like APIs - mandatory for a compliant OpenKODE.

Standalone conformance tests

Selected Media APIs PLUS any EGL necessary for those APIs

to communicate.(OpenGL ES 2.0, OpenMAX AL 1.0 and

OpenSL ES 1.0 will be in OpenKODE 1.1 – added through extensions to

OpenKODE 1.0)

A conformant OpenKODE must pass all individual conformance tests PLUS trans-API tests to ensure APIs can

communicate and operate concurrently

OpenKODE provides RELIABLE foundation-level acceleration for media applications that MIX multiple media types

OpenKODE provides RELIABLE foundation-level acceleration for media applications that MIX multiple media types


SoftBank Mobile AnnouncementSoftBank Mobile Announcement• New-generation mobile platform will provide native access to OpenKODE- POP-I – Portable Open Platform Initiative

• Mandated native API set for advanced UI and games


OpenKODE MilestonesOpenKODE Milestones

• OpenKODE 1.0 Provisional released at 3GSM 2007- Twelve months from kick-off meeting to specification on web-site- Encouraging developer feedback before spec finalization

• Full conformance tests being prepared in parallel with specification- Written by Futuremark – tests OpenKODE Core AND trans-API operation

1Q07 2Q07 3Q07

OpenKODE 1.0 Provisional Release

4Q07 1Q08

Provisional Conformance Tests to be Released

2Q08 3Q08

OpenKODE 1.1?Release – add OpenMAX and OpenSL ES support

OpenKODE 1.0 Final Release Target with

full conformance tests


OpenKODE 1.1 - 3D Video TexturesOpenKODE 1.1 - 3D Video Textures• EGL is a Khronos API for abstracting rendering surfaces- OpenKODE 1.1 working to define how surfaces shared between OpenGL ES and OpenMAX- Aiming to eliminate unnecessary copies for power and processing efficiency

Without EGLImage the application must copy the OpenMAX buffer into an OpenGL ES texture

MPEGVideo

OpenMAXMPEG-4 Decode

OpenMAXBuffer

OpenGL ES3D Engine

glTeximage2DSurface

An EGLImage surface can be used as both the destination of the decode and a texture without copying the data

MPEGVideo

OpenMAXMPEG-4 Decode

EGLImageSurface

OpenGL ES3D Engine

Data copy wastes cycles and space


Raising 2D and 3D Visual QualityRaising 2D and 3D Visual Quality

Older generation APIs Provide rudimentary graphics functionality and quality

State-of-the-art APIs enable compelling consumer displays

Advanced functionality, fast interactivity and extremely high quality

High-quality 2D graphics using OpenVG

High-quality 3D displays using OpenGL ES

Video processing with OpenMAX


COLLADA = Powerful AuthoringCOLLADA = Powerful Authoring• COLLADA is a XML database schema for 3D assets- Can hold geometry, animation, visual effects, physics – everything to do with a scene

• COLLADA transports 3D assets between applications- Enables binding of diverse DCC and 3D processing tools into a production pipeline

• COLLADA can be lossless – never lose information- Retains all information - even multiple versions of the same asset

• COLLADA is an open, archive-grade format that retains meta information- When your DCC tool upgrades, you keep your assets

Custom Tools

COLLADA is non-destructive and so supports round-

tripping of tools to enable powerful authoring pipelines


COLLADA MomentumCOLLADA Momentum• COLLADA 1.4 supported by all major tools and thousands of users- 3ds Max, Photoshop CS3, Blender, DAZ|Studio, C4 Game Engine, Poser- NVIDIA FX Composer, Google Earth, Houdini, Maya, Sketchup, and XSI

• DAZ|Studio 1.7 exports COLLADA- Complete export of mesh, skinning, rigging, lighting, camera, and animation data.

• Caligari trueSpace7.5 exports COLLADA • Adobe Photoshop CS3 Extended imports COLLADA- For texture editing on 3D objects


COLLADA UICOLLADA UI

• Bringing 2D vector graphics to COLLADA file format• Enables deep intermixing of 2D and 3D graphics• Support for 2D and 3D Font libraries• Font glyphs to support effects and animation for advanced rendering


COLLADA ConditioningCOLLADA Conditioning

• Conditioning pipelines take authored assets and:

• 1. Strips out authoring-only information

• 2. Re-sizes to suit the target platform• 3. Compresses and formats binary

data for the target platform

• Different target platforms can use the same asset database with the appropriate conditioning pipeline

Multiple tools create assets and scenes in a COLLADA

Database

Conditioning Pipeline

Conditioning Pipeline

• COLLADA is an interchange format - not a delivery format or a scene graph


Application

Runtime API

glFX – Effects FrameworkglFX – Effects Framework

- Existing Khronos standards- New glFX standard

COLLADA FXTextures, shaders programs, geometry, control and pass information

COLLADAFile format for 3D asset interchange – widely adopted by DCC tools vendors and Google, Adobe, Epic etc.

Runtime file format

glFX File Format - OPTIONAL Contains effects optimized for deployment

Conditioning

OpenGL 2.1 / OpenGL 3.0 / ES 2.0 (Also ES 1.1 / Mount

Evans)

glFX Run-time APIApplication traverses scene data, uses the glFX Runtime API to use effects information to setup the rendering pipeline


Benefits of One OrganizationBenefits of One Organization• OpenGL and OpenGL ES are now evolving under one IP framework- Design innovations can be freely shared between the APIs

• OpenGL and OpenGL ES can share same resources and outreach- Common Conformance tests, marketing and web-site, tool chains etc.

• OpenGL and OpenGL ES Working Groups will remain independent- Both groups will be able to make decisions that best serve their own markets

Embedded Markets

Desktop Markets

Architectural design expertise

Market feedback on streamlining functionality

Momentum - hundreds of millions of OpenGL ES devices


OpenGL Roadmap SynergyOpenGL Roadmap Synergy• OpenGL ES has leveraged desktop OpenGL architecture- OpenGL ES 1.1 streamlined OpenGL 1.5- OpenGL ES 2.0 streamlined OpenGL 2.0 and the GLSL shading language

• Next Generation OpenGL 3 and “Mount Evans” leverages OpenGL ES 2.0- Learning from the streamlining of OpenGL ES 2.0 core

• OpenGL ES will benefit from the architectural innovation of “Mount Evans”- Nexgen OpenGL ES – “Halti” will use desktop innovations for performance and functionality

OpenGL 1.5/2.0Architectural Foundation

OpenGL ES 1.1/2.0Functional

Streamlining

“OpenGL 3 / Mount Evans”Architectural Streamlining and next

generation functionality

“Halti”Next generation

functionality for embedded markets


OpenGL 3 and Mount EvansOpenGL 3 and Mount Evans• OpenGL 3 – before end of 2007- Architectural streamlining of OpenGL- Elimination of fixed functionality – evolving work in OpenGL ES 2.0- New object model for speed and simpler programming- GL allocates handles, Direct editing, Efficient state grouping, Efficient sharing

• OpenGL 3 provides backwards compatibility- OpenGL 3 requires a new context- Both OpenGL 2.1 and OpenGL 3 contexts may bind to same drawable- Legacy code and LP code may peacefully co-exist - developers migrate at their own pace

• Mount Evans – early 2008- Adds significant new features to OpenGL 3- Geometry shaders- Greatly increased integer support in the OpenGL Shading Language- Stream out of vertex data to a buffer object- Texture arrays- Texture buffer objects- Numerous new texture formats- Instanced rendering


Khronos 2007 MilestonesKhronos 2007 Milestones

OpenMAX IL 1.1 Public Release

with Conformance Tests

OpenKODE 1.0 ProvisionalReleased

1Q07

iPhone launched – uses OpenGL ES 1.1

for UI and display composition

COLLADA Adopted by Adobe for

Photoshop CS3

OpenVG 1.0 Conformance

Tests Released

OpenVG 1.0 Reference

Implementation open sourced

2Q07

glFX Working Group Announced

OpenWF Working Group

Announced

COLLADA UI TSG Announced

OpenGL 3 features revealed

3Q07

OpenMAX AL 1.0 Provisional Released

OpenSL ES 1.0 Provisional Released

OpenVG 1.1 Public Release

OpenKODE 1.0 Final Release

COLLADA 1.4.2 Conformance Tests Release

OpenGL 3 Release

4Q07


• “Foundation Level” APIs to enable software to effectively use silicon- Close to the silicon – fundamental functionality needed on every platform

• Synergistic mix of authoring and acceleration standards- Being designed together

• Hundreds of man years invested- Beyond any single company now to produce specifications of this breadth and depth

• Royalty-free- Khronos is committed to generating market opportunities for its members and the industry

Why Are Khronos Standards Key?Why Are Khronos Standards Key?


Help Khronos Help You!Help Khronos Help You!• Please consider joining Khronos!- Developer feedback and participation is always very welcome

• Please review our specifications and provide feedback- Public forums and developer resources here- http://www.khronos.org/developers/

Slides and Khronos membership details at www.khronos.org

© Copyright Khronos Group, 2007 - Page 1 Khronos Overview Neil Trevett Vice President Mobile Content, NVIDIA President, Khronos Group.

Documents

khronos media apis

copyright khronos group

khronos group slide

khronos open standards

media acceleration slide

media players

media content

d apis