WT-4064, Build Rich Applications with HTML5 and WebGL, by Tony Parisi

BUILD RICH APPLICATIONS IN HTML5 AND WEBGL

TONY PARISI NOVEMBER 12, 2013

2 | BUILD RICH APPLICATION S IN HTML5 AND WEBGL | NOVEMBER 21, 2013 http://www.tonyparisi.com/

ABOUT ME

serial entrepreneur

founder, Vizi (stealth startup)

consulting architect and CTO

co-creator, VRML and X3D web standards

co-designer, glTF

author, speaker instructor

contact information

[email protected]

skype: auradeluxe

http://twitter.com/auradeluxe

http://www.tonyparisi.com/

http://www.learningwebgl.com/

book source code

https://github.com/tparisi/WebGLBook

https://github.com/tparisi/Programming3DApplications


mailto:[email protected]














RIP: 1995-2013

Eric Dye Image:


http://churchm.ag/author/ericdye/




LONG LIVE…



HTML5: THE BROWSER AS PLATFORM

graphics APIs– Canvas, WebGL, SVG

CSS3 – animations, transitions, transforms and filter effects

hardware-accelerated compositing

‒ all elements blend seamlessly on page

JavaScript performance and language enhancements

and a whole lot more…

‒ first-class streaming media types - <audio>, <video>

‒ device input

‒ location and mobile-inspired features

‒ database, local storage, file system

‒ networking – sockets and real-time connections (WebRTC)

‒ multi-threading (Workers)



WebGL: REAL-TIME 3D RENDERING

the 3D API standard ‒ OpenGL ES™ in a browser

‒ JavaScript API bindings

‒ supported in all modern browsers

‒ shipped since early 2011

supported in

• desktop Safari, Firefox, Chrome, Opera, IE

• Android – mobile Chrome, mobile Firefox

• FireOS (Kindle Fire HDX)

• Blackberry, Tizen, Firefox OS

• Surface (Windows 8.1)

• iOS mobile Safari – iAds only

• 500M+ seats -> 1B

…and it’s awesome.

100,000 Stars Google Experiment


http://workshop.chromeexperiments.com/stars/

http://chrysaora.com/

http://workshop.chromeexperiments.com/stars/


WebGL APPLICATIONS

advertising and media products and e-commerce

data visualization page graphics


http://makeourmark.levi.com/project-overview-whatmovesyou.html

http://workshop.chromeexperiments.com/projects/armsglobe/

http://acko.net/blog/making-mathbox/

http://vizi.gl/engine/tests/futurgo.html


JUST HOW RICH?

ported in 5 days Unreal native C++ engine -> JavaScript Emscripten + asm.js 60FPS

Epic Games’ Unreal Citadel Running in Firefox http://www.unrealengine.com/html5/


http://www.unrealengine.com/html5/




HOW WebGL WORKS

it’s a JavaScript drawing API

‒ draw to a canvas element using a special context (“webgl”)

‒ low-level drawing – buffers, primitives, textures and shaders

‒ accelerated by graphics hardware (GPU)

‒ can draw 2D as well as 3D graphics

integrates seamlessly with other page content

there is no file format; no markup language; no DOM.

libraries and frameworks are key to fast ramp up and productive development



A SIMPLE WebGL PROGRAM

1. create a <canvas> element

2. obtain a drawing context

3. initialize the viewport

4. create one or more buffers

5. create one or more matrices

6. create one or more shaders

7. initialize the shaders

8. draw one or more primitives



CREATE THE CANVAS, CONTEXT AND VIEWPORT

function initWebGL(canvas) { var gl = null; var msg = "Your browser does not support WebGL, " + "or it is not enabled by default."; try { gl = canvas.getContext(“webgl"); } catch (e) { msg = "Error creating WebGL Context!: " + e.toString(); } if (!gl) { alert(msg); throw new Error(msg); } return gl; } function initViewport(gl, canvas) { gl.viewport(0, 0, canvas.width, canvas.height); }

detect WebGL

set WebGL drawing region



BUFFERS AND TYPED ARRAYS

var vertexBuffer; vertexBuffer = gl.createBuffer(); gl.bindBuffer(gl.ARRAY_BUFFER, vertexBuffer); var verts = [ // Front face -1.0, -1.0, 1.0, 1.0, -1.0, 1.0, 1.0, 1.0, 1.0, -1.0, 1.0, 1.0, // Back face -1.0, -1.0, -1.0, -1.0, 1.0, -1.0, 1.0, 1.0, -1.0, 1.0, -1.0, -1.0, … ]; gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(verts), gl.STATIC_DRAW);

WebGL drawing functions use buffers of data

Typed Arrays: new low-level data type stores arrays of floats and ints compactly



SHADERS

var vertexShaderSource = " attribute vec3 vertexPos;\n" + " attribute vec2 texCoord;\n" + " uniform mat4 modelViewMatrix;\n" + " uniform mat4 projectionMatrix;\n" + " varying vec2 vTexCoord;\n" + " void main(void) {\n" + " // Return the transformed and projected vertex value\n" + " gl_Position = projectionMatrix * modelViewMatrix * \n" + " vec4(vertexPos, 1.0);\n" + " // Output the texture coordinate in vTexCoord\n" + " vTexCoord = texCoord;\n" + " }\n"; var fragmentShaderSource = " precision mediump float;\n" + " varying vec2 vTexCoord;\n" + " uniform sampler2D uSampler;\n" + " void main(void) {\n" + " // Return the pixel color: always output white\n" + " gl_FragColor = texture2D(uSampler, vec2(vTexCoord.s, vTexCoord.t));\n" + "}\n";

the vertex shader transforms model-space positions into screen space

the fragment shader outputs a color value for each pixel



DRAWING

function draw(gl, obj) { // clear the background (with black) gl.clearColor(0.0, 0.0, 0.0, 1.0); gl.enable(gl.DEPTH_TEST); gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT); // set the shader to use gl.useProgram(shaderProgram); // connect up the shader parameters: vertex position, texture coordinate, // projection/model matrices and texture // set up the buffers gl.bindBuffer(gl.ARRAY_BUFFER, obj.buffer); gl.vertexAttribPointer(shaderVertexPositionAttribute, obj.vertSize, gl.FLOAT, false, 0, 0); gl.bindBuffer(gl.ARRAY_BUFFER, obj.texCoordBuffer); gl.vertexAttribPointer(shaderTexCoordAttribute, obj.texCoordSize, gl.FLOAT, false, 0, 0); gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, obj.indices); gl.uniformMatrix4fv(shaderProjectionMatrixUniform, false, projectionMatrix); gl.uniformMatrix4fv(shaderModelViewMatrixUniform, false, modelViewMatrix); gl.activeTexture(gl.TEXTURE0); gl.bindTexture(gl.TEXTURE_2D, webGLTexture); gl.uniform1i(shaderSamplerUniform, 0); // draw the object gl.drawElements(obj.primtype, obj.nIndices, gl.UNSIGNED_SHORT, 0); }

clear the canvas

set the shader

set up buffers for vertices and texture coordinates

pass transform and projection matrices to the shader

set the texture and pass to the shader

draw the object



A WebGL CLIENT-SIDE STACK

rendering – Three.js library

animation – Three.js, Tween.js libraries

application functionality – game engine/framework

‒ setup/teardown

‒ interaction – picking, rollovers, rotating models

‒ behaviors

‒ run loop and event dispatching

content creation pipeline

‒ 3D tools e.g. Autodesk Maya

‒ COLLADA, glTF formats; conversion tools



Three.js: A JAVASCRIPT 3D ENGINE

the most popular WebGL library

renderer = new THREE.WebGLRenderer( { canvas: canvas, antialias: true } ); scene = new THREE.Scene(); camera = new THREE.PerspectiveCamera( 45, canvas.width / canvas.height, 1, 4000 ); scene.add(camera); var light = new THREE.DirectionalLight( 0xffffff, 1.5); scene.add( light ); var mapUrl ="../images/webgl-logo-256.jpg“; var map = THREE.ImageUtils.loadTexture(mapUrl ); var material = new THREE.MeshPhongMaterial({ map: map }); var geometry = new THREE.CubeGeometry(2, 2, 2); cube = new THREE.Mesh(geometry, material); scene.add( cube );

https://github.com/mrdoob/three.js/

represents WebGL at a high level using standard 3D graphics concepts

can render to WebGL, 2D canvas, SVG and CSS3




3D ANIMATION

requestAnimationFrame() http://www.paulirish.com/2011/requestanimationframe-for-smart-animating/

with JavaScript we can animate anything: materials, lights, textures…. Shaders

Three.js has support for key frames, morphs and skins

Tween.js – simple tweening library https://github.com/sole/tween.js/

var tween = new TWEEN.Tween( { x: 50, y: 0 } ) .to( { x: 400 }, 2000 ) .easing( TWEEN.Easing.Elastic.InOut ) .start();

function animate() { requestAnimationFrame( animate ); TWEEN.update(); }

create and run tween

call TWEEN.update() each frame to update values


http://www.paulirish.com/2011/requestanimationframe-for-smart-animating/







https://github.com/sole/tween.js/

https://github.com/sole/tween.js/

http://threejs.org/examples/webgl_morphtargets_md2_control.html


WebGL PIPELINE TOOLS

WebGL has no file format; no markup language; no DOM. ‒ apps/libraries must define their own formats

a lot of people are still creating content by writing code

existing 3D formats proprietary, incomplete or ill-suited for web/mobile delivery

help is on the way: glTF ‒ a “JPEG for 3D”

‒ bridges the gap between existing 3D formats/tools and today’s GL based APIs

‒ compact, efficient to load representation ‒ JSON scene description and high-level objects ‒ binary vector and animation data

a common publishing format for content tools ‒ open specification; open process

‒ Khronos Group initiative within the COLLADA working group ‒ F. Robinet (lead), R. Arnaud, P. Cozzi, T. Parisi

‒ https://github.com/KhronosGroup/glTF ‒ http://gltf.gl/

glTF Three.js Loader Prototype


https://github.com/KhronosGroup/glTF

http://gltf.gl/


WebGL ENGINES AND FRAMEWORKS

game engines/IDEs

PlayCanvas http://www.playcanvas.com/

Turbulenz https://turbulenz.com/

Goo Enginehttp://www.gootechnologies.com/

Artillery Engine https://artillery.com/

Verold http://verold.com/

Sketchfab https://sketchfab.com/

Unreal… ? http://www.extremetech.com/gaming/151900-unreal-engine-3-ported-to-javascript-and-webgl-works-in-any-modern-browser

libraries/frameworks

Three.js

http://threejs.org/

SceneJS

http://scenejs.org/

BabylonJS

http://www.babylonjs.com/

Voodoo.js

http://www.voodoojs.com/

tQuery

http://jeromeetienne.github.io/tquery/

Vizi

https://github.com/tparisi/Vizi

Provide application-level functionality

‒ setup/teardown

‒ interaction – picking, rollovers, rotating models

‒ behaviors

‒ run loop and event dispatching

‒ FSM objects (states)

‒ physics, dynamics, collision, …

‒ view models, camera controllers, …


http://www.playcanvas.com/

https://turbulenz.com/

http://www.gootechnologies.com/

http://www.gootechnologies.com/

https://artillery.com/



http://verold.com/

https://sketchfab.com/

http://www.extremetech.com/gaming/151900-unreal-engine-3-ported-to-javascript-and-webgl-works-in-any-modern-browser



























http://threejs.org/

http://threejs.org/

http://threejs.org/

http://scenejs.org/

http://scenejs.org/







https://github.com/tparisi/Vizi


CSS3: ADVANCED PAGE EFFECTS

CSS transitions

CSS animations

CSS 3D transforms

CSS filter effects

CSS custom filters



CSS3 TRANSITIONS

allow gradual changes to CSS properties over time

#box img { -webkit-transition: transform .5s ease-in; -webkit-transform: translate3d(0, -350px, 0); }

#box img:hover { -webkit-transition: transform .5s ease-in; -webkit-transform: translate3d(0, 0px, 0); cursor: pointer; }

-webkit-transition: all .5s ease-in -.1s;

transition-property: all; transition-duration: .5s; transition-timing-function: ease-in; transition-delay: .1s;

source: http://www.kirupa.com/html5/all_about_css_transitions.htm

shortcut for

specify property to transition, duration of transition, easing function

new CSS3 properties currently require vendor-specific prefixes


http://www.kirupa.com/html5/all_about_css_transitions.htm




CSS3 ANIMATIONS

allow fine control over animation behavior with key frames

#bigcloud { animation: bobble 2s infinite; }

@keyframes bobble { 0% { transform: translate3d(50px, 40px, 0px); animation-timing-function: ease-in; } 50% { transform: translate3d(50px, 50px, 0px); animation-timing-function: ease-out; }

100% { transform: translate3d(50px, 40px, 0px); } }

keys are used to interpolate transform values

control timing with ease in/out functions

each key frame contains a transform

source: http://www.kirupa.com/html5/all_about_css_animations.htm


http://www.kirupa.com/html5/all_about_css_animations.htm




CSS3 3D TRANSFORMS

translate, rotate, scale page elements with perspective

.bk-list li { perspective: 1800px; }

.bk-list li .bk-front { transform-style: preserve-3d; transform-origin: 0% 50%; transform: translate3d(0,0,20px); }

.bk-list li .bk-book.bk-bookdefault:hover { transform: rotate3d(0,1,0,35deg); }

http://tympanus.net/codrops/2013/01/08/3d-book-showcase/

browser will render element in 3D perspective

add origin to translation

apply to child elements

apply 35 degree rotation about Y axis Using 3D transform forces hardware acceleration


http://localhost/3DBookShowcase/index.html









CSS3 FILTER EFFECTS

apply post-processing effects to image elements

source: http://html5-demos.appspot.com/static/css/filters/index.html

img { -webkit-filter: sepia(100%); } img { -webkit-filter: grayscale(100%); } img { -webkit-filter: blur(2px); } img { -webkit-filter: brightness(33%); } img { -webkit-filter: contrast(67%); }


http://html5-demos.appspot.com/static/css/filters/index.html






CSS3 CUSTOM FILTERS

.shader { -webkit-filter: custom(url(shaders/crumple.vs) mix(url(shaders/crumple.fs) multiply source-atop), 50 50, amount 0, strength 0.2, lightIntensity 1.05, transform rotateX(0deg) translateZ(0px) ); -webkit-transition: -webkit-filter linear 1.5s; box-shadow: 0 0 2em #111; } .shader:hover { -webkit-filter: custom(url(shaders/crumple.vs) mix(url(shaders/crumple.fs) multiply source-atop), 50 50, amount 1, strength 0.2, lightIntensity 1.05, transform rotateX(0deg) translateZ(0px) ); }

source: http://alteredqualia.com/css-shaders/crumple.html

Crumple shader by Branislav Ulicny (AlteredQualia) http://alteredqualia.com/css-shaders/crumple.html


http://localhost/CSS Filters/Crumple.html

http://alteredqualia.com/css-shaders/crumple.html










CSS3 VERTEX SHADER

void main() { // Compute displacement float time = 10.0 * quadraticInOut( amount ); vec2 surfaceCoordinate = 0.025 * a_texCoord; float n = surface( surfaceCoordinate, time ); float curve = strength * n; vec4 pos = a_position; pos.z = amount * ( 0.5 * strength - curve ); // Compute normal const float e = 0.001; float nx = surface( surfaceCoordinate + vec2( e, 0.0 ), time ); float ny = surface( surfaceCoordinate + vec2( 0.0, e ), time ); vec3 normal = normalize( vec3( n - nx, n - ny, 1.0 - strength * 1.25 ) ); // Compute lighting (directional light) vec3 lightPosition = normalize( vec3( 0.0, 0.5, 1.0 ) ); float lightWeight = lightIntensity * max( dot( normal, lightPosition ), 0.0 ); // Set vertex position gl_Position = u_projectionMatrix * perspective( 0.9 ) * transform * pos; v_uv = a_texCoord; v_height = n; v_light = lightWeight;

browser predefines a_texCoord and a_position, passes into shader

shader outputs new vertex position, passes lighting information to fragment shader



CSS3 FRAGMENT SHADER

// Uniform values from CSS uniform float amount; // Varyings passed in from vertex shader varying vec2 v_uv; varying float v_height; varying float v_light; void main() { const float a = 1.0; float r, g, b; // Light variant float n = v_light; float v = mix( 1.0, n * n, amount ); r = g = b = sqrt( v ); // Set color matrix css_ColorMatrix = mat4( r, 0.0, 0.0, 0.0, 0.0, g, 0.0, 0.0, 0.0, 0.0, b, 0.0, 0.0, 0.0, 0.0, a ); }

Fragment shader outputs blend value based on light calculated in vertex shader and user-supplied amount



CSS3 SUPPORT

CSS3 Transitions

supported since • Safari 3.2: 13/11/2008 • Firefox 4.0: Late 2010 • Chrome 1.0: 02/09/2008 • Opera 10.5: 02/03/2010 • Internet Explorer 10: 09/2011

CSS3 Animations

supported since • Safari 4.0: 11/06/2008 • Chrome 1.0: 02/09/2008 • Firefox 5: 20/04/2011 • IE 10: 09/2011 • Opera: 03/2012

CSS 3D Transforms

supported since • Safari 4.0: 11/06/2008 • Chrome: 28/08/2010 • IE 10: 09/2011

• Firefox: 27/10/2011

source: http://css3.bradshawenterprises.com/

CSS3 Filters

supported since • Safari 6.0 • Chrome: 18.0

CSS3 Custom Filters

supported only in Chrome


http://css3.bradshawenterprises.com/


FUN WITH THREE.JS AND CSS3

http://mrdoob.github.io/three.js/examples/css3d_periodictable.html






LANGUAGE INNOVATIONS

asm.js

optimizable, low-level subset of JavaScript; strictly typed; improves performance (2x native vs. 3-10x) runs in FF and Chrome nightlies

http://asmjs.org/

Emscripten

LLVM-to-JavaScript compiler translates C++ native code to JavaScript; can output to asm.js

https://github.com/kripken/emscripten/wiki

new Web languages - Dart, TypeScript

optionally strongly typed; class-based

https://www.dartlang.org/

http://www.typescriptlang.org/

tackling performance and memory challenges


http://asmjs.org/

http://asmjs.org/








RESOURCES

WebGL specification http://www.khronos.org/registry/webgl/specs/latest/

Learning WebGL http://www.learningwebgl.com/

Three.js https://github.com/mrdoob/three.js/

requestAnimationFrame http://www.paulirish.com/2011/requestanimationframe-for-smart-animating/

CSS specifications http://www.w3.org/TR/css3-animations/ http://www.w3.org/TR/css3-transitions/ http://www.w3.org/TR/css3-transforms/ http://www.w3.org/TR/filter-effects/

CSS animation demos http://www.creativebloq.com/css3/animation-with-css3-712437

CSS 3D transforms explained http://desandro.github.io/3dtransforms/

Adobe filter lab - playing with filter effectshttp://html.adobe.com/webplatform/graphics/customfilters/cssfilterlab/

Alan Greenblatt’s blog – CSS filter effects http://blattchat.com/


http://www.khronos.org/registry/webgl/specs/latest/















http://www.w3.org/TR/css3-animations/




http://www.w3.org/TR/css3-transitions/




http://www.w3.org/TR/css3-transforms/




http://www.w3.org/TR/filter-effects/




http://www.creativebloq.com/css3/animation-with-css3-712437







http://desandro.github.io/3dtransforms/



http://html.adobe.com/webplatform/graphics/customfilters/cssfilterlab/





http://blattchat.com/




STAY IN TOUCH…

contact information

[email protected]

skype: auradeluxe

http://twitter.com/auradeluxe http://www.tonyp

arisi.com/


get the books!

WebGL: Up and Running

http://www.amazon.com/dp/144932357X Programming 3D Applications with HTML and WebGL

http://www.amazon.com/Programming-Applications-HTML5-WebGL-Visualization/dp/1449362966

book source code












http://www.amazon.com/dp/144932357X

http://www.amazon.com/dp/144932357X















WT-4064, Build Rich Applications with HTML5 and WebGL, by Tony Parisi

Technology

webgl context

webgl drawing functions

build rich applications

webgl applications advertising

firefox http

webgl tony parisi

simple webgl program1

var gl