Glam - A Declarative Language for 3D Web Content

A DECLARATIVE LANGUAGE FOR

3D WEB CONTENT

TONY PARISISEPTEMBER, 2014

GLAM:

04/10/2023

http://www.tonyparisi.com

CODE IS HARD.I LIKE MARKUP!

THE WEB BROWSER IS NOW A 3D PLATFORM

POWERFUL APIS PROVIDE EVERYTHING WE NEED: GRAPHICS, COMPOSITING, ANIMATION TIMING, THREADING, NETWORKING…

INSANE 3D APPLICATIONS ARE NOW POSSIBLE

API APPROACH PROVIDES MAXIMUM FLEXIBILITY – RENDER WHATEVER YOU WANT WITHOUT THE BROWSER NEEDING TO SUPPORT NEW OBJECTS

BUT THE PENDULUM HAS SWUNG TOO FAR

JAVASCRIPT MUCKING AND JSON HEFTING REQUIRED, EVEN FOR SIMPLE STUFF

NO EASY WAY TO STYLE 3D CONTENT

WE NEED A SIMPLE WAY TO CREATE 3D CONTENT IN PAGES

THAT CAN BE STYLED AND PROGRAMMED LIKE ANY OTHER PAGE CONTENT

04/10/2023


DON’T BE SQUARE

LET’S MAKE A CUBE, BABY!

04/10/2023


WebGwhatnow?var vertexBuffer;

vertexBuffer = gl.createBuffer();gl.bindBuffer(gl.ARRAY_BUFFER, vertexBuffer);

var verts = [ // Front face -1.0, -1.0, 1.0, 1.0, -1.0, 1.0, 1.0, 1.0, 1.0, -1.0, 1.0, 1.0,

// Back face -1.0, -1.0, -1.0, -1.0, 1.0, -1.0, 1.0, 1.0, -1.0, 1.0, -1.0, -1.0, …];

gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(verts), gl.STATIC_DRAW);

var vertexShaderSource =

" attribute vec3 vertexPos;\n" +" attribute vec2 texCoord;\n" +" uniform mat4 modelViewMatrix;\n" +" uniform mat4 projectionMatrix;\n" +" varying vec2 vTexCoord;\n" +" void main(void) {\n" +" gl_Position = projectionMatrix * modelViewMatrix * \n" +" vec4(vertexPos, 1.0);\n" +" // Output the texture coordinate in vTexCoord\n" +" vTexCoord = texCoord;\n" +" }\n";

var fragmentShaderSource = " precision mediump float;\n" +" varying vec2 vTexCoord;\n" +" uniform sampler2D uSampler;\n" + " void main(void) {\n" +" // Return the pixel color: always output white\n" +

" gl_FragColor = texture2D(uSampler, vec2(vTexCoord.s…);\n" + "}\n";

function draw(gl, obj) {

// clear the background (with black) gl.clearColor(0.0, 0.0, 0.0, 1.0); gl.enable(gl.DEPTH_TEST); gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT); // set the shader to use gl.useProgram(shaderProgram);

// connect up the shader parameters: position, tex coord // projection/model matrices and texture // set up the buffers gl.bindBuffer(gl.ARRAY_BUFFER, obj.buffer); gl.vertexAttribPointer(shaderVertexPositionAttribute, obj.vertSize, gl.FLOAT, false, 0, 0); gl.bindBuffer(gl.ARRAY_BUFFER, obj.texCoordBuffer); gl.vertexAttribPointer(shaderTexCoordAttribute, obj.texCoordSize, gl.FLOAT, false, 0, 0); gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, obj.indices); gl.uniformMatrix4fv(shaderProjectionMatrixUniform, false, projectionMatrix); gl.uniformMatrix4fv(shaderModelViewMatrixUniform, false, modelViewMatrix);

gl.activeTexture(gl.TEXTURE0); gl.bindTexture(gl.TEXTURE_2D, webGLTexture); gl.uniform1i(shaderSamplerUniform, 0);

// draw the object gl.drawElements(obj.primtype, obj.nIndices, gl.UNSIGNED_SHORT, 0); }

1. CREATE BUFFERS

2. DEFINE VERTEX AND FRAGMENT SHADERS

3. DRAW THE CUBE

300 LINES OF JAVASCRIPT. UNIFORMS AND SHADY

THINGS.NOT GROOVY.

04/10/2023


Three.jsA 3D LIBRARY WITH MOJO

renderer = new THREE.WebGLRenderer( { canvas: canvas, antialias: true } );

scene = new THREE.Scene();

camera = new THREE.PerspectiveCamera( 45, canvas.width / canvas.height, 1, 4000 );scene.add(camera);

var light = new THREE.DirectionalLight( 0xffffff, 1.5);scene.add( light );

var mapUrl ="../images/webgl-logo-256.jpg“;var map = THREE.ImageUtils.loadTexture(mapUrl );var material = new THREE.MeshPhongMaterial({ map: map });

var geometry = new THREE.CubeGeometry(2, 2, 2);cube = new THREE.Mesh(geometry, material);scene.add( cube );

1. CREATE RENDERER. CREATE SCENE. ADD TEXTURE MAP. DRAW CUBE. DONE.

40 LINES OF JAVASCRIPT.

THAT’S COOL.

TURN AND FACE THE STRANGE

IT TAKES 40 LINES OF JAVASCRIPT CODE TO MAKE A CUBE.

ABOUT THAT, YES

BUT THE CUBE SPINS, RIGHT?NOPE - THAT’S LOTS MORE LINES OF CODE

NO MARKUP LANGUAGE? I HAVE TO WRITE CODE?

YES

BUT SURELY I CAN STYLE IT WITH CSS?

ER… NO…

THEY CAN PUT A MAN ON THE MOON (THEY WILL DO SOON!) BUT I CAN’T JUST SAY

<cube>

THAT’S MOST ODD.

04/10/2023


INTRODUCING:GLAM

GL AND MARKUP

DECLARATIVE TAGS FOR DEFINING 3D VISUALS AND SCENE STRUCTURE

SHAPES – 3D MESHES, LINES, SOLIDS, PATHS, EXTRUSIONS

ANIMATIONS

CAMERAS

LIGHTS

GROUPING AND SCENE HIERARCHY

COMPLEX MODEL/SCENE IMPORT

PARTICLES, POST-PROCESSING EFFECTS, VR RENDERING AND OTHER FUN

EXTENDED CSS SYNTAX FOR STYLING AND ANIMATIONS

DOM-LIKE APICREATING ELEMENTS

CHANGING PROPERTIES

HANDLING EVENTS

DEFAULT BEHAVIORS BASED ON COMMON USAGE PATTERNS

FULLY EXTENSIBLE USING JAVASCRIPT AND GLSL SHADERS

RENDERED WITH WEBGL, BUILT WITH WEB COMPONENTS AND/OR POLYFILLS

04/10/2023


EIGHT LINE CUBE

<glam> <scene> <cube id="photocube”></cube> </scene></glam>

1. THE HTML

#photocube { image:url(../images/bowieorangeglam.png);}

2. THE CSS

NOW THAT’S GLAM.

Anthony Parisi

maybe skip bowie texture, go back to flower

04/10/2023


THE ASCENT OF GLAMA BRIEF HISTORY OF GRAPHICS MARKUP

VRML 1 VRML 2 X3D XAML GLAM

1994 1997 2004 2008 2014

STATICMODELS,LO RES

RENDERING

+SIMPLEANIMATION

VRML + XML+COMPLEXANIMATION,

+HI RES (FOR WEB)RENDERING

PLUGIN REQUIRED

MARKUP,FULL PAGE

INTEGRATION,LAME 3DIE ONLY

GAME-QUALITY 3D,NO PLUGIN,

FULL PAGE INTEGRATIONAND NOW MARKUP!

1999

SVG

2DONLY

04/10/2023


SHAPESAND

MATERIALS

polygon meshes, geometry primitives, extruded text

phong shading, specular highlights,wireframe rendering,transparency, textures

<cube id="texturedcube" x="1.5" y='2' z='-5' depth='.5' height='1.5' ></cube><sphere id="sphere1" radius='1.5'></sphere><cone id="cone1" x='-1' y='-.5' z='2' rz='90deg'></cone><cylinder id="cylinder1"></cylinder><text id="text1" value="glam"></text>

set visual properties via elements’ attributes – the DOM way!

04/10/2023


GROUPSAND

HIERARCHY

groups’ transform values apply to children <group y='-1'>

<cube id="texturedcube" x="1.5" y='2' z='-5' depth='.5' height='1.5' ></cube><group id="aGroup" x="1" y=".5" z="-2" >

<sphere id="sphere1" radius='1.5'></sphere><group id="nestedGroup" x='-3' y='1' z='-5'>

<cone id="cone1" x='-1' y='-.5' z='2' rz='90deg'></cone><cylinder id="cylinder1"></cylinder>

</group></group><text id="text1" value="glam"></text>

</group>

groups can nest

04/10/2023


INTERACTIONvar photocube = document.getElementById('photocube');

photocube.addEventListener("mouseover", function(event) {stat.innerHTML = 'mouseover';photocube.setAttribute('color-diffuse', 'red');

});photocube.addEventListener("mouseout", function(event) {

stat.innerHTML = 'mouseout';photocube.setAttribute('color-diffuse', 'white');

});photocube.addEventListener("mousedown", function(event) {

stat.innerHTML = 'mousedown';});photocube.addEventListener("mouseup", function(event) {

stat.innerHTML = 'mouseup';});photocube.addEventListener("mousemove", function(event) {

stat.innerHTML = 'mousemove';});photocube.addEventListener("click", function(event) {

stat.innerHTML = 'click';});

mouse click on items in 3D generates DOM-style events; just add a listener to deal with it

…JUST LIKE DOM USED TO MAKE!

04/10/2023


ANIMATION

<scene> <cube id="photocube" animation="animRotateY"></cube> <animation id="animRotateY" duration="10s" iteration-count="infinite" timing-function="linear">

<keyframe time="0%" property="transform" value="rotateY(0deg);"></keyframe><keyframe time="50%" property="transform" value="rotateY(180deg);"></keyframe><keyframe time="100%" property="transform" value="rotateY(360deg);"></keyframe>

</animation> </scene>

OR target this node with named animation – similar to SVG animation

@-webkit-keykeyframes kfRotateY { from { -webkit-transform: rotateY(0deg); }

to { -webkit-transform: rotateY(360deg); }}.animRotateY { -webkit-animation-duration: 2s; -webkit-animation-name: kfRotateY; -webkit-animation-iteration-count: infinite; -webkit-animation-timing-function:linear;}

animate 3D scene with CSS Animations!

04/10/2023


STYLING#photocube {

image:url(../images/ash_uvgrid01.jpg);transform: translateX(-1.5) translateZ(-5) rotateY(30deg);

}#cone1 {

color:blue;radius:1;shader:phong;

}#cylinder1 {

color:green;radius:1;render-mode:wireframe;

}#sphere1 {

radius:1.5;shader:phong;diffuse-color:red;specular-color:#004400;opacity:.8;

}

transform properties (standard CSS3): translate, rotate, scale, skew, matrix

texture map(custom property)

visual CSS properties for materials

size and position properties for shapes

ALL WITH GOOD OLD CSS!

TOO BAD THE BROWSER THROWS OUT UNKNOWN PROPERTIES… I HAD

TO WRITE A CUSTOM PARSER! (BASED ON JQUERY PLUGIN)

04/10/2023


IMPORTING MODELS

<import src="../models/futurgo_city/futurgo_city.dae”></import>

EMBED POPULAR/STANDARD 3D FORMATS

WE CAN’T LIVE ON CUBES ALONE…

MODEL FORMATS – OBJ, STLSIMPLE MODELS, NO SCENE HIERARCHY, CAMERAS, LIGHTS OR ANIMATION

ANIMATION FORMATS – BVH, MD2, MD5

SCENE FORMATS – COLLADA, glTFSUPPORT FULL SCENE STRUCTURE, CAMERAS, LIGHTS, ANIMATIONS

04/10/2023


SHADERS AND EXTENSIBILITY

WE CAN’T THINK OF EVERY POSSIBLE COOL SHAPE, MATERIAL AND BEHAVIOR

GLSL SHADER LANGUAGE LETS US CONTROL EVERY VERTEX AND PIXEL

.bubble {radius:.5;color:lightgray;vertex-shader:url(../shaders/fresnel.vs);fragment-shader:url(../shaders/fresnel.fs);shader-uniforms:mRefractionRatio f 1.02 mFresnelBias f 0.1 mFresnelPower f 2.0

mFresnelScale f 1.0 tCube t null;}

declare vertex and fragment shaders in CSS

declare shader language inputs so engine knows how to pass in values

04/10/2023


DOM APIfunction createBubble(scene, x, y, z, animClass) {

var groupelt = document.createElement('group');

groupelt.setAttribute('x', x);groupelt.setAttribute('z', z);groupelt.setAttribute('y', y);

var sphereelt = document.createElement('sphere');sphereelt.setAttribute('class', "bubble skybox");

sphereelt.addEventListener("click", function(event) {pop.pause();pop.currentTime = 0;pop.play();scene.removeChild(groupelt);

});

sphereelt.setAttribute('animation', animClass);

groupelt.appendChild(sphereelt);

return groupelt;}

mouse click on items in 3D generates DOM-style events; just add a listener to deal with it

use standard DOM methods to add/remove children

get/set values using standard DOM method

04/10/2023


POST-PROCESSING AND EFFECTS

PARTICLE SYSTEMS… IN JUST A FEW TAGS

PIXELATE, BLUR, SCREEN, DEPTH OF FIELD…IN JUST A FEW TAGS

<particles maxAge='2' id="fire"> <emitter particlesPerSecond='50' size='10' sizeEnd='5' opacityStart='0.2’ opacityMiddle='0.4' opacityEnd='0.1' velocity='0 2.5 0' acceleration='0 3 0' positionSpread='2 3.5 2' accelerationSpread='1.5 .5 1.5' colorStart="yellow blending='additive' > </emitter></particles>

particle system age and emitter params are defined in the markup

post-processing passesaffect the whole scene

#fire { image:url(../../images/smokeparticle.png);}

particle sprite is defined in the CSS

<scene> <cube id="photocube" class=""></cube> <effect type="RGBShift" amount=".005"> </effect> <effect type="DotScreenRGB" scale='2'> </effect></scene>

04/10/2023


VR + ML

<glam> <renderer type="rift"></renderer> <scene> <controller type="rift"></controller> <background id="sb1" class="skybox"></background> <group y ='1' z='-3'> <sphere class="bubble skybox”></sphere> <sphere x='-1' z='-2' class="bubble skybox”></sphere> </group> </scene></glam>

set up VR renderer

set up VR controller to track head motion

04/10/2023


IMPLEMENTATION (I)THE RENDERING/ANIMATION STACK

VIZI – A COMPONENT-BASED FRAMEWORK FOR BUILDING INTERACTIVE 3D APPLICATIONS

https://github.com/tparisi/vizi

THREE.JS – A JAVASCRIPT 3D RENDERING LIBRARY/ENGINE

https://github.com/mrdoob/three.js/

TWEEN.JS – A SIMPLE BUT POWERFUL TWEENING UTILITY

https://github.com/sole/tween.js

https://github.com/tparisi/Vizi








04/10/2023


IMPLEMENTATION (II)BROWSER STUFF

BROWSER PARSES UNKNOWN DOM ELEMENTS; DOESN’T KNOW WHAT TO DO WITH THE DOM TREE BUT SAVES IT ANYWAY

WE TRAVERSE THAT TREE AND BUILD 3D RENDERING/ANIMATION STRUCTURESWE USE MUTATION OBSERVERS TO MONITOR CHANGES TO THE DOM AND UPDATE THE LIVE SCENE GRAPH

BROWSER IGNORES UNKNOWN CSSGLAM WAS BUILT WITH A JQUERY CSS PARSER PLUGIN

IMPLEMENTATION (III)

04/10/2023


WHAT ABOUT WEB COMPONENTS?

LOVE ‘EM. THEY LOOK LIKE THE FUTURE. BUT I STARTED THIS PROJECT BEFORE THEY WERE GETTING TRACTION. NOW I NEED TO GO BACK AND REWORK THIS TO USE THEM.

BUT…

WEB COMPONENTS AREN’T SUPPORTED IN RETAIL BROWSERS YET – YOU NEED NIGHTLIES

WE CAN POLYFILL WITH POLYMER… BUT IT’S HARDCORE AND I'M A SOFTIEhttp://www.polymer-project.org/

BUT… BUT… BUT…

WEB COMPONENTS-Y PEOPLE SEEM TO BE AVERSE TO EXTENDING CSS*. MAYBE IT’S JUST ME BUT I THINK 3D NEEDS NEW ATTRIBUTES (Z, DEPTH, RADIUS, NORMAL-IMAGE, SPECULAR-COLOR ETC.)

POLYMER TAKES A HANDS-OFF ATTITUDE – JUST DO EVERYTHING IN ELEMENT ATTRIBUTES, THE “NEW WAY?”

* WHAT’S THE BLOODY POINT OF BUILDING AN EXTENSIBLE MARKUP SYSTEMTHAT IGNORES THE BLOODY STYLING? BLOODY HELL…

http://www.polymer-project.org/

http://www.polymer-project.org/

04/10/2023


PROJECT STATUS

WE ARE AT ABOUT V0.7… NEEDS REVIEW, REFACTORING AND DESIGN INPUT

WE NEED MORE DEMOS AND EXAMPLES, DOCUMENTATIONAND WE NEED CONTRIBUTORS!

PORT TO WEB COMPONENTS/POLYMERI STARTED THIS PROJECT BEFORE WEB COMPONENTS WAS TAKING OFF; IT’S TIME TO REFACTOR IT SO AS TO NOT REINVENT WHEELS.

DISCUSS WITH BROWSER MAKERS– WE COULD USE HELP WITH CSS EXTENSIBILITY. WE NEED THOSE CUSTOM PROPERTIES.

I THINK THIS IS CRITICAL; BUT IS IT JUST ME?

CAN YOU DIG IT?DECLARE YOURSELF: GET INVOLVED. HELP ME MAKE THIS REAL.

http://www.tonyparisi.com 04/10/2023

CAN YOU DIG IT?

[email protected]: auradeluxehttp://twitter.com/auradeluxe http://www.tonyparisi.com/http://www.learningwebgl.com/

GET THE BOOKS!

WebGL: Up and Runninghttp://www.amazon.com/dp/144932357XProgramming 3D Applications with HTML and WebGLhttp://www.amazon.com/Programming-Applications-HTML5-WebGL-Visualization/dp/1449362966

GET VIZIhttps://github.com/tparisi/Vizi

MEETUPShttp://www.meetup.com/WebGL-Developers-Meetup/

http://www.meetup.com/Web-VR/

BOOK CODEhttps://github.com/tparisi/WebGLBook

https://github.com/tparisi/Programming3DApplications

GET GLAMhttp://www.glamjs.org/https://github.com/tparisi/glam/

WORKhttp://www.vizi.gl/

CREDSCo-creator, VRML and X3D

mailto:[email protected]

http://twitter.com/auradeluxe

http://www.tonyparisi.com/

http://www.tonyparisi.com/

http://www.learningwebgl.com/

http://www.learningwebgl.com/

http://www.amazon.com/dp/144932357X

http://www.amazon.com/dp/144932357X

http://www.amazon.com/Programming-Applications-HTML5-WebGL-Visualization/dp/1449362966









http://www.meetup.com/WebGL-Developers-Meetup/

http://www.meetup.com/WebGL-Developers-Meetup/



https://github.com/tparisi/WebGLBook

https://github.com/tparisi/WebGLBook



http://www.glamjs.org/



https://github.com/tparisi/glam/



http://www.vizi.gl/

Glam - A Declarative Language for 3D Web Content

Technology

modelviewmatrix gl

projectionmatrix gl

glam gl

buffers gl

createbuffer gl

test gl

texture0 gl

indices gl