SpiderGL A JavaScript 3D Graphics Library for Next-Generation WWW Marco Di Benedetto, Federico Ponchio, Fabio Ganovelli, Roberto Scopigno Visual Computing Lab – ISTI – CNR
Dec 27, 2015
SpiderGL
A JavaScript 3D Graphics Libraryfor Next-Generation WWW
Marco Di Benedetto, Federico Ponchio, Fabio Ganovelli, Roberto Scopigno
Visual Computing Lab – ISTI – CNR
Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 2
3D Content and the WWW
Many attempts have been done so far ActiveX objects Proprietary browser extensions or plug-ins
Most libraries/plugins are too high level to directly access underlying graphics system Developers forced to comply to some paradigm (scene graphs) CG experts often find limitations due to high-level abstraction
Standardization needed WebGL : JavaScript bindings to OpenGL|ES 2.0
Outline
Introduction to the WebGL Rendering Pipeline
The SpiderGL Library
Case Study: Mesh Rendering
Library Features & Demos
Conclusions & Future Work
Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 3
Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 4
What is WebGL
Actually what will be (standardization in progress) Specification owned by the Khronos Group Supported (and defined) by all major web browsers devs
JavaScript bindings to OpenGL|ES 2.0 Almost 1-to-1 mapping, some modifications to meet
JS developers’ habits and security issues
Enables home computers and mobile devices to natively access 3D content directly from web pages(no external plug-ins)
Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 5
OpenGL|ES 2.0 Stripped version of OpenGL, focused on Embedded Systems
Programmable : NO fixed function pipeline Immediate mode (glBegin / End) Use vertex / index buffers Transform Stage (matrix stacks) Explicit shaders uniforms Lighting (lighting model, light sources, materials) Lighting computation
through shaders code Named vertex attributes (glVertexPointer(…), …) Generic attributes
through glVertexAttribPointer(index, …)
Data entirely resides on GL resources (buffers, textures) Buffer Centric API
Restrictions Texture formats, data types, shading language limitations, ...
Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 6
Framebuffer
OpenGL|ES 2.0: Architecture (simplified)
0 N-1…
Vertex Attributes Stream
Vertex Puller
Index Stream
Vertex Processor Fragment ProcessorPrimitive Assembler &Rasterizer
Output Merger
VS FS UNICode
Memory
Pixel Operations
VB RBIB TEXData
MemoryFB
Screen
0 M-1
Texture Units
…
Uniforms
Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 7
OpenGL|ES 2.0: API Structure Context
Capabilities Query Error Query Creation / Activation / Destruction not part of specifications (EGL for this)
Objects Resources creation / edit / bind / destruction Data Set & Get
Configurable Stages Enable / Disable Constants / Parameters Set & Get
Programmable Stages Accept user-defined programs (shaders)
Vertex Pipeline Geometric primitives draw (activate Vertex Puller)
Pixel Pipeline Framebuffer clear & readback (activate Pixel Operations)
Synchronization Command Buffer flush & wait
PipelineConfiguration
&Data
Execution
Motivations: Bring CG to the WWW WebGL as a link between Web and CG developers
But... it’s very low-level!
Gaps to be filled: Speed, of course Algebraic and Geometric structures & algorithms Asynchronous data fetch facilities Assets loading Ease the use of WebGL
SpiderGL aims at filling these gaps
Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 8
SpiderGL: 3D Graphics for WWW
Lightweight LGPL library ( http://spidergl.org ) provides typical CG data structures & algorithms
Sits on top of WebGL for realtime rendering
Goals: Efficiency
asymptotic bounds are not the only concern Simplicity & Short Learning Time
Key for fast application coding Flexibility
Help with common tasks and create a robust middle layer for more complicated designs
Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 9
Leading desing considerations
Using a third-party library should be easy Do not impose any design choice a priori
Avoid over-abstraction In CG simple and self-contained types works very well
Users must be able to reuse as much as possible of their former knowledge on the subject All the entities have de-facto standardized names and behavior Experienced users often want fine control low-level access
Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 10
SpiderGL Architecture
Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 11
GL: rendering
Mesh: editable or renderable 3D model
Space: geometric s. & a.
Async: asinchronous data requests managing
UI: user and html interface
Case Study: Mesh Rendering
How a typical 3D model is represented: A list of vertices (each vertex is a bundle of data) A list of vertex indices, specifying how they are connected to
form basic geometric primitives (points, segments, triangles)
What we need to draw it POV intrinsic and extrinsic parameters Some procedure to make light sources and model material
interact to form colors on the final image
Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 12
Vertex & Index Buffers
Shaders & Uniforms
Meshes in SpiderGL
Mesh Structure Set of named vertex attribute streams
(position, normal, color, custom, ...) with NO predefined semantic Set of named array or indexed primitive streams
(points list, triangulated surface indices, wireframe segments, ...)
Mesh twins: SglMeshJS: an editable data structure SglMeshGL: a renderable graphics resource
Unified interface, conversion functions
Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 13
Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 14
Create a Renderable Mesh
/*******************************************************************/var mesh = new SglMesh(gl);
// add a vertex attribute named “position” (type is inferred)mesh.addVertexAttribute("position", 3, nativePositions);
// add a NORMALIZED vertex attribute named “color”mesh.addVertexAttribute("color", 3, nativeColors, true);
// add connectivity information named “triangles” (type is inferred)mesh.addIndexedPrimitives("triangles", gl.TRIANGLES, nativeIndices);
// add “array” primitives named “vertices”mesh.addArrayPrimitives("vertices", gl.POINTS, 0, 3);/*******************************************************************/
Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 15
Create a Shader Program
/*******************************************************************/// vertex shader source code stringvar vertSource = "";vertSource += “uniform mat4 u_mvp; \n";vertSource += "...";vertSource += "attribute vec3 a_position; \n";vertSource += "attribute vec3 a_color; \n";vertSource += "...";
// fragment shader source code stringvar fragSource = "...";
// program setupvar prog = new SglProgram(gl, [vertSource], [fragSource]);/*******************************************************************/
Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 16
Stream Mapping The mesh has 2 vertex attribute streams (position & color) The vertex shader has 2 input vertex attributes A correspondence between the two sets must be established
/**********************************************/… … …vertSource += "attribute vec3 a_position; \n";vertSource += "attribute vec3 a_color; \n";… … …/**********************************************/
/**********************************************************/… … …mesh.addVertexAttribute("position", 3, nativePositions);mesh.addVertexAttribute("color", 3, nativeColors, true);… … …/**********************************************************/
Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 17
Rendering
/*******************************************************************/// <name : string> connect shader attr “name” to mesh attr “string”var streamMapping = {
a_position : "position",a_color : "color"
};
// <name : value> set program uniform “name” to value “value”var uniforms = {
u_mvp : getModelViewProjectionMatrix()};
// utility function (full control available through SglMeshRenderer)sglRenderMeshPrimitives(mesh, "triangles", prog, streamMapping, uniforms);/*******************************************************************/
Option Parameters
The GL module aims at simplifying common WebGL tasks Creation of WebGL objects
Textures, programs, framebuffers, ... Common sequence of several commands
SpiderGL offers reasonable but overridable defaults
Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 18
/*********************************************************/var textureOpts = {
minFilter : gl.LINEAR_MIPMAP_LINEAR,generateMipmap : true,onload : function () { ... }
};
// create texture from remote imagevar tex = new SglTexture2D(gl, "image_url", textureOpts);/*********************************************************/
Dealing with WebGL Objects
Contstructors: SpiderGL provides easy-to-use configurable functions to create WebGL Objects
Wrappers: each native handle can be wrapped by optimized high-level objects
Flexibility: experienced users may want direct low-level control: obj.handle native WebGL object obj.synchronize() update state after low-level usage
Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 19
Overcoming Mesh Limitations
Ex.: OpenGL|ES 2.0 (and so WebGL) does not allow 32 bit vertex indices
Limited to 64K vertices per chunk
SpiderGL uses packed-indexed primitive stream to seamlessly allow very large meshes Mesh is automatically subdivided
into sub-meshes
Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 20
On the Math / Geometry Side
Support CG-related linear algebra entities 2, 3, 4-dimensional vectors, 2x2, 3x3, 4x4 matrices, quaternions Two choices: operate on native JS arrays or boxing objects
Standard geometric entities Planes, spheres, boxes, ... Intersection queries
Transformation Stack
Editable Mesh (SglMeshJS) Several (and more coming) algorithms on trimeshes
Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 21
Assets, Asynchronous Fetch and UI
COLLADA, OBJ, JSON (and more coming) importers Every kind of handled document can be loaded with a
uniform asynchronous request interface Images, 3D Models, shaders code, ...
Dynamic priority queues for multiresolution Requests can be interrupted, priority can change, transfer
callbacks can be installed Special watcher object for completion of batched transfers
GLUT-like interface and Interactors first-person camera, trackball, ...
Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 22
What can be done with SpiderGL
Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 23
I’m tired up here...
Let’s see some demos!
http://spidergl.org/code.php
SpiderGL to Assist Content Creation
Multimedia Web repositories are widespread Images, Video, Audio, Text, ... What about 3D Repositories?
WebGL will bring HW Accelerated 3D Graphics to WWW Lots of 3D databases
We need an effective way to deploy content to users
MeShade: a content authoring tool Let’s have a look! http://spidergl.org/meshade/index.html
Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 24
Conclusions
SpiderGL as a Geometry processing and Visualization library
Do not over-abstract graphics objects, ease the completion of common task, allow low-level access
Real-time performances with WebGL with several case studies
Future Work: Continuous improvements, growing set of algorithms Toward an automated process to make high-end 3D content
to be available to handheld devices
Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 25
Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Los Angeles 26
Thank You!
http://spidergl.org