Elastic Image Software LLC iOS OpenGL Douglass Turner Elastic Image Software email: [email protected] mobile: 781 775 3708
May 11, 2015
Elastic Image Software LLC
iOS OpenGL
Douglass TurnerElastic Image Software
email: [email protected]: 781 775 3708
Elastic Image Software LLC
2
A Talk in Three Acts
Elastic Image Software LLC
• Introduction to Computer Graphics• Computer Graphics in OpenGL• OpenGL and CocoaTouch Playing Nice Together
3
Elastic Image Software LLC
4
Introduction to Computer Graphics
A Trip Down the Graphics Pipeline
Elastic Image Software LLC
Copyright Steve Jubinville5
Build and Tesselate Model into Facets
Elastic Image Software LLC
Copyright Steve Jubinville6
Pose Model
Elastic Image Software LLC
Aim Camera
7
Elastic Image Software LLC
8
Design the Look
Elastic Image Software LLC
Design Appearance. Light Scene.
Copyright © 2006 Holger Schömann9
Elastic Image Software LLC
Copyright © 2006 Florian Wild
Design Appearance. Light Scene.
10
Elastic Image Software LLC
Copyright © 2006 Jack Qiao
Design Appearance. Light Scene.
11
Elastic Image Software LLC
12
Render Scene
Elastic Image Software LLC
13
Clip scene to the bounds of the viewing frustrum.
Elastic Image Software LLC
Rasterize Scene
14
Elastic Image Software LLC
15
for each object in the scene
! determine its corresponding pixels
! for each corresponding pixel
! ! calculate depth of object at pixel
! ! if depth is closer than any object yet seen
! ! ! put this object in the pixel
! ! endif
! end
end
The GPU implements a z-buffer algorithm in hardware
Elastic Image Software LLC
In OpenGL, the Look is evaluated at the same time that depth comparison is performed.
Together these are the core activities of computer graphics. They are the raison d'être of the GPU. Compute intensive, Highly optimized and parallelized.
16
Elastic Image Software LLC
17
Update Model State. This could entail geometric transformation or perhaps taking a time step in the physic engine.
Elastic Image Software LLC
18
Rinse, repeat.
Elastic Image Software LLC
19
Example.
Beautiful Panoramas iPad app. In the App Store: http://bit.ly/QVIDgg
Elastic Image Software LLC
20
Computer Graphics in OpenGL
Elastic Image Software LLC
OpenGL Cocoa
21
Elastic Image Software LLC
OpenGL is Old School
22
Elastic Image Software LLC
You are flipping levers on a large state machine.
23
Elastic Image Software LLC
OpenGL is brutally spare. Just the essentials.
float oldSchool[]
glPushMatrix()
glPopMatrix()!
24
Elastic Image Software LLCKeep in mind OpenGL is a dragster, not a Prius. So, just buckle up ...
25
Elastic Image Software LLC
... and enjoy the ride, baby!
26
Elastic Image Software LLC
27
Posing models. Positioning light sources. Aiming the camera. All these task require and understanding of transformation matrices.
Elastic Image Software LLC
28
Transformation matrix. Coordinate frameWorld space, Screen space, etc.
All are different ways of thinking about the same underlying concept.
Elastic Image Software LLC
29
When we move an object we apply a transformation via a transformation matrix. This transformation is defined in terms of a coordinate frame.
A specific location on the transformed object can be interpreted in terms of any coordinate frame we choose.
Elastic Image Software LLC
30
The motivation for a coordinate frame transformation is not only motivated by the need to pose a model or aim a camera as in the physical world.
In computer graphics land it often has to do with the convenience of performing a particular calculation in one space rather then another.
Elastic Image Software LLC
31
We refer to the transformation pipeline: the successive application of a series of transformations as a 3D model wends its way from application land to GPU land.
Elastic Image Software LLC
Hierarchical modeling is based on the hierarchical organization of transformations matrices. Many objects - trees, robots, cars - form natural hierarchies.
32
Elastic Image Software LLC
33
The UIView class supports all the key concepts of transformations and hierarchy. It is a nice, friendly way to gain intuition about facility with this powerful concept.
Elastic Image Software LLC
34
Hierarchical modeling in Cocoa Touch
Elastic Image Software LLC
35
struct CGAffineTransform { CGFloat a, b, c, d; CGFloat tx, ty;};
struct CATransform3D { CGFloat m11, m12, m13, m14; CGFloat m21, m22, m23, m24; CGFloat m31, m32, m33, m34; CGFloat m41, m42, m43, m44;};
Transforms
Elastic Image Software LLC
36
UIView properties
// Coordinate frame in parent space@property(nonatomic) CGRect frame;
// Dimensions of view in object space@property(nonatomic) CGRect bounds;
// View center in parent space @property(nonatomic) CGPoint center;
// View transformation matrix@property(nonatomic) CGAffineTransform transform;
// Parent view@property(nonatomic,readonly) UIView *superview;
// Child views@property(nonatomic,readonly,copy) NSArray *subviews;
Elastic Image Software LLC
37
UIView methods
// Transform 2D point from/to another view’s coordinate frame- (CGPoint)convertPoint:(CGPoint)point toView:(UIView *)view;- (CGPoint)convertPoint:(CGPoint)point fromView:(UIView*)view;
// Transform rectangle from/to another view’s coordinate frame- (CGRect)convertRect:(CGRect)rect toView:(UIView *)view;- (CGRect)convertRect:(CGRect)rect fromView:(UIView*)view;
// Manipulate view hierarchy- (void)removeFromSuperview;- (void)insertSubview:(UIView *)view atIndex:(NSInteger)index;- (void)addSubview:(UIView *)view;- (void)insertSubview:(UIView *)view belowSubview:(UIView *)siblingSubview;- (void)insertSubview:(UIView *)view aboveSubview:(UIView *)siblingSubview;
Elastic Image Software LLC
38
- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions { [window addSubview:viewController.view]; [window makeKeyAndVisible]; return YES;}
View hierarchy in iOS
Elastic Image Software LLC
39
Transformation matrices in iOS
- (void)setTransform:(CGAffineTransform)newValue {
// Scale along the y-axis only CGAffineTransform constrainedTransform = CGAffineTransformScale(CGAffineTransformIdentity, 1.0, newValue.a); newValue.a = constrainedTransform.a; newValue.b = constrainedTransform.b; newValue.c = constrainedTransform.c; newValue.d = constrainedTransform.d;
[super setTransform:newValue];}
Elastic Image Software LLC
HelloTeapot Demo and Code Walkthrough
http://github.com/turner/HelloTeapot
40
Elastic Image Software LLC
In what follows, we will be referring to the ES1 “fixed-function” transformation pipeline provided by the GPU.
In ES2 (GLSL) the transformation pipeline must be handled entirely in application space.
41
Elastic Image Software LLC
42
Elastic Image Software LLC
43
A camera with a spotlight attached to it orbits around a stationary teapot model lit by three stationary colored point light sources.
Elastic Image Software LLC
In iOS we never explicitly say “render scene”. Rather we inform the system that now would be a good time to redisplay the scene:
[myView setNeedsDisplay] and [myView drawRect]
44
Elastic Image Software LLC
- (void)startAnimation {
animationTimer = [NSTimer scheduledTimerWithTimeInterval:animationInterval target:self selector:@selector(drawView) userInfo:nil repeats:YES];}
45
In OpenGL we manage rendering ourselves
Elastic Image Software LLC
46
In OpenGL we manage rendering ourselves
- (void)startAnimation {! if (!self.isAnimating) {! ! self.displayLink = [NSClassFromString(@"CADisplayLink") displayLinkWithTarget:self selector:@selector(drawView:)];
[self.displayLink setFrameInterval:animationFrameInterval]; [self.displayLink addToRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
! ! ! self.animating = YES;! ! } // if (!self.isAnimating)!}
Elastic Image Software LLC
! Position the camera
! glMatrixMode(GL_MODELVIEW);!! glLoadIdentity(); ! glLoadMatrixf(_openGLCameraInverseTransform);
47
http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.m
Elastic Image Software LLC
Position spotlight at camera location glPushMatrix();! glMultMatrixf(_cameraTransform);!!! // A white spotlight for a camera headlight! GLfloat spotLightColor[] = { 1.0, 1.0, 1.0, 1.0 }; GLfloat spotLightPosition[] = { 0.0, 0.0, 2.0, 1.0 }; GLfloat spotLightDirection[] = { 0.0, 0.0, -1.0, 1.0 }; GLfloat spotCutOff!! ! = 3.5;!! glEnable(GL_LIGHT3);! glLightfv(GL_LIGHT3, GL_DIFFUSE, spotLightColor);!! glLightfv(GL_LIGHT3, GL_POSITION, spotLightPosition);! glLightfv(GL_LIGHT3, GL_SPOT_DIRECTION, spotLightDirection);! glLightf(GL_LIGHT3, GL_SPOT_CUTOFF, spotCutOff);!! glPopMatrix();!
48
http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.m
Elastic Image Software LLC
49
glPushMatrix & glPopMatrix
These functions push/pop a transformation matrix on/off the transformation stack.
This stack is the key data structure for presenting a hierarchical model to the GPU in fixed-function OpenGL.
Elastic Image Software LLC
50
glPushMatrix - Create a new matrix and push it on the stack. Subsequent matrix concatenation is done with stack.top.
stack.top is the current coordinate frame.
glPopMatrix - Pop the stack. The pre-existing matrix is now stack.top.
We revert to the coordinate frame the existed prior to the push.
! glPushMatrix();! glMultMatrixf(_cameraTransform);!! glEnable(GL_LIGHT3);! glLightfv(GL_LIGHT3, GL_DIFFUSE, spotLight);! glPopMatrix();!
Elastic Image Software LLC
51
http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.m
Position the red stationary light source
! const GLfloat redLightColor[]! ! = { 1.0f, 0.0f, 0.0f, 1.0f };! const GLfloat redLightPosition[]! = { 10.0f, 0.0f, 0.0f, 1.0f }; ! glEnable(GL_LIGHT0);! glLightfv(GL_LIGHT0, GL_DIFFUSE, redLightColor);!! glLightfv(GL_LIGHT0, GL_POSITION, redLightPosition);
Elastic Image Software LLC
52
http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.m
Position the green stationary light source
! const GLfloat greenLightColor[]! = { 0.0f, 1.0f, 0.0f, 1.0f };! const GLfloat greenLightPosition[]!= { 0.0f, 10.0f, 0.0f, 1.0f }; ! glEnable(GL_LIGHT0);! glLightfv(GL_LIGHT0, GL_DIFFUSE, greenLightColor);!! glLightfv(GL_LIGHT0, GL_POSITION, greenLightPosition);
Elastic Image Software LLC
53
http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.m
Position the blue stationary light source
! const GLfloat blueLightColor[]!! = { 0.0f, 0.0f, 1.0f, 1.0f };! const GLfloat blueLightPosition[]! = { 0.0f, 0.0f, 10.0f, 1.0f }; ! glEnable(GL_LIGHT0);! glLightfv(GL_LIGHT0, GL_DIFFUSE, blueLightColor);!! glLightfv(GL_LIGHT0, GL_POSITION, blueLightPosition);
Elastic Image Software LLC
54
http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.m
Position and draw the teapot
! glPushMatrix();
! JLMMatrix3DSetZRotationUsingDegrees(rotation, -45.0f);! JLMMatrix3DSetScaling(scale, sx, sy, sz);! JLMMatrix3DMultiply(rotation, scale, concatenation);! glMultMatrixf(concatenation);
! for(int i = 0; i < num_teapot_indices; i += new_teapot_indicies[i] + 1) {
! ! glDrawElements( GL_TRIANGLE_STRIP, indices[i], GL_UNSIGNED_SHORT, &indices[i+1] );
! } // for (num_teapot_indices)
! glPopMatrix();
Elastic Image Software LLC
55
http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.m
Position and draw the teapot
! glPushMatrix();
! JLMMatrix3DSetZRotationUsingDegrees(rotation, -45.0f);! JLMMatrix3DSetScaling(scale, sx, sy, sz);! JLMMatrix3DMultiply(rotation, scale, concatenation);
! glMultMatrixf(concatenation);
! for(int i = 0; i < num_teapot_indices; i += new_teapot_indicies[i] + 1) {
! ! glDrawElements( GL_TRIANGLE_STRIP, indices[i], GL_UNSIGNED_SHORT, &indices[i+1] );
! } // for (num_teapot_indices)
! glPopMatrix();
Elastic Image Software LLC
56
http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.m
Position and draw the teapot
! glPushMatrix();
! JLMMatrix3DSetZRotationUsingDegrees(rotation, -45.0f);! JLMMatrix3DSetScaling(scale, sx, sy, sz);! JLMMatrix3DMultiply(rotation, scale, concatenation);! glMultMatrixf(concatenation);
! for(int i = 0; i < num_teapot_indices; i += new_teapot_indicies[i] + 1) {
! ! glDrawElements( GL_TRIANGLE_STRIP, indices[i], GL_UNSIGNED_SHORT, &indices[i+1] );
! } // for (num_teapot_indices)
! glPopMatrix();
Elastic Image Software LLC
57
Notice the absence of GL_CAMERA?
You may be surprised to learn there is no concept of camera in OpenGL. Camera posing is equivalent to inverting the camera transform and applying it to the scene observed by the camera.
Elastic Image Software LLC
camera teapot
world
wTtwTc
58
Elastic Image Software LLC
camera teapot
glMatrixMode(GL_MODELVIEW);glLoadMatrixf(_openGLCameraInverseTransform);
?
cTt
59
Elastic Image Software LLC
camera teapot
world
wTt(wTc)
cTt
cTt wTtcTw *=
-1
60
Elastic Image Software LLC
cTt wTtcTw *=
61
Elastic Image Software LLC
- (void)placeCameraAtLocation:(M3DVector3f)location target:(M3DVector3f)target up:(M3DVector3f)up;
62
http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.m
Elastic Image Software LLC
The vertices comprising the 3D geometry of the teapot
float teapot_vertices [] = { 0.0663, 0.1178, 0.0, 0.0672, 0.1152, 0.0, 0.0639, 0.1178, 0.0178043,
... };
The surface normals associated with each vertex
// The list of surface normals corresponding to the verticesfloat teapot_normals[] = { -0.987635, -0.156768, 0, -0.902861, -0.429933, 0, -0.953562, -0.156989, -0.257047,
... };
63
Elastic Image Software LLC
The teapot model is defined as a collection of triangle stripes that combine to form the teapot surface.
short indicies[] = {
// how many vertices in vertex strip26,
// vertex strip indices1122, 1243, 1272, 1242, ... ,1283, 1199, ... };
64
Elastic Image Software LLC
glEnableClientState(GL_VERTEX_ARRAY);glVertexPointer(3 ,GL_FLOAT, 0, teapot_vertices);
glEnableClientState(GL_NORMAL_ARRAY);glNormalPointer(GL_FLOAT, 0, teapot_normals);glEnable(GL_NORMALIZE);!
65
! for(int i = 0; i < num_teapot_indices; i += new_teapot_indicies[i] + 1) {
! ! glDrawElements( GL_TRIANGLE_STRIP, indices[i], GL_UNSIGNED_SHORT, &indices[i+1] );
! }
Elastic Image Software LLC
Design the Look
66
Elastic Image Software LLC
67
We now begin leaving behind the fixed-function pipeline of ES1 and focus on ES2, GLSL, and visual coolness in general.
Elastic Image Software LLC
The Look: the lighting, illumination model, and surface shaders that in combination achieve the desired look of a model in a scene.
68
Elastic Image Software LLC
Lights
69
Elastic Image Software LLC
70
Point Light Directional Light
Spot Light Area Light
Elastic Image Software LLC
71
Elastic Image Software LLC
Illumination Model
72
Elastic Image Software LLC
Ambient Diffuse
SpecularDiffuse
+Specular
73
Elastic Image Software LLC
Diffuse(NormalVector, LightVector)Ambient(ConstantReflectance)
Specular(LightReflectionVector, EyeVector) Shiny(Specular, Diffuse)74
Elastic Image Software LLC
Surface Shaders
75
Elastic Image Software LLC
Surface shaders were introduced to the field of computer graphics by Pixar with its rendering API and shading language called RenderMan.
A shader is a small functions evaluated at every location on a being rendered.
GLSL borrows heavily from the RenderMan model.
76
Elastic Image Software LLC
Copyright © Douglass Turner77
I created this gallery of shaders using a rendering/shading system I wrote that was inspired by RenderMan.
Elastic Image Software LLC
78
Copyright © Douglass Turner
Elastic Image Software LLC ZBrush Modeling Daniel LieskeMaterial Design Ralf Stumpf
79
Elastic Image Software LLCCopyright: Stephen Molyneaux
80
Elastic Image Software LLC
Hello Shader Demo and Code Walkthrough
https://github.com/turner/HelloShader
81
Elastic Image Software LLC
82
Before we dive into shaders lets get a lay of the land. There are two flavors of shaders in GLSL.
A vertex shader is evaluated at each vertex.
A fragment shader is evaluated at each screen space pixel corresponding to a sampled on the facet being rasterized.
Elastic Image Software LLC
83
Vertex and fragment shaders work together in a pipeline fashion. Vertex attributes - color, surface normal, texture coordinate - are evaluated in the vertex shader then passed on to the fragment shader where those values are interpolated across the surface.
Elastic Image Software LLC
84
ShowST. A shader that visualizes the texture coordinate parameterization of a surface.
Elastic Image Software LLC
85
https://github.com/turner/HelloShader/blob/master/HelloShader/Shaders/ShowST.fsh
// ShowST.vshattribute highp vec4 myVertexXYZ;attribute highp vec2 myVertexST;
// P * V * M - Projection spaceuniform mediump mat4 myProjectionViewModelMatrix;
varying vec2 v_st;void main() {
gl_Position = myProjectionViewModelMatrix * myVertexXYZ; v_st = myVertexST;
}
ShowST - vertex shader
Elastic Image Software LLC
86
attribute vec2 myVertexST - Vertex attribute
varying vec2 v_st - Varies across the surface
uniform sampler2D myTexture_0 - Invariant throughout rendering cycle
Shader variables come in different flavors
Elastic Image Software LLC
87
https://github.com/turner/HelloShader/blob/master/HelloShader/Shaders/ShowST.fsh
Vertex attribute myVertexST is a texture coordinate passed to the fragment shader via varying variable v_st.
// ShowST.vshattribute highp vec4 myVertexXYZ;attribute highp vec2 myVertexST;
// P * V * M - Projection spaceuniform mediump mat4 myProjectionViewModelMatrix;
varying vec2 v_st;void main() {
gl_Position = myProjectionViewModelMatrix * myVertexXYZ; v_st = myVertexST;
}
Elastic Image Software LLC
88
https://github.com/turner/HelloShader/blob/master/HelloShader/Shaders/ShowST.fsh
// ShowST.fshvarying vec2 v_st;void main() { // Visualize the s-t parameterization of the underlying surface gl_FragColor.r = v_st.s; gl_FragColor.g = v_st.t; gl_FragColor.b = 0.0; gl_FragColor.a = 1.0;}
ShowST - fragment shader
Elastic Image Software LLC
89
http://github.com/turner/HelloiPadGLSL/blob/master/Shaders/ShowST.fsh
// ShowST.fshvarying vec2 v_st;void main() { // Visualize the s-t parameterization of the underlying surface gl_FragColor.r = v_st.s; gl_FragColor.g = v_st.t; gl_FragColor.b = 0.0; gl_FragColor.a = 1.0;}
The texture coordinate values v_st.s and v_st.t are for the red and green channels of the shader color.
Elastic Image Software LLC
90
TexturePairShader. A shader that mixes the colors of a pair of textures using the tweaked alpha channel of one of the pair.
Fail Whale over Mandril
Elastic Image Software LLC
91
TexturePairShader - vertex shader
https://github.com/turner/HelloShader/blob/master/HelloShader/Shaders/TEITexturePairShader.vsh
// TEITexturePairShader.vshattribute mediump vec2 myVertexST;attribute highp vec4 myVertexXYZ;
// P * V * M - Projection spaceuniform mediump mat4 myProjectionViewModelMatrix;
varying mediump vec2 v_st;
void main() {
gl_Position = myProjectionViewModelMatrix * myVertexXYZ;
v_st = myVertexST;}
Elastic Image Software LLC
92
TexturePairShader - vertex shader
// TEITexturePairShader.vshattribute mediump vec2 myVertexST;attribute highp vec4 myVertexXYZ;
// P * V * M - Projection spaceuniform mediump mat4 myProjectionViewModelMatrix;
varying mediump vec2 v_st;
void main() {
gl_Position = myProjectionViewModelMatrix * myVertexXYZ;
v_st = myVertexST;}
https://github.com/turner/HelloShader/blob/master/HelloShader/Shaders/TEITexturePairShader.vsh
Elastic Image Software LLC
93
TexturePairShader - fragment shader
// TEITexturePairShader.fshvarying mediump vec2 v_st;uniform sampler2D myTexture_0;uniform sampler2D myTexture_1;void main() {
vec4 rgba_0 = texture2D(myTexture_0, v_st); vec4 rgba_1 = texture2D(myTexture_1, v_st);
// Interpolate between the two textures using the alpha// channel of texture 0 as the interpolant.
float interpolate = rgba_0.a; gl_FragColor.r = rgba_0.r + (1.0 - interpolate) * rgba_1.r; gl_FragColor.g = rgba_0.g + (1.0 - interpolate) * rgba_1.g; gl_FragColor.b = rgba_0.b + (1.0 - interpolate) * rgba_1.b; gl_FragColor.a = rgba_0.a + (1.0 - interpolate) * rgba_1.a;}
https://github.com/turner/HelloShader/blob/master/HelloShader/Shaders/TEITexturePairShader.vsh
Elastic Image Software LLC
94
attribute vec2 myVertexST - Vertex attribute
varying vec2 v_st - Varies across the surface
uniform sampler2D myTexture_0 - Invariant throughout rendering cycle
Shader variables come in different flavors
Elastic Image Software LLC
95
An application communicates with its shaders in the follow ways:
• attributes - geometry, color, normal, texture coordinate• uniforms - texture sampler, matrix, eye vector, light vector• textures - texture channels - r, g, b, a - are not just for color
attribute vec2 myVertexST - Vertex attribute
varying vec2 v_st - Varies across the surface
uniform sampler2D myTexture_0 - Invariant throughout rendering cycle
Elastic Image Software LLC
96
Lets look at how we wire our iOS app together with our shaders. We will use a texture shading example.
https://github.com/turner/HelloShader/blob/master/HelloShader/Classes/Renderer/GLRenderer.m
Elastic Image Software LLC
97
First make a texture object ...
TEITexture *t = [[ [TEITexture alloc] initWithImageFile:@"twitter_fail_whale_red_channnel_knockout" extension:@"png" mipmap:YES ] autorelease];[glView.renderer.rendererHelper.renderables setObject:t forKey:@"texture_0"];
Elastic Image Software LLC
98
Load shaders, Bind shader attribute to application handle.
// Vertex shader NSString *vertShaderPathname = [[NSBundle mainBundle] pathForResource:@"TEITextureShader" ofType:@"vsh"];
[self compileShader:&vertShader type:GL_VERTEX_SHADER file:vertShaderPathname];!
// Fragment shader NSString *fragShaderPathname = [[NSBundle mainBundle] pathForResource:@"TEITextureShader" ofType:@"fsh"];
[self compileShader:&fragShader type:GL_FRAGMENT_SHADER file:fragShaderPathname];!
// Bind attribute glBindAttribLocation(m_program, VertexSTAttributeHandle,! "myVertexST");
Elastic Image Software LLC
99
Associate shader uniform with application object instance ...
TEITexture *t = (TEITexture *)[self.rendererHelper.renderables objectForKey:@"texture_0"];
t.location = glGetUniformLocation(m_program, "myTexture_0");
glActiveTexture( GL_TEXTURE0 );
glBindTexture(GL_TEXTURE_2D, t.name);
glUniform1i(t.location, 0);
Activate a texture unit, bind it to a texture object, and assign a number to the corresponding texture sampler used in the fragment shader
Elastic Image Software LLC
100
We now have a linkage between our application texture object and a texture sampler in our fragment shader. Rock.
Elastic Image Software LLC
101
Elastic Image Software LLC
102
Elastic Image Software LLC
103
Lets Talk Textures
Elastic Image Software LLC
104
Texture mapping is the fundamental tool for creating visual complexity
Elastic Image Software LLC
105
By establishing a mapping between a surface and a texture we can “attach” the texture to the surface.
The interpretation of the a can go far beyond that of a decal to be applied to a surface. Bumps, opacity, displacement, and much more can be designed with a texture.
Elastic Image Software LLC
106
Elastic Image Software LLC
107
Shaders can have it both ways: Compute surface color (or surface roughness, or opacity) algorithmically or look it up in a texture. Or a bit of both.
Elastic Image Software LLC
OpenGL and iOS
Playing Nice Together
108
Elastic Image Software LLC
• OpenGL is mostly M• Keep V dumb
• Lots of chatter between M and C
MVC
109
Elastic Image Software LLC
110
Lets take a look at how iOS and OpenGL interact. We will us Hello iPad OpenGL as our guide.
https://github.com/turner/HelloShader
Elastic Image Software LLC
111
-(id)initializeEAGL {!! CAEAGLLayer *eaglLayer = (CAEAGLLayer *)self.layer;!! eaglLayer.opaque = TRUE;! eaglLayer.drawableProperties = ! [NSDictionary dictionaryWithObjectsAndKeys:! [NSNumber numberWithBool:FALSE], kEAGLDrawablePropertyRetainedBacking, ! kEAGLColorFormatRGBA8, kEAGLDrawablePropertyColorFormat, ! nil];!! self.renderer = [[[ES2Renderer alloc] init] autorelease];!! m_animating = FALSE;! animationFrameInterval = 1;! m_displayLink = nil;!! return self;!}
The view’s layer is used as the rendering surface
Elastic Image Software LLC
112
Rendering frame rate is sync-ed to the screen refresh rate. The actual rendering is handled via a selector.
- (void)startAnimation { if (!self.isAnimating) { self.displayLink = [NSClassFromString(@"CADisplayLink") displayLinkWithTarget:self selector:@selector(drawView:)]; [self.displayLink setFrameInterval:animationFrameInterval]; [self.displayLink addToRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode]; self.animating = YES; } // if (!self.isAnimating)}
Elastic Image Software LLC
113
// framebuffer glGenFramebuffers(1, &m_framebuffer); glBindFramebuffer(GL_FRAMEBUFFER, m_framebuffer); // rgb buffer glGenRenderbuffers(1, &m_colorbuffer); glBindRenderbuffer(GL_RENDERBUFFER, m_colorbuffer); glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_RENDERBUFFER, m_colorbuffer); [m_context renderbufferStorage:GL_RENDERBUFFER fromDrawable:layer]; glGetRenderbufferParameteriv(GL_RENDERBUFFER, GL_RENDERBUFFER_WIDTH, &m_backingWidth); glGetRenderbufferParameteriv(GL_RENDERBUFFER, GL_RENDERBUFFER_HEIGHT, &m_backingHeight); // z-buffer glGenRenderbuffers(1, &m_depthbuffer); glBindRenderbuffer(GL_RENDERBUFFER, m_depthbuffer); glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH_COMPONENT16, m_backingWidth, m_backingHeight); glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_RENDERBUFFER, m_depthbuffer);
A framebuffer object is created. A color and depth buffer are attached.
Elastic Image Software LLC
114
- (void) render {! ! [EAGLContext setCurrentContext:m_context]; glBindFramebuffer(GL_FRAMEBUFFER, m_framebuffer);
// transform, light, shade, etc.
glBindRenderbuffer(GL_RENDERBUFFER, m_colorbuffer); [m_context presentRenderbuffer:GL_RENDERBUFFER];}
The render loop. Draw to the colorbuffer then present to the display. This is the classic “ping pong” between back buffer and front buffer.
Elastic Image Software LLC
Thank You
Douglass TurnerElastic Image Software
email: [email protected]: 781 775 3708
115
Elastic Image Software LLC
116
Copyright © Douglass Turner
Elastic Image Software LLC