Geometry shader-based bump mapping setup Mark Kilgard NVIDIA Corporation February 5, 2007 Updated October 18, 2007
Sep 07, 2014
Geometry shader-based bump mapping setup
Mark KilgardNVIDIA Corporation
February 5, 2007Updated October 18, 2007
Overview
• GPU bump-mapping traditionally involves complicated “hard to get right” per-primitive setup math to compute how object-space quantities map into texture space– Existing art requires CPU work
• This invention uses a geometry shader to accomplish this more efficiently– Artist authors traditional textured 3D model– Authors height field with same texture parameterization as the
model’s decal texture– Vertex shader can animate & skin model– Geometry shader constructs object-to-texture space mapping
• Object-space light and view vectors can now be transformed to texture space
– Pixel shader does efficient texture-space bump mapping
Geometry Shader in CgTRIANGLE voidmd2bump_geometry(AttribArray<float4> position : POSITION, AttribArray<float2> texCoord : TEXCOORD0, AttribArray<float3> objPosition : TEXCOORD1, AttribArray<float3> objNormal : TEXCOORD2, AttribArray<float3> objView : TEXCOORD3, AttribArray<float3> objLight : TEXCOORD4){ float3 dXYZdU = objPosition[1] - objPosition[0]; float dSdU = texCoord[1].s - texCoord[0].s; float3 dXYZdV = objPosition[2] - objPosition[0]; float dSdV = texCoord[2].s - texCoord[0].s; float3 tangent = normalize(dSdV * dXYZdU - dSdU * dXYZdV); for (int i=0; i<3; i++) { float3 normal = objNormal[i], binormal = cross(tangent,normal); float3x3 basis = float3x3(tangent, binormal, normal); float3 surfaceLightVector : TEXCOORD1 = mul(basis, objLight[i]); float3 surfaceViewVector : TEXCOORD2 = mul(basis, objView[i]);
emitVertex(position[i], texCoord[i], surfaceLightVector, surfaceViewVector); }} warning: not tolerant of mirroring (see next version)
Canonical TriangleA B
C
A = ( u=0, v=0, x0, y0, z0, s0, t0 )B = ( u=1, v=0, x1, y1, z1, s1, t1 )C = ( u=0, v=1, x2, y2, z2, s2, t2 )
∂xyz / ∂u = (x1,y1,z1) - (x0,y0,z0)∂xyz / ∂v = (x2,y2,z2) - (x0,y0,z0)∂s / ∂u = s1 – s0∂t / ∂v = t2 – t0
v
u
vs
us
us
vxyz
vs
uxyz
vs
vxyz
us
uxyz
s
xyz
quotient rule for
partial derivatives:
us
vxyz
vs
uxyz
es
xyznormaliznormalizetangent
Forming anObject-to-Texture Space Basis
z
y
x
z
y
x
obj
obj
obj
tex
tex
tex
NzBzTz
NyByTy
NxBxTx
us
vxyz
vs
uxyz
TzTyTx normalize),,(tangent
normalvertex -per),,(normal NzNyNx
),,(),,(binormal NzNyNxTzTyTx assumes left-handedtexture space
provided by vertex shader
Transform Object-Space Vectors for Lighting to Texture-Space
z
y
x
z
y
x
L
L
L
L
L
L
NzBzTz
NyByTy
NxBxTx
z
y
x
z
y
x
V
V
V
V
V
V
NzBzTz
NyByTy
NxBxTx
object-spacetexture-space
light vector
view vector
Issue: Texture Mirroring
• Texture space is ad-hoc– Mapping defined by artists during model construction– Artists often “re-use” regions of texture space– Includes mirroring regions of the texture
• Example: bilateral symmetry of face or humanoid
• Mathematical consequence of mirroring– Signed area of triangle in texture space is negative– Requires reversal of the sense of gradients– Geometry shader can compensate
Compensating for Texture Mirroing
AtCtAsCs
AtBtAsBs
space in texture area signed
If signed area in texture space “less than” zero, compute basis with negated tangent:
z
y
x
z
y
x
obj
obj
obj
tex
tex
tex
NzBzTz
NyByTy
NxBxTx
2x2 determinant
Example of Mirroring
artist’s original decal derived height fieldRGB normal map
generated from height field
Visualization of3D model’s signed area
in texture spacegreen=front-facingred=back-facing
only half of faceappears in decal
Consequence on Lighting from Accounting for Mirroring
Bad: mirrored skull buckle and beltlighting wrong on left side (indents in)
Correct lighting
Mirroring-tolerantGeometry Shader in Cg
TRIANGLE voidmd2bump_geometry(AttribArray<float4> position : POSITION, AttribArray<float2> texCoord : TEXCOORD0, AttribArray<float3> objPosition : TEXCOORD1, AttribArray<float3> objNormal : TEXCOORD2, AttribArray<float3> objView : TEXCOORD3, AttribArray<float3> objLight : TEXCOORD4){ float3 dXYZdU = objPosition[1] - objPosition[0]; float dSdU = texCoord[1].s - texCoord[0].s; float3 dXYZdV = objPosition[2] - objPosition[0]; float dSdV = texCoord[2].s - texCoord[0].s; float3 tangent = normalize(dSdV * dXYZdU - dSdU * dXYZdV); float area = determinant(float2x2(dSTdV, dSTdU)); float3 orientedTangent = area >= 0 ? tangent : -tangent; for (int i=0; i<3; i++) { float3 normal = objNormal[i], binormal = cross(tangent,normal); float3x3 basis = float3x3(orientedTangent, binormal, normal); float3 surfaceLightVector : TEXCOORD1 = mul(basis, objLight[i]); float3 surfaceViewVector : TEXCOORD2 = mul(basis, objView[i]);
emitVertex(position[i], texCoord[i], surfaceLightVector, surfaceViewVector); }}
additional &changed codefor mirroring
Discarding Triangles Significantly Back Facing w.r.t. the Light
TRIANGLE voidmd2bump_geometry(AttribArray<float4> position : POSITION, AttribArray<float2> texCoord : TEXCOORD0, AttribArray<float3> objPosition : TEXCOORD1, AttribArray<float3> objNormal : TEXCOORD2, AttribArray<float3> objView : TEXCOORD3, AttribArray<float3> objLight : TEXCOORD4){ float3 dXYZdU = objPosition[1] - objPosition[0]; float dSdU = texCoord[1].s - texCoord[0].s; float3 dXYZdV = objPosition[2] - objPosition[0]; float dSdV = texCoord[2].s - texCoord[0].s; float3 tangent = normalize(dSdV * dXYZdU - dSdU * dXYZdV); float maxLightZ, maxLightThreshold = -0.3; for (int i=0; i<3; i++) { float3 normal = objNormal[i], binormal = cross(tangent,normal); float3x3 basis = float3x3(tangent, binormal, normal); float3 surfaceLightVector : TEXCOORD1 = mul(basis, objLight[i]); maxLightZ = i==0 ? normalize(surfaceLightVector).z : max(maxLightZ, normalize(surfaceLightVector).z); float3 surfaceViewVector : TEXCOORD2 = mul(basis, objView[i]);
if (i < 2 || maxLightZ > maxLightThreshold) emitVertex(position[i], texCoord[i], surfaceLightVector, surfaceViewVector); }}
Graphics pipeline dataflowwith geometry shader-based setup
application
Vertex shader
Primitive assembly
Geometry shader
Rasterizer
Fragment shader
Raster operations
framebuffer
•Animation and skinning of object-space vertices•Output texture coordinates•Output object-space light & view vectors•Output clip-space position
•Compute object-to-texture space basis for triangle•Transform object-space vectors to texture space•Output triangle
•Access perturbed normal from normal map texture•Compute lighting in texture space•Output resulting color
Sans Per-vertex Normals
us
vxyz
vs
uxyz
e
s
xyz
(Tx,Ty,Tz)
normaliz
normalize
tangent
ut
vxyz
vt
uxyz
e
t
xyz
BzByBx
normaliz
normalize
),,(tangent2
tangenttangent2),,(normal NzNyNx
Geometry Shader in Cg Sans Normals
TRIANGLE voidmd2bump_geometry_sans_normal( AttribArray<float4> position : POSITION, AttribArray<float2> texCoord : TEXCOORD0, AttribArray<float3> objPosition : TEXCOORD1, // IGNORE per-vertex normal! AttribArray<float3> objView : TEXCOORD3, AttribArray<float3> objLight : TEXCOORD4){ float3 dXYZdU = objPosition[1] - objPosition[0]; float2 dSTdU = texCoord[1] - texCoord[0]; float3 dXYZdV = objPosition[2] - objPosition[0]; float2 dSTdV = texCoord[2] - texCoord[0]; float3 tangent = normalize(dSTdV.s * dXYZdU - dSTdU.s * dXYZdV); float3 tangent2 = normalize(dSTdV.t * dXYZdU - dSTdU.t * dXYZdV); float area = determinant(float2x2(dSTdV, dSTdU)); tangent = area >= 0 ? tangent : -tangent; float3 normal = cross(tangent2,tangent); tangent2 = area >= 0 ? tangent2 : -tangent2; for (int i=0; i<3; i++) { float3x3 basis = float3x3(tangent, tangent2, normal); float3 surfaceLightVector : TEXCOORD1 = mul(basis, objLight[i]); float3 surfaceViewVector : TEXCOORD2 = mul(basis, objView[i]); emitVertex(position[i], texCoord[i], surfaceLightVector, surfaceViewVector); }}
Geometry shader setup with and without per-vertex normals
Setup without per-vertex normals, relyingon normalized gradients only; faceted look
Setup with per-vertex normals;smoother lighting appearance
Visualizing Tangent, Normal, and Bi-normals for Discontinuities
• Using per-vertex normals
Visualize normalsReasonably smooth
Visualize tangentsFlat per-triangle
Visualize tangentsSemi-smooth, semi-flat
Issue: Lighting Discontinuities
• Problem: Geometry shader computes single tangent vector for entire triangle– See tangent visualization (prior slide)– Still each vertex of triangle has its own unique
normal so basis has some variation of triangle
• This per-triangle “tangent flatness” can lead to lighting discontinuities
Lighting Discontinuities
• Example of unsightly discontinuities when applying geometry shader bump mapping setup to a sphere:
Dealing with Lighting Discontinuities
• Final shading when bump mapping can hide these discontinuities somewhat– Imperfect solution
• Perturb the normals within the normal map to compensate for the discontinuities– How?– I have some (unproven, uninvestigated) ideas
More Information and Examples• Get the Cg Toolkit
– Cg 3.1 is available now– http://developer.nvidia.com/cg-toolkit-download
• Or http://http.developer.nvidia.com/Cg/cg_3_1_0010.html– See the “Download” section of this page
• Find source code once installed in– examples/OpenGL/advanced/gs_md2render
gs_md2bumpGeometryshaderbump mapexample
gs_md2shadowGeometry
shaderbump map +
shadow volumesexample
Supplemental Slides
Key frameblended3D model
Decalskin
Bumpskin
Glossskin
GPURendering
Authored 3D Model Inputs
Animation via Key Framesor Vertex Skinning
GPURendering
Frame A
Frame B
Other possible key frames