04 Graphics for games 1 - University of Helsinki€¦ · 04. Graphics for games Juha Vihavainen University of Helsinki Outline Creating and drawing game entities from 2D images (

Introduction to Game ProgrammingIntroduction to Game ProgrammingAutumn 2017Autumn 2017Autumn 2017Autumn 2017

04. Graphics for games04. Graphics for games

Juha VihavainenJuha VihavainenJuha VihavainenJuha Vihavainen

University of HelsinkiUniversity of Helsinki

OutlineOutline

�� Creating and drawing game entities Creating and drawing game entities

�� from 2D images (from 2D images (spritessprites) to 3D models) to 3D models�� from 2D images (from 2D images (spritessprites) to 3D models) to 3D models

�� the position, dimensions, and visual data of game the position, dimensions, and visual data of game objects represented for computer graphicsobjects represented for computer graphics

�� 3D models ~ triangle meshes ~ geometry/shape 3D models ~ triangle meshes ~ geometry/shape (plus other info)(plus other info)

�� On rendering pipelineOn rendering pipeline�� On rendering pipelineOn rendering pipeline

�� Creating and using Creating and using virtual virtual (synthetic)(synthetic)inin--world world camera camera

223.3.20173.3.2017 Juha Vihavainen / University of HelsinkiJuha Vihavainen / University of Helsinki

Literature on 3D graphicsLiterature on 3D graphics

�� (Madhav, 2014) (Madhav, 2014) Ch. Ch. 22 2D 2D GraphicsGraphics, Ch. , Ch. 44 3D 3D Graphics, Graphics, Ch. Ch. 8 8 CamerasCameras

�� (Gregory, 2014) Ch. (Gregory, 2014) Ch. 1010The Rendering Engine, The Rendering Engine, pp. 443pp. 443--541; 541; 10.210.2�� (Gregory, 2014) Ch. (Gregory, 2014) Ch. 1010The Rendering Engine, The Rendering Engine, pp. 443pp. 443--541; 541; 10.210.2The Rendering Pipeline, p. 489.The Rendering Pipeline, p. 489.

�� Dunn Fletcher, Parberry Ian: Dunn Fletcher, Parberry Ian: 3D Math Primer for Graphics and 3D Math Primer for Graphics and Game DevelopmentGame Development. A K Peters/CRC Press, 2011.. A K Peters/CRC Press, 2011.

�� Edward Angel, Dave ShreinerEdward Angel, Dave Shreiner, Interactive Computer Graphics: A , Interactive Computer Graphics: A TopTop--Down Approach with ShaderDown Approach with Shader--Based OpenGL, Based OpenGL, 6th Ed. Addison6th Ed. Addison--Wesley, 2011.Wesley, 2011.Wesley, 2011.Wesley, 2011.

�� Tomas AkenineTomas Akenine--Moller, Eric Haines, Naty Hoffman, Moller, Eric Haines, Naty Hoffman, RealReal--Time Time RenderingRendering, 3rd Ed. A K Peters/CRC Press, 2008., 3rd Ed. A K Peters/CRC Press, 2008.

�� Junghyan Han, Junghyan Han, 3D Graphics for Game Programming3D Graphics for Game Programming. CRC, 2011. CRC, 2011

3.3.20173.3.2017 Juha Vihavainen / University of HelsinkiJuha Vihavainen / University of Helsinki 33

Motivation for linear algebraMotivation for linear algebra

�� A game is (essentially) a mathematical model of a virtual world A game is (essentially) a mathematical model of a virtual world simulated on a computer simulated on a computer

�� a game engine needs to keep track of the positions, orientations, and a game engine needs to keep track of the positions, orientations, and �� a game engine needs to keep track of the positions, orientations, and a game engine needs to keep track of the positions, orientations, and scales of objects, animate them in the game world, and transform scales of objects, animate them in the game world, and transform them into screen space so they can be rendered on screenthem into screen space so they can be rendered on screen

�� 3D objects are (almost always) made up of triangles, the vertices of 3D objects are (almost always) made up of triangles, the vertices of which are represented by vectors/pointswhich are represented by vectors/points

�� Mathematics pervades everything in game development Mathematics pervades everything in game development

games make use many branches of mathematics, e.g., trigonometry, games make use many branches of mathematics, e.g., trigonometry, �� games make use many branches of mathematics, e.g., trigonometry, games make use many branches of mathematics, e.g., trigonometry, algebra, and calculus (often probability, too)algebra, and calculus (often probability, too)

�� the most important kind of mathematics for a game programmer is the most important kind of mathematics for a game programmer is vectorvectorand and matrixmatrix mathmath, i.e., , i.e., linear algebralinear algebra


Some 3D terminologySome 3D terminology

�� RasterizationRasterization

�� set of algorithms that draw 3D objects into a 2D set of algorithms that draw 3D objects into a 2D color color bufferbuffer(part of (part of frame buffer frame buffer with other data)with other data)bufferbuffer(part of (part of frame buffer frame buffer with other data)with other data)

�� Modern computers use a Modern computers use a graphics processing unit graphics processing unit (GPU) that (GPU) that can do most rasterizationcan do most rasterization

�� but need to tell the GPU but need to tell the GPU whatwhatwe want to draw and we want to draw and howhow

�� Due to limited resources, can't achieve photoDue to limited resources, can't achieve photo--realistic imagesrealistic images�� Due to limited resources, can't achieve photoDue to limited resources, can't achieve photo--realistic imagesrealistic images

�� graphic graphic ((visualvisual)) artifact artifact = some unwanted result = some unwanted result (anomaly) from digital image processing (of using (anomaly) from digital image processing (of using approximate algorithms, or imprecise or corrupted data)approximate algorithms, or imprecise or corrupted data)

3.3.20173.3.2017 55Juha Vihavainen / University of HelsinkiJuha Vihavainen / University of Helsinki

Types of art assetsTypes of art assets (Joseph Hocking, Unity in Action, 2015, p.70)

textures + shaders


A graphics system (sketch)A graphics system (sketch)

generalappl.

�� Consists of: Consists of:

input devices, CPU, GPU, memory, frame buffer, output input devices, CPU, GPU, memory, frame buffer, output

matrix & othercomputations

�� input devices, CPU, GPU, memory, frame buffer, output input devices, CPU, GPU, memory, frame buffer, output devicesdevices

�� E.g., PCs, workstations, mobile phones, video game consoles, E.g., PCs, workstations, mobile phones, video game consoles, GPS systems, etc.GPS systems, etc.


CRT monitor basicsCRT monitor basics�� 2D image = an array of 2D image = an array of picture elements picture elements known as known as pixelspixels

�� for color displays, each pixel contains a for color displays, each pixel contains a redred, , greengreen, and , and bluebluesubsub--pixelpixel

an an RGBARGBAvalue value -- for Red, Green, and Blue plus for Red, Green, and Blue plus AlphaAlpha�� an an RGBARGBAvalue value -- for Red, Green, and Blue plus for Red, Green, and Blue plus AlphaAlpha(alpha = 0 => (alpha = 0 => totally transparent image/parttotally transparent image/part))

�� Resolution width x height determines the number of pixelsResolution width x height determines the number of pixels

�� 300 x 200 means each row (or scan line) has 300 pixels 300 x 200 means each row (or scan line) has 300 pixels and there's a total of 200 rows.and there's a total of 200 rows.

�� CRTs used an electron gun to activate the various pixelsCRTs used an electron gun to activate the various pixels

�� Modern display technologies: Modern display technologies: plasmaplasma, , LCD LCD ((liquidliquid--crystalcrystaldisplaydisplay), etc. replaced ), etc. replaced CRT displays in (most) applicationsCRT displays in (most) applications


CRT monitor basics (historically)CRT monitor basics (historically)

�� Electron gun draws one scan Electron gun draws one scan line at a timeline at a time

�� When it gets to bottomWhen it gets to bottom--right right

Basic CRT drawingBasic CRT drawing

�� When it gets to bottomWhen it gets to bottom--right right corner, it takes some time to get corner, it takes some time to get the aim back to top left (vertical the aim back to top left (vertical blank interval or VBLANK)blank interval or VBLANK)

�� a VSYNC a VSYNC signalsignalreports that reports that the drawing is completethe drawing is complete

�� LCD displays require the timing LCD displays require the timing signals of their own: VSYNC signals of their own: VSYNC signals of their own: VSYNC signals of their own: VSYNC (reset row pointer to top), (reset row pointer to top), HSYNC (reset column pointer HSYNC (reset column pointer to edge), LCDCLK (LCD clock to edge), LCDCLK (LCD clock to control display refresh rate) to control display refresh rate)


Color bufferColor buffer

�� A buffer is memory that A buffer is memory that stores the color data for all stores the color data for all the pixels on the screenthe pixels on the screen

Screen tearing caused by updating videodata while drawingthe pixels on the screenthe pixels on the screen

�� With only a single color With only a single color buffer, writing to it while buffer, writing to it while still displaying the old image still displaying the old image results in screen tearingresults in screen tearing

�� Meshes up Meshes up

data while drawing

new image�� Meshes up Meshes up

info from two or more info from two or more frames in a single screen frames in a single screen drawdraw


old contents

Double buffering (as a game pattern)Double buffering (as a game pattern)�� Use two color buffers Use two color buffers AA and and BB::

�� while buffer while buffer AA is written to, display buffer is written to, display buffer BB�� then next frame, write to then next frame, write to BB and display and display AA

�� An interrupt service routine can run to modify data in the video display An interrupt service routine can run to modify data in the video display �� An interrupt service routine can run to modify data in the video display An interrupt service routine can run to modify data in the video display memory while it is not being read (by a special signal from the device)memory while it is not being read (by a special signal from the device)

�� Double BufferDouble Buffergame programming patterngame programming pattern

�� cause a series of sequential operations to appear instantaneous or cause a series of sequential operations to appear instantaneous or simultaneous simultaneous (i.e., as(i.e., asatomicatomic))

�� e.g., in game graphics, e.g., in game graphics, the scene must update smoothly and quickly, the scene must update smoothly and quickly, displaying a series of complete frames, each appearing instantlydisplaying a series of complete frames, each appearing instantlydisplaying a series of complete frames, each appearing instantlydisplaying a series of complete frames, each appearing instantly

�� Sometimes, even Sometimes, even triple buffering triple buffering (three color buffers)(three color buffers)

�� if the back buffer becomes filled while display is not ready => start if the back buffer becomes filled while display is not ready => start filling a third bufferfilling a third buffer

�� potential drawback: potential drawback: input laginput lag


What is a graphic engine?What is a graphic engine?

�� An API for realAn API for real--time 2D and 3D applications (games)time 2D and 3D applications (games)

�� often runs the "often runs the "main loopmain loop" " =>=> a a software framework software framework

�� Provides Provides development environmentdevelopment environmentfor programmersfor programmers�� Provides Provides development environmentdevelopment environmentfor programmersfor programmers

�� may target multiple platforms, e.g., PC, consoles, may target multiple platforms, e.g., PC, consoles, smartphones, tabletssmartphones, tablets

�� A graphics engine provides a level of abstractionA graphics engine provides a level of abstraction

�� wraps wraps DirectX, OpenGLDirectX, OpenGLor or VulkanVulkaninside the provided APIinside the provided API

API calls may still closely mirror API calls may still closely mirror the lowthe low--level C level C APIAPI�� API calls may still closely mirror API calls may still closely mirror the lowthe low--level C level C APIAPI

�� shader programsshader programsare written and run on the GPUare written and run on the GPU


""Getting started is easier..Getting started is easier..""�� Writing graphic applications Writing graphic applications can be hard to get startedcan be hard to get started

�� create create your "your "graphics devicegraphics device""

�� lots lots of lowof low--level level initialization initialization and setand set--up up codecode

�� handle input and handle input and other events (in a platformother events (in a platform--independent way)independent way)�� handle input and handle input and other events (in a platformother events (in a platform--independent way)independent way)

�� load/unload load/unload various file formats various file formats of graphic/audio resourcesof graphic/audio resources

�� A graphics engine A graphics engine tries to tries to hide (most of) the complexityhide (most of) the complexity�� the framework provides already running "empty" the framework provides already running "empty" applicationapplication

�� Ideally, tools are integrated (XNA/MonoGame, Unreal, Unity)Ideally, tools are integrated (XNA/MonoGame, Unreal, Unity)

�� games are written and debugged within an IDE or buildergames are written and debugged within an IDE or builder�� games are written and debugged within an IDE or buildergames are written and debugged within an IDE or builder

�� separate project kinds/builds for consoles, smartphonesseparate project kinds/builds for consoles, smartphones

�� content pipeline content pipeline modifies assets (textures, 3D models) to a form more modifies assets (textures, 3D models) to a form more suitable for the game program (e.g., part of suitable for the game program (e.g., part of importimportor or build phasebuild phase))

�� plusplusaudio tools to author, organize, and play audio assets, etcaudio tools to author, organize, and play audio assets, etc


Making a 2D game entity ("Making a 2D game entity ("spritesprite")")

�� A"sprite" is often used to mean an element of a 2D A"sprite" is often used to mean an element of a 2D gamegame

�� usually usually both a texture and a position on the both a texture and a position on the screen (state)screen (state)

�� the texture is often shared but the position changesthe texture is often shared but the position changes

�� possibly using "possibly using "speedspeed" variable (say, " variable (say, new new vectorvector2 (3, 3)2 (3, 3)))�� possibly using "possibly using "speedspeed" variable (say, " variable (say, new new vectorvector2 (3, 3)2 (3, 3)))

�� The position of a 2D object on the screen is given as coordinates The position of a 2D object on the screen is given as coordinates based on screen pixelsbased on screen pixels

�� some appropriate window size (e.g., 800 some appropriate window size (e.g., 800 ** 480 pixels)480 pixels)

�� these define the range for display coordinatesthese define the range for display coordinates

�� Specify Specify source rectanglesource rectangle(from a (from a sprite sheetsprite sheet) if want to draw ) if want to draw �� Specify Specify source rectanglesource rectangle(from a (from a sprite sheetsprite sheet) if want to draw ) if want to draw only a part of an imageonly a part of an image

�� Scale an image (uniform, nonScale an image (uniform, non--uniform along uniform along xxand and y, rectangley, rectangle))

�� Sprite depth Sprite depth ((floatfloat) between 0 (= front) and 1 (= back)) between 0 (= front) and 1 (= back)

�� We can draw things off the screen but then nothing is seen We can draw things off the screen but then nothing is seen 14143.3.20173.3.2017 Juha Vihavainen / University of HelsinkiJuha Vihavainen / University of Helsinki

Implementation of spritesImplementation of sprites�� Sprites are 2D visual objects that representSprites are 2D visual objects that represent

�� game characters game characters

�� other dynamic objectsother dynamic objects

for simple games, backgrounds, toofor simple games, backgrounds, too�� for simple games, backgrounds, toofor simple games, backgrounds, too

�� The corresponding image format can vary depending on the platform The corresponding image format can vary depending on the platform (PNG, TGA, and PVR are potential options)(PNG, TGA, and PVR are potential options)

class Spriteclass Sprite

ImageFile image; ImageFile image; // // ((currentcurrent) ) texture texture int drawOrder;int drawOrder; // // using an using an intint value, herevalue, hereint drawOrder;int drawOrder; // // using an using an intint value, herevalue, hereint x, y;int x, y; // // specify location specify location (e.g., (e.g., top left cornertop left corner))

function Draw ( ) function Draw ( ) // draw the // draw the imageimageat the correct at the correct ((xx,, yy) . .) . .


Painter's algorithmPainter's algorithm

�� Sprites are drawn from back to front: nearest images are placed at the top Sprites are drawn from back to front: nearest images are placed at the top so that background images become hidden (more or less)so that background images become hidden (more or less)

PainterPainter’’s algorithm applied to a 2D s algorithm applied to a 2D space scenespace scene


From 2D to 3DFrom 2D to 3D

�� Definitely Definitely more more complex than 2D complex than 2D graphicsgraphics

�� 2D is 2D is �� 2D is 2D is

�� like painting on a canvas: placement of sprites on a 2D screenlike painting on a canvas: placement of sprites on a 2D screen

�� 3D simulates3D simulates

�� moremore--oror--less "solid" game entities located in a 3D worldless "solid" game entities located in a 3D world

�� recording them with a freelyrecording them with a freely--moving video cameramoving video camera


LeftLeft-- vs. vs. rightright--handedhandedcoordinate systemscoordinate systems

left-handed right-handed

�� itemitem


DirectXUnity

OpenGLXNA/MonoGame

Unreal(z-up world )

""3D scene3D scene" in computer graphics" in computer graphics

�� A 3D A 3D scenesceneisis

�� "geometry" (triangles, lines, points, and more)"geometry" (triangles, lines, points, and more)

�� materialmaterialsurface properties of geometry surface properties of geometry �� materialmaterialsurface properties of geometry surface properties of geometry

�� howhowto draw it (a to draw it (a shadershaderprogram)program)

�� texturestextures(images to glue onto the geometry)(images to glue onto the geometry)

�� light sources and their propertieslight sources and their properties

�� A triangle consists of 3 verticesA triangle consists of 3 vertices

�� a vertex is 3D position, and may include normals and morea vertex is 3D position, and may include normals and more

�� To take a picture, we need a To take a picture, we need a cameracamera–– a a virtualvirtual oneone

�� decides what will end up in the final imagedecides what will end up in the final image


"Virtual" (synthetic) 3D camera"Virtual" (synthetic) 3D camera

�� Defined by Defined by positionposition, , directiondirectionvector, vector, upupvector, vector, field of view field of view ((angleangle), ), nearnearand and farfar planeplane

�� Used by Used by OpenGLOpenGL, , DirectXDirectX, ray tracing image processing, etc., ray tracing image processing, etc.

�� Create the image of those shapes located inside the frustumCreate the image of those shapes located inside the frustum

near

far world 2D projection


point

dirfov(angle)

3D camera (cont.)3D camera (cont.)

�� Can have objects in a scene that aren’t (currently) visibleCan have objects in a scene that aren’t (currently) visible

�� objects can be rotated around three axesobjects can be rotated around three axesa freely moving camera can also be rotated around all 3 axesa freely moving camera can also be rotated around all 3 axes�� a freely moving camera can also be rotated around all 3 axesa freely moving camera can also be rotated around all 3 axes

�� the camera determines what shows up on screenthe camera determines what shows up on screen�� of course, if the camera is pointing in the opposite direction of course, if the camera is pointing in the opposite direction

from an object, we won't see itfrom an object, we won't see it


Models in 3D gamesModels in 3D games

�� 3D games consist of large numbers of visible objects3D games consist of large numbers of visible objects

�� In practice, these objects are In practice, these objects are notnot created by game code by defining created by game code by defining each individual primitive (triangle)each individual primitive (triangle)each individual primitive (triangle)each individual primitive (triangle)

�� Instead, artists create objects for a game world, game designers Instead, artists create objects for a game world, game designers focus on behavior of those objects, and programmers mostly focus on behavior of those objects, and programmers mostly concentrate on support systemsconcentrate on support systems

�� Special 3D modeling tools are used, e.g.,Special 3D modeling tools are used, e.g.,

�� BlenderBlender, 3ds , 3ds MaxMax, , MayaMaya. .. .�� BlenderBlender, 3ds , 3ds MaxMax, , MayaMaya. .. .

�� modeling tools permit separate editing of modelsmodeling tools permit separate editing of models

�� plugplug--in tools allow models to be saved in a number of different in tools allow models to be saved in a number of different file formatsfile formats


A sphere made of trianglesA sphere made of triangles


Drawing the triangles in GPUDrawing the triangles in GPU�� In 3D graphics engines, all rendering is ultimately handled by the GPU In 3D graphics engines, all rendering is ultimately handled by the GPU

and and shadershaderprogramsprograms

�� shadersshadersare small specialare small special--purpose programs that execute inside the purpose programs that execute inside the GPU and define how the data received from a game program is GPU and define how the data received from a game program is processed in the programmable stages of the rendering pipelineprocessed in the programmable stages of the rendering pipelineprocessed in the programmable stages of the rendering pipelineprocessed in the programmable stages of the rendering pipeline

�� shader programs are defined using a shader programs are defined using a high level shader languagehigh level shader language(e.g., Microsoft's (e.g., Microsoft's HLSLHLSL, OpenGL , OpenGL GLSLGLSL))

�� Vulkan supports shaders in a bytecode format, called Vulkan supports shaders in a bytecode format, called SPIRSPIR--V, as V, as opposed to humanopposed to human--readable syntax like GLSL readable syntax like GLSL and HLSLand HLSL

�� provide provide wide range of visual effectswide range of visual effects�� not not part of this part of this coursecourse�� not not part of this part of this coursecourse

�� see (Gregory) for explanations andsee (Gregory) for explanations andexamples of GPU programmingexamples of GPU programming

�� Luckily, the game IDE / tools oftenLuckily, the game IDE / tools oftenprovide many provide many predefinedpredefineddefault shadersdefault shaders


Rendering dataRendering data

�� Shared data between the CPU and the GPUShared data between the CPU and the GPU

�� 3D model data: streams of vertices specifying triangles..3D model data: streams of vertices specifying triangles..

�� matrices matrices that represent object transforms and skeletal that represent object transforms and skeletal animation animation

�� lighting parameters, and lighting parameters, and

�� other kinds of "shader constants" used for computationsother kinds of "shader constants" used for computations

�� Data (almost exclusively) produced and managed by the GPU Data (almost exclusively) produced and managed by the GPU

�� vertex and index buffers, frame buffersvertex and index buffers, frame buffers�� vertex and index buffers, frame buffersvertex and index buffers, frame buffers


Drawing shapes as sets of Drawing shapes as sets of trianglestriangles

�� 3D shapes (in games) are (ultimately) composed of triangles3D shapes (in games) are (ultimately) composed of triangles

�� A A triangletrianglehas 3 points, one for each cornerhas 3 points, one for each cornerA A triangletrianglehas 3 points, one for each cornerhas 3 points, one for each corner

�� is the minimum number of is the minimum number of vertices vertices (points) to define a (points) to define a planeplane

�� these vertices need to be sent to these vertices need to be sent to GPUGPU for processingfor processing

vert01 = new vert01 = new vertexPositionColorvertexPositionColor (new (new vectorvector3 (0, 1, 0), 3 (0, 1, 0), Color.Blue);Color.Blue);

�� represents the represents the locationlocationof of oneonevertex in a trianglevertex in a triangle

many alternativesmany alternatives

�� represents the represents the locationlocationof of oneonevertex in a trianglevertex in a triangle

�� defines the defines the colorcolor of a vertex (to be interpolated over)of a vertex (to be interpolated over)

�� A vertex structure may include other relevant info processed by A vertex structure may include other relevant info processed by shader programs..shader programs..


Typical Typical vertexvertexstructures (used by graphics API)structures (used by graphics API)11. . VertexPositionColorVertexPositionColor

�� xx, , yy, , zz plus a plus a colorcolor for each vertexfor each vertex

�� useful for basic shapes, basic colorsuseful for basic shapes, basic colors

2. 2. VertexPositionTextureVertexPositionTexture2. 2. VertexPositionTextureVertexPositionTexture

�� x, y, x, y, zz plus relative plus relative uu, , vvcoordinates into a texture (bitmap) coordinates into a texture (bitmap)

�� can overlay a bitmap texture onto a shapecan overlay a bitmap texture onto a shape

3. 3. VertexPositionNormalTextureVertexPositionNormalTexture

�� xx, , yy, , zz plus plus uu, , vv, coordinates plus a , coordinates plus a normalnormalvectorvector

�� normal permits calculating lighting effectsnormal permits calculating lighting effects

4. 4. VertexPositionColorTextureVertexPositionColorTexture4. 4. VertexPositionColorTextureVertexPositionColorTexture

�� xx, , yy, , zz plus plus uu, , vvcoordinates plus coordinates plus colorcolor

�� color information changes color of bitmap texturecolor information changes color of bitmap texture

�� can potentially reuse same texture in different contextscan potentially reuse same texture in different contexts


What are What are texture coordinatestexture coordinates??

�� vertexPositionTexture vertexPositionTexture takes two parameterstakes two parameters

�� vector3 vector3 represents the position (represents the position (xx,,yy,,zz) of a vertex) of a vertex

�� vector2vector2represents relative texture coordinates represents relative texture coordinates uu and and vv�� vector2vector2represents relative texture coordinates represents relative texture coordinates uu and and vvtelling which image point (telling which image point (texeltexel) hits the vertex) hits the vertex

�� u u is a horizontal coordinate in the range 0.0 . . 1.0is a horizontal coordinate in the range 0.0 . . 1.0

�� vv is a vertical coordinate in the range 0.0 . . 1.0is a vertical coordinate in the range 0.0 . . 1.0

�� toptop--left corner = { 0, 0 }left corner = { 0, 0 }

�� a point exactly in the middle of a texture = { 0.5, 0.5 }a point exactly in the middle of a texture = { 0.5, 0.5 }�� a point exactly in the middle of a texture = { 0.5, 0.5 }a point exactly in the middle of a texture = { 0.5, 0.5 }

�� bottombottom--right corner = { 1, 1 }right corner = { 1, 1 }

�� Note that these relative (Note that these relative (uu, , vv) coordinates are now independent of ) coordinates are now independent of the size of the image (good for potential scaling or other changes)the size of the image (good for potential scaling or other changes)


What is a What is a virtual game cameravirtual game camera�� In game graphics, a In game graphics, a virtual camera virtual camera is implemented by software to is implemented by software to

simulate the way a real camera (or an eye) would work in realsimulate the way a real camera (or an eye) would work in real--world situationsworld situations

�� In the game/engine, the virtual camera is made up from In the game/engine, the virtual camera is made up from �� In the game/engine, the virtual camera is made up from In the game/engine, the virtual camera is made up from mathematical calculations that determine how objects will be mathematical calculations that determine how objects will be finally rendered based on the location and angle of the virtual finally rendered based on the location and angle of the virtual camera in the programcamera in the program

�� 3D is like a video camera 3D is like a video camera recording recording -- what camera sees what camera sees (inside its(inside itsviewingviewingfrustumfrustum) is ) is


(inside its(inside itsviewingviewingfrustumfrustum) is ) is shown on screenshown on screen

Implementing Implementing 3D graphics system (GPU)3D graphics system (GPU)�� VertexVertexand and indexindex buffersbuffersare used to stream lots of 3D points are used to stream lots of 3D points

(vertices) to the graphics device for rendering(vertices) to the graphics device for rendering

�� indicesindicesare just indirect pointers to are just indirect pointers to sharedsharedvertices (to save space vertices (to save space and bandwidth)and bandwidth)

�� Separately must tell Separately must tell howhowconsequent vertices are to be interpreted as consequent vertices are to be interpreted as primitive shapes (e.g., triangles forming a 3D object): whether primitive shapes (e.g., triangles forming a 3D object): whether adjacent adjacent triangles use triangles use sharedsharedvertices or not (vertices or not (listslists, , stripsstrips, , fansfans))

�� ShadersShadersand related data structures (registers, memory) in GPUand related data structures (registers, memory) in GPU

�� processing of vertices, primitives, and pixelsprocessing of vertices, primitives, and pixels

�� nowadays, always need to use shaders (default/custom)nowadays, always need to use shaders (default/custom)�� nowadays, always need to use shaders (default/custom)nowadays, always need to use shaders (default/custom)

�� apply matrix transformations (apply matrix transformations (TransformTransformcomponent) to verticescomponent) to vertices

�� translationtranslation(i.e., movement), (i.e., movement), rotationrotation, and , and scalescale

�� apply colors and textures to trianglesapply colors and textures to triangles

�� specify lighting, its location (plus other properties)specify lighting, its location (plus other properties)


Painter's algorithm (again)Painter's algorithm (again)

�� Painter's algorithm doesn't work well for 3D scenesPainter's algorithm doesn't work well for 3D scenes

�� We would have to sort every triangle/pixel back to frontWe would have to sort every triangle/pixel back to front

worse, the camera is dynamic, so we'd have to reworse, the camera is dynamic, so we'd have to re--sort!sort!�� worse, the camera is dynamic, so we'd have to reworse, the camera is dynamic, so we'd have to re--sort!sort!

�� In 3D, there is a possibility In 3D, there is a possibility that no one triangle is the that no one triangle is the furthest backfurthest back

�� The painter's algorithm has a lot of overdraw The painter's algorithm has a lot of overdraw

OverlappingOverlapping--trianglestriangles--failure failure casecase

�� The painter's algorithm has a lot of overdraw The painter's algorithm has a lot of overdraw

�� unnecessary redrawing the same pixel multiple times per unnecessary redrawing the same pixel multiple times per frame frame =>=> inefficientinefficient


Solution: ZSolution: Z--bufferingbuffering

�� Which elements (parts) of a rendered Which elements (parts) of a rendered scene are visible, and which are hiddenscene are visible, and which are hidden

�� Create a new screenCreate a new screen--sized buffer called sized buffer called the zthe z--buffer (buffer (depth bufferingdepth buffering), to ), to the zthe z--buffer (buffer (depth bufferingdepth buffering), to ), to determine determine what is what is closer to the cameracloser to the camera

�� It stores the It stores the depthdepth((zzcoordinate) of coordinate) of every pixel that is drawn (or actually, of every pixel that is drawn (or actually, of the the lastlastone)one)

�� As we draw pixels in the scene, only As we draw pixels in the scene, only draw it if the depth stored in the zdraw it if the depth stored in the z--buffer buffer draw it if the depth stored in the zdraw it if the depth stored in the z--buffer buffer is greater than the depth of the current is greater than the depth of the current pixel (and so located further out in the pixel (and so located further out in the scene) => pixels can be drawn in any scene) => pixels can be drawn in any appropriate appropriate order, simplifying the order, simplifying the overall processing of trianglesoverall processing of triangles


A sample scene and its corresponding z-buffer

(Wikipedia)

On the On the rendering pipelinerendering pipeline

�� Note: "Note: "rasterizationrasterization" " -- a scene with 3D models (shapes) converted a scene with 3D models (shapes) converted into a raster image (consisting of pixels or dots)into a raster image (consisting of pixels or dots)

�� need only to represent the surface/outer boundary of an object need only to represent the surface/outer boundary of an object

the most popular technique for producing realthe most popular technique for producing real--time 3D computer time 3D computer �� the most popular technique for producing realthe most popular technique for producing real--time 3D computer time 3D computer graphics; nearly all visual models used in game graphicsgraphics; nearly all visual models used in game graphics

�� colors and textures can be appliedcolors and textures can be applied

�� A pipeline is a sequence of stages operating in a fixed order (and A pipeline is a sequence of stages operating in a fixed order (and potentially potentially in parallelin parallel), e.g.), e.g.

�� vertex processingvertex processing, , rasterizationrasterization, , pixel pixel ((fragmentfragment)) processing, ..processing, ..vertex processingvertex processing, , rasterizationrasterization, , pixel pixel ((fragmentfragment)) processing, ..processing, ..

�� each stage receives its input from the prior stage and sends its each stage receives its input from the prior stage and sends its output to the subsequent stageoutput to the subsequent stage

�� A 3D application (game) sends to the GPU a sequence of vertices A 3D application (game) sends to the GPU a sequence of vertices batched into geometric "batched into geometric "primitivesprimitives", typically triangles (in buffers)", typically triangles (in buffers)

Juha Vihavainen / University of HelsinkiJuha Vihavainen / University of Helsinki 33333.3.20173.3.2017

[Gregory, 10.2 The Rendering Pipeline, p. 489][Gregory, 10.2 The Rendering Pipeline, p. 489]Here, the Here, the pipelinepipelineincludes tools used to create scenes and game entitiesincludes tools used to create scenes and game entities

1.1. Tools stage Tools stage (offline): defining geometry and surface properties (offline): defining geometry and surface properties (materials), animations, audio resources, etc.(materials), animations, audio resources, etc.

2.2. Asset conditioningAsset conditioning(offline): geometry and materials are processed (offline): geometry and materials are processed 2.2. Asset conditioningAsset conditioning(offline): geometry and materials are processed (offline): geometry and materials are processed by asset conditioning pipeline (ACP) into an engineby asset conditioning pipeline (ACP) into an engine--ready formatready format

3.3. Application stageApplication stage(CPU): potentially visible mesh instances are (CPU): potentially visible mesh instances are identified and submitted to the graphics hardware along with their identified and submitted to the graphics hardware along with their materials for renderingmaterials for rendering

4.4. Geometry processing Geometry processing (GPU): vertices are transformed and lit and (GPU): vertices are transformed and lit and projected into (homogeneous) clip space; triangles are processed by projected into (homogeneous) clip space; triangles are processed by projected into (homogeneous) clip space; triangles are processed by projected into (homogeneous) clip space; triangles are processed by the optional geometry shader and then clipped to the frustum (lines the optional geometry shader and then clipped to the frustum (lines and surfaces outside the view volume are removed)and surfaces outside the view volume are removed)

5.5. Rasterization stageRasterization stage(GPU): triangles are converted into fragments (GPU): triangles are converted into fragments that are shaded, passed through various tests (z test, alpha test, that are shaded, passed through various tests (z test, alpha test, stencil test, etc.) and finally blended into the frame buffer stencil test, etc.) and finally blended into the frame buffer


The rendering pipeline (simplified overview)The rendering pipeline (simplified overview)

(1) (offline) (2) (offlline)

ACP = Asset

(3) (CPU) (4) (GPU)

ACP = Asset Conditioning Pipeline

fragment = potential pixel (OpenGL)


(5) (GPU)

<= <= For explanations, see the For explanations, see the previousprevious slide slide [Greg, Fig. 10.41, p. 491]

Rendering pipeline Rendering pipeline transforms datatransforms data

�� The The tools and asset conditioning tools and asset conditioning stages deal with meshes and stages deal with meshes and materials materials

�� The The application application stage deals in terms of mesh instances and stage deals in terms of mesh instances and submeshes, each of which is associated with a single material submeshes, each of which is associated with a single material

�� During the During the geometry geometry stage, each submesh is broken down into stage, each submesh is broken down into individual vertices, which are processed largely in parallel;individual vertices, which are processed largely in parallel;at its conclusion, the triangles are reconstructed from the fully at its conclusion, the triangles are reconstructed from the fully transformed and shaded vertices transformed and shaded vertices

�� In the In the rasterization rasterization stage, each triangle is broken into fragments, stage, each triangle is broken into fragments, and these fragments are either discarded, or they are eventually and these fragments are either discarded, or they are eventually written into the frame buffer as colors written into the frame buffer as colors


ZZ--buffering in buffering in GPU GPU hardwarehardware

�� WWhen using zhen using z--buffering, we buffering, we don'tdon't draw triangles draw triangles backback--toto--frontfront, or , or many draws will become waisted (when overridden)many draws will become waisted (when overridden)

�� TTransparentransparentobjects, such as water, need some special arrangementsobjects, such as water, need some special arrangements

ff irstirst draw all nondraw all non--transparent objects, and transparent objects, and �� ff irstirst draw all nondraw all non--transparent objects, and transparent objects, and

�� then then transparent ones (transparent ones (partlypartly disabling zdisabling z--buffering buffering -- to blend)to blend)

�� otherwise transparent objects would totally hide objects otherwise transparent objects would totally hide objects behind them (background opaque ones would not be drawnbehind them (background opaque ones would not be drawn))

�� OcclusionOcclusion cullingculling tehniques tehniques eliminate, at application stage, entire eliminate, at application stage, entire scene objects that cannot be visible on a current scene objects that cannot be visible on a current frame frame


scene objects that cannot be visible on a current scene objects that cannot be visible on a current frame frame

�� e.g., a e.g., a tree totally occluded by a tree totally occluded by a building building

�� uses uses spatial partitioning spatial partitioning for for efficiencyefficiency

�� An algorithm in pseudoAn algorithm in pseudo--language, see next slidelanguage, see next slide

// // clear clear the frame and depth buffersthe frame and depth buffers

fillColorBuffer fillColorBuffer (backgroundColor);(backgroundColor);fillDepthBuffer (infinity);fillDepthBuffer (infinity);// // outer loop iterates over all the primitives outer loop iterates over all the primitives ((usually trianglesusually triangles))

for each geometric primitive dofor each geometric primitive do// // first rasterize the primitive . .first rasterize the primitive . .

(3D Math Primer for Graphics andGame Development, 2011), p. 346

// // first rasterize the primitive . .first rasterize the primitive . .

for each pixel x , y in the projection of the primitive do for each pixel x , y in the projection of the primitive do // // test the depth buffer , to see if a closer pixel has been written test the depth buffer , to see if a closer pixel has been written

float primDepth = getDepthOfPrimitiveAtPixel (x , y) ;float primDepth = getDepthOfPrimitiveAtPixel (x , y) ;// // pixel of this primitive is obscured , discard itpixel of this primitive is obscured , discard it

if primDepth > readDepthBuffer (x, y) continue;if primDepth > readDepthBuffer (x, y) continue;// // determine primitive color at this pixeldetermine primitive color at this pixel

Color c = getColorOfPrimitiveAtPixel (x, y)Color c = getColorOfPrimitiveAtPixel (x, y)// // update the color and depth buffersupdate the color and depth buffers

writeColorBuffer writeColorBuffer (x, y, c)(x, y, c)writeDepthBuffer (x, y, primDepth)writeDepthBuffer (x, y, primDepth)

04 Graphics for games 1 - University of Helsinki€¦ · 04. Graphics for games Juha Vihavainen University of Helsinki Outline Creating and drawing game entities from 2D images (

Documents