Image Synthesis

computer graphics & visualization

Image Synthesis

Point-Based Computer Graphics


Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group

Why Points?• huge geometry complexity of current CG

models• overhead introduced by connectivity of

polygonal meshes• acquisition devices generate point samples

“digital 3D photography”• points complement triangles



Polynomials• Rigorous mathematical concept• Robust evaluation of geometric entities• Shape control for smooth shapes

• Require proper parameterization• Discontinuity modeling• Topological flexibility



Polynomilas Triangles• Piecewise linear approximations• Irregular sampling of the surface• No parameterization needed (geometry only)



Triangles• Simple and efficient representation• Hardware pipelines support triangles• Advanced geometric processing• The widely accepted queen of graphics primitives

• Sophisticated modeling is difficult• (Local) parameterizations still needed• Complex LOD management• Compression and streaming is highly non-trivial



Triangles Points• Piecewise linear functions to Delta distributions• Discrete samples of geometry• No connectivity or topology – most simple• Store all attributes per surface sample



Points• geometry complexity of current CG models• connectivity overhead of polygonal meshes• acquisition devices generate point samples• points complement triangles

• holes• compression



Acquisition

Rendering

Representation

Processing & Editing

Point Based Graphics

Taxonomy



How can we capture reality?



Acquisition• Contact digitizers

– intensive manual labor• Passive methods

– require texture, Lambertian BRDF• Active light imaging systems

– restrict types of materials• in general fuzzy, transparent,

and refractive objects are difficult



First Method, Laser Range Scanner



Basic Idea

Laser

Detector

Laser

Detector



Computing the Distance

Laser

DetectorH

LOd

d’a’

a

''

''

adad

da

da



Scattering Issues

How optically cooperative is marble?



Image based Acquisition



Image based AcquisitionAcquisition Stage 2



Image based AcquisitionAcquisition Stage 3



IBA - Process



Visual Hull



Visual Hull

• the quality of the visual hull geometry is a function of the number of viewpoints / silhouettes

• the method is unable to capture all concavities image based lighting



Point Based RenderingSurfels (surface element)



Extended Surfels



Rendering Pipeline



Uniform Reconstruction• For uniform samples, use signal processing theory• Reconstruction by convolution with low-pass

(reconstruction) filter• Exact reconstruction of band-limited signals

using ideal low-pass filters



Non-Uniform Reconstruction• Signal processing theory not applicable for

nonuniform samples• Local weighted average filtering

– Normalized sum of local reconstruction kernels



Reconstruction 1D in 2D



Reconstruction 2D in 3D



Algorithm

for each sample point {shade surface sample;splat = projected reconstruction kernel;rasterize and accumulate splat;

}for each output pixel {normalize;

}



Resultswithout Normalization with Normalization



Visibilityε-z-buffering



ImplementationUse a three pass algorithm:1. Draw depth image with a small depth offset ε

away from the viewpointPerform regular z-buffering (depth tests and updates), no color updates



Second Pass• Draw colored splats with additive blending

enabled• Perform depth tests, but no updates• Accumulate– Weighted colors of visible splats in the color

channels– Weights of visible footprint functions in the alpha

channel



Third Pass• Normalization of the color channels by dividing

through the alpha channel• Implemented by– render to texture– drawing a screen filling quad with this texture– performing the normalization in the pixel shader



Efficient Data Structures

DuoDecimA Structure for Point ScanCompression and Rendering



a.d. 1500• created beautiful statues

… but missed to make them portable

• David: 434 cm• Atlas: 208 cm• Barbuto: 248 cm• …

Florence, Galleria dell'Accademia



a.d. 1999• Marc Levoy et al. (Stanford University) did a

great job scanning the statues… but still “missed” to make them “portable”

• David:1.1 GB• Atlas: 10 GB• …

All in all 32 gigabytes raw data !!!Jens Krüger - Computer Graphics and Visualization Group, TU-München

Image courtesy of Marc Levoy



Today…… we present novel algorithms to make the statues of Michelangelo „portable“ and „viewable“ on PCs.



The NumbersThe Atlas Statue Scan

Size of raw scans: approx 600 mio. pointsReconstruction: at 1/4 mmSize of reconstruction: approx 250 mio. vertices,

approx 500 mio. trianglesFilesize (without normals): 9.94 GB

„Your Home PC“Size of Main Memory: 1 – 2 GBSize of Graphics Card Memory: 0.125 – 0.5 GB



230 MB vs. 10 GB

Rendering and decoding time in full resolution 4 sec.



The Key IdeaCPU• Sample the point scan into a regular grid• Divide the grid up into 2D slices• Look for connected runs within the slices• Store the starting position/normal per run• Store position/normal delta for the rest

GPU• Store the compressed runs into textures• Upload the compressed runs onto the GPU• Decode the points with normals on the fly



1. Sample the point scan into a regular gridHexagonal close sphere packing (HCP) grid



Cell Search (HCP)2D Simplification



2. Divide the grid up into 2D slices

1

sqrt(3)

2

sqrt(3)



3. Look for connected runs within the slices

lk

f gi

j

h

mn

oq

p

dbe

ca

abcdklmn

opoq

defghgij

defghgijigfedklmnopoqopoq

ghgij



3. Connected runs (cont.)



4. Store the starting position/normal per run

• Start Position two (16bit) indices per point– one (16bit) index per slice

• Start Normal one (16bit) index– one (16bit) Codebook per dataset



5. Store position/normal delta for the rest• Delta Position 6 cases 2.25 bit

• Delta Normal 5bit index– one (5bit) Codebook per dataset

• contains sin/cos of delta angles



Rendering

GPU Decoding and Rendering

• Store the compressed runs into textures• Upload the compressed runs onto the GPU• Decode the points with normals on the fly



1. Store the compressed runs into texturesStart Position/Normal Delta Position/Normal 16bit (RG)(B) 8bit (3+5)

Per Slice:•16bit height•8 Floats to the GPU



n

m

3. Decode the points with normals on the flyStart Position/Normal

n

m

Position

Normal

Render as Points via:PBO/VBO CopyCast to Vertex ArrayVertex Texture Fetch



n

m

3. Decode the points with normals on the flyStart Position/Normal

n

m

Position

Normal

Delta Position/Normal 1

n

m

n

m

Position

NormalPixelshader

Render as Points



One Image about quality…

Image Synthesis

Documents

jens krger computer

visualization groupcomputing

visualization grouphow

visualization groupwhy

point samplespoints

point samplesdigital

smooth shapes

d photographypoints