Manifold Bootstrapping for SVBRDF Capture Yue Dong, Jiaping Wang, Xin Tong, John Snyder, Moshe Ben-Ezra, Yanxiang Lan, Baining Guo Tsinghua University.

Manifold Bootstrapping for SVBRDF Capture

Yue Dong, Jiaping Wang, Xin Tong, John Snyder, Moshe Ben-Ezra, Yanxiang Lan, Baining Guo

Tsinghua University Microsoft Research Asia Microsoft Research

High-Quality SVBRDF Acquisition

high spatial variation high angular variation

fast and simple too!

Related Work

• brute force (6D) measurement (gonioreflectometer)[Dana et al. 1999, McAllister et al. 2002, Lawrence et al. 2006]

• slow• expensive, specialized rig

Related Work

• single-pass fitting methods[Lensch et al. 2003, Goldman et al. 2005, Zickler et al. 2005]

• measures large dataset• fits limited models

(parametric/isotropic)

Related Work

• two-pass methods– linearly combine two representatives based on diffuse

color[Debevec et al. 2003]

Related Work

• two-pass methods– linearly combine two representatives based on diffuse

color[Debevec et al. 2003]

Related Work

• two-pass methods– linearly combine two representatives based on diffuse

color[Debevec et al. 2003]

– use existing BRDF database of representatives: non-specialized and isotropic[Matusik et al. 2003b; Weyrich 2006]

Observation

• BRDF spatial variation is complex:– tangent/normal/local frame rotates

– specularity/anisotropy varies

– specular lobe’s falloff and cross-section changes

• forms low-dimensional manifold over given target.

• manifold isn’t globally linear [Matusik et al. 2003a]

• manifold is locally linear.

SVBRDF Manifold

globally non-linear

locally linear

Local vs. Global Interpolation

local interpolation

global interpolation

Local vs. Global Interpolation

SVBRDF Manifold Bootstrapping

SVBRDF ManifoldRepresentativeSpace

SVBRDF Manifold Bootstrapping

Representative Measurements

RepresentativeSpace

SVBRDF Manifold Bootstrapping

Representative Measurements

RepresentativeSpace

Material Sample

SVBRDF Manifold Bootstrapping

Representative Measurements

Key Measurements

RepresentativeSpace

Material Sample

SVBRDF Manifold Bootstrapping

Representative Measurements

RepresentativeSpace

Key SpaceKey Measurements

every pixelevery pixel

SVBRDF Manifold Bootstrapping

Representative Measurements

RepresentativeSpace

Local EmbeddingIn Key SpaceKey Measurements

Key Space

x

SVBRDF Manifold Bootstrapping

Representative Measurements

RepresentativeSpace

Key MeasurementsLocal Embedding

In Key Space

x

SVBRDF Manifold Bootstrapping

Representative Measurements

RepresentativeSpace

Key Measurements

Local Embedding of x In Representative Space

Local EmbeddingIn Key Space

x

SVBRDF Manifold Bootstrapping

Representative Measurements

RepresentativeSpace

ReconstructedBRDF of x

Key MeasurementsLocal Embedding

In Key Space

Local Embedding of x In Representative Space

x

Results

Real Material Sample

Outline

• Data Acquisition

• SVBRDF Reconstruction

• Validation

Representative BRDFs

• portable BRDF scanner– 6 LED light directions, 320x240 view directions– data amplification by microfacet model– 0.1s per BRDF

Key Measurements

• fixed camera• background environmental lighting

+ moving area source

Timing

• representative BRDFs and key measurements– 10-15 minutes

• data processing– less than 5 minutes

Outline

• Data Acquisition

• SVBRDF Reconstruction

• Validation

SVBRDF Reconstruction

Representative BRDFs

Representative Local Interpolation

Representative BRDFs

x

Material Sample

= w1 + w2 + w3

BRDF of x

?

Representative Local Interpolation

• choose which representatives to interpolate from

• solve for weights wi

= w1 + w2 + w3w1 w2 w3

x

Material Sample Representative BRDFs

BRDF of x

?

Representative BRDFsMaterial Sample

Key Measurement

Environment Lighting

Projected Keys of Representative BRDFsKey Measurements

Key Measurement

Projected Keys of Representative BRDFsKey Measurements

Key Measurements

Key Local Interpolation

Projected Keys of Representative BRDFs

x

Key of x

nearest neighbor in key space

Key Local Interpolation

Key of x

= w1 + w2 + w3

• solve for weights: LLE [Roweis & Saul 2000]

Key Measurements

x

where

BRDF Reconstruction

Key of x

= w1 + w2 + w3Local Embedding

in Key Space

NeighborhoodNeighborhood

BRDF Reconstruction

= w1 + w2 + w3

BRDF of x

Key of x

= w1 + w2 + w3Local Embedding

in Key Space

weightsweights

Outline

• Data Acquisition

• SVBRDF Reconstruction

• Validation

• Projection depend on the environmental lighting conditions• preserve distances ⇒ preserve BRDF manifold structure

Key Space vs. Representative Space

• Projection depend on the environmental lighting conditions• preserve distances ⇒ preserve BRDF manifold structure

global distances⇒ preserve neighborhoods

Key Space vs. Representative Space

local distances⇒ preserve weights

Distance Preservation

• preservation evaluation

Distance Preservation

• preservation evaluation

• # of lighting conditions

Distance Preservation

• preservation evaluation

• # of lighting conditions

• criterion: global: τg > 0.9

local: τl > 0.85

Results

Real Material Sample

Extension to local frame variations

• Normal variations • Tangent rotations

Representative Enlargement

…

enlarged BRDFs over tangent rotation

…enlarged BRDFs over normal rotation

Results

Real Material Sample

Results

Real Material Sample

Conclusion

• Manifold bootstrapping captures high-resolution SVBRDF– assumes BRDF forms low-dimensional manifold– decomposes acquisition into two phases– makes sparse measurement in both

• phase one (representatives) = sparse spatial, dense angular• phase two (keys) = sparse angular, dense spatial

– simplifies and accelerates the capture process

Conclusion

• Manifold bootstrapping captures high-resolution SVBRDF– assumes BRDF forms low-dimensional manifold– decomposes acquisition into two phases– makes sparse measurement in both

• phase one (representatives) = sparse spatial, dense angular• phase two (keys) = sparse angular, dense spatial

– simplifies and accelerates the capture process

Acknowledgements

• Paul Debevec for HDR images• Steve Lin for video narration • Anonymous reviewers for helpful comments

THANKS

THANKS

Uniform Measurement Scaling

• Representative Projection

Uniform Measurement Scaling

• Representative Projection

Future Work

• improving the hand-held BRDF scanner• handling self-shadowing and masking effects

Implementation

• capturing parameters: