Manifold Bootstrapping for SVBRDF Capture Yue Dong, Jiaping Wang, Xin Tong, John Snyder, Moshe Ben-Ezra, Yanxiang Lan, Baining Guo Tsinghua University Microsoft Research Asia Microsoft Research
Dec 19, 2015
Manifold Bootstrapping for SVBRDF Capture
Yue Dong, Jiaping Wang, Xin Tong, John Snyder, Moshe Ben-Ezra, Yanxiang Lan, Baining Guo
Tsinghua University Microsoft Research Asia Microsoft Research
High-Quality SVBRDF Acquisition
high spatial variation high angular variation
fast and simple too!
Related Work
• brute force (6D) measurement (gonioreflectometer)[Dana et al. 1999, McAllister et al. 2002, Lawrence et al. 2006]
• slow• expensive, specialized rig
Related Work
• single-pass fitting methods[Lensch et al. 2003, Goldman et al. 2005, Zickler et al. 2005]
• measures large dataset• fits limited models
(parametric/isotropic)
Related Work
• two-pass methods– linearly combine two representatives based on diffuse
color[Debevec et al. 2003]
Related Work
• two-pass methods– linearly combine two representatives based on diffuse
color[Debevec et al. 2003]
Related Work
• two-pass methods– linearly combine two representatives based on diffuse
color[Debevec et al. 2003]
– use existing BRDF database of representatives: non-specialized and isotropic[Matusik et al. 2003b; Weyrich 2006]
Observation
• BRDF spatial variation is complex:– tangent/normal/local frame rotates
– specularity/anisotropy varies
– specular lobe’s falloff and cross-section changes
• forms low-dimensional manifold over given target.
• manifold isn’t globally linear [Matusik et al. 2003a]
• manifold is locally linear.
SVBRDF Manifold
globally non-linear
locally linear
Local vs. Global Interpolation
local interpolation
global interpolation
Local vs. Global Interpolation
SVBRDF Manifold Bootstrapping
SVBRDF ManifoldRepresentativeSpace
SVBRDF Manifold Bootstrapping
Representative Measurements
RepresentativeSpace
SVBRDF Manifold Bootstrapping
Representative Measurements
RepresentativeSpace
Material Sample
SVBRDF Manifold Bootstrapping
Representative Measurements
Key Measurements
RepresentativeSpace
Material Sample
SVBRDF Manifold Bootstrapping
Representative Measurements
RepresentativeSpace
Key SpaceKey Measurements
every pixelevery pixel
SVBRDF Manifold Bootstrapping
Representative Measurements
RepresentativeSpace
Local EmbeddingIn Key SpaceKey Measurements
Key Space
x
SVBRDF Manifold Bootstrapping
Representative Measurements
RepresentativeSpace
Key MeasurementsLocal Embedding
In Key Space
x
SVBRDF Manifold Bootstrapping
Representative Measurements
RepresentativeSpace
Key Measurements
Local Embedding of x In Representative Space
Local EmbeddingIn Key Space
x
SVBRDF Manifold Bootstrapping
Representative Measurements
RepresentativeSpace
ReconstructedBRDF of x
Key MeasurementsLocal Embedding
In Key Space
Local Embedding of x In Representative Space
x
Results
Real Material Sample
Outline
• Data Acquisition
• SVBRDF Reconstruction
• Validation
Representative BRDFs
• portable BRDF scanner– 6 LED light directions, 320x240 view directions– data amplification by microfacet model– 0.1s per BRDF
Key Measurements
• fixed camera• background environmental lighting
+ moving area source
Timing
• representative BRDFs and key measurements– 10-15 minutes
• data processing– less than 5 minutes
Outline
• Data Acquisition
• SVBRDF Reconstruction
• Validation
SVBRDF Reconstruction
Representative BRDFs
Representative Local Interpolation
Representative BRDFs
x
Material Sample
= w1 + w2 + w3
BRDF of x
?
Representative Local Interpolation
• choose which representatives to interpolate from
• solve for weights wi
= w1 + w2 + w3w1 w2 w3
x
Material Sample Representative BRDFs
BRDF of x
?
Representative BRDFsMaterial Sample
Key Measurement
Environment Lighting
Projected Keys of Representative BRDFsKey Measurements
Key Measurement
Projected Keys of Representative BRDFsKey Measurements
Key Measurements
Key Local Interpolation
Projected Keys of Representative BRDFs
x
Key of x
nearest neighbor in key space
Key Local Interpolation
Key of x
= w1 + w2 + w3
• solve for weights: LLE [Roweis & Saul 2000]
Key Measurements
x
where
BRDF Reconstruction
Key of x
= w1 + w2 + w3Local Embedding
in Key Space
NeighborhoodNeighborhood
BRDF Reconstruction
= w1 + w2 + w3
BRDF of x
Key of x
= w1 + w2 + w3Local Embedding
in Key Space
weightsweights
Outline
• Data Acquisition
• SVBRDF Reconstruction
• Validation
• Projection depend on the environmental lighting conditions• preserve distances ⇒ preserve BRDF manifold structure
Key Space vs. Representative Space
• Projection depend on the environmental lighting conditions• preserve distances ⇒ preserve BRDF manifold structure
global distances⇒ preserve neighborhoods
Key Space vs. Representative Space
local distances⇒ preserve weights
Distance Preservation
• preservation evaluation
Distance Preservation
• preservation evaluation
• # of lighting conditions
Distance Preservation
• preservation evaluation
• # of lighting conditions
• criterion: global: τg > 0.9
local: τl > 0.85
Results
Real Material Sample
Extension to local frame variations
• Normal variations • Tangent rotations
Representative Enlargement
…
enlarged BRDFs over tangent rotation
…enlarged BRDFs over normal rotation
Results
Real Material Sample
Results
Real Material Sample
Conclusion
• Manifold bootstrapping captures high-resolution SVBRDF– assumes BRDF forms low-dimensional manifold– decomposes acquisition into two phases– makes sparse measurement in both
• phase one (representatives) = sparse spatial, dense angular• phase two (keys) = sparse angular, dense spatial
– simplifies and accelerates the capture process
Conclusion
• Manifold bootstrapping captures high-resolution SVBRDF– assumes BRDF forms low-dimensional manifold– decomposes acquisition into two phases– makes sparse measurement in both
• phase one (representatives) = sparse spatial, dense angular• phase two (keys) = sparse angular, dense spatial
– simplifies and accelerates the capture process
Acknowledgements
• Paul Debevec for HDR images• Steve Lin for video narration • Anonymous reviewers for helpful comments
THANKS
THANKS
Uniform Measurement Scaling
• Representative Projection
Uniform Measurement Scaling
• Representative Projection
Future Work
• improving the hand-held BRDF scanner• handling self-shadowing and masking effects
Implementation
• capturing parameters: