Visual Texture Analysis: From Similarity To Material Properties · 2015-07-09 · Restoration Based on Nonlocal Self-Similarity . Dabov, Foi, Katkovnik, Egiazarian, “Image denoising
Post on 06-Jul-2020
3 Views
Preview:
Transcript
LLNL-PRES-670573 This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344. Lawrence Livermore National Security, LLC
Visual Texture Analysis:
From Similarity To Material Properties CASIS, LLNL
Thrasos Pappas EECS, Northwestern University
On Sabbatical Leave at LLNL
May 13, 2015
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 2
People
Jana Zujovic, Northwestern Univ. – now at FutureWei Guoxin Jin, Northwestern Univ. Jing Wang, Northwestern Univ. Dzung Nguyen, Northwestern Univ. Shengxin Zha, Northwestern Univ. Xiaonan Zhao, Northwestern Univ. – now at Google Pubudu Madhawa Silva, Northwestern Univ. Qian Yu, Northwestern Univ. David Neuhoff, Univ. of Michigan Rene van Egmond, TU Delft Huib de Ridder, TU Delft Alessandro Foi, Tampere University of Technology Matteo Maggioni, Tampere University of Technology Matthew Reyes, Univ. of Michigan Yuanhao Zhai, Univ. of Michigan Randy Roberts, LLNL NNSA, SONY Labs
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 3
What is Texture?
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 4
What is Texture?
“An image of visual texture is spatially homogeneous and typically contains repeated structures, often with some random variation, e.g., random positions, orientations or colors.” [Portilla & Simoncelli]
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 5
Texture Similarity
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 6
Material Identification
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 7
Material Identification
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 8
Texture Similarity and Identification
Applications
Content-Based Indexing and Retrieval
Compression
Visual to tactile conversion Semantic Information Extraction
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 9
Texture Similarity and Identification
Applications
Content-Based Indexing and Retrieval • Retrieval of similar textures
Compression
Visual to tactile conversion Semantic Information Extraction
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 10
Texture Similarity and Identification
Applications
Content-Based Indexing and Retrieval • Retrieval of similar textures
Compression • Perceptually lossless • Perceptually lossy
Visual to tactile conversion Semantic Information Extraction
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 11
Texture Similarity and Identification
Applications
Content-Based Indexing and Retrieval • Retrieval of similar textures
Compression • Perceptually lossless • Structurally lossless • Perceptually lossy
Visual to tactile conversion Semantic Information Extraction
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 12
Texture Similarity and Identification
Applications
Content-Based Indexing and Retrieval • Retrieval of similar textures
Compression • Perceptually lossless • Structurally lossless • Perceptually lossy
Visual to tactile conversion Semantic Information Extraction
• Computer vision: Focus on objects rather than material perception and texture [Adelson, HVEI’01]
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 13
Restoration Based on Nonlocal Self-Similarity
Dabov, Foi, Katkovnik, Egiazarian, “Image denoising by sparse 3D transform-domain collaborative filtering”, IEEE T-IP, 2007
Create groups of similar patches associated with a given “reference” block
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 14
Microvascular Image Classification
Control Mucosa Images – Sarah Ruderman
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 15
Microvascular Image Classification
Tumor Vasculature Images – Sarah Ruderman
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 16
Feature Vector Distance (FVD) Matrix
The darker the more similar
Control
Con
trol
Tumor
Tum
or
5 10 15 20 25 30 35 40 45
5
10
15
20
25
30
35
40
45
20
40
60
80
100
120
140
160
180
200
220
Distance Index
Microvascular Image Classification
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 17 17
Huib de Ridder, Rene van Egmond Faculty of Industrial Design Engineering
Delft University of Technology
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 18
Subjective vs. Objective Texture Similarity
11.3 9.0 8.2
1 9 10
PSNR
Subjective
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 19
Subjective vs. Objective Texture Similarity
0.83 0.96 0.98
1 9 10
STSIM-2 global
Subjective
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 20
Texture Similarity – PSNR?
8.2 17.4 17.5 10.4
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 21
Texture Similarity – STSIM-2 global
0.98 0.99 0.83 0.96
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 22
Separating Grayscale and Color
Different subjects put different emphasis on structure and color composition for texture similarity
Separate metrics for grayscale and color [Zujovic, ICIP’09] • Use grayscale component to isolate/approximate structure • Structure in chrominance? • End user/application decides how to combine
Can develop more effective metrics separately
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 23
SSIMs – Grayscale
Compare local
image statistics
Point-by-point
Based on papers by Z. Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 24
CW-SSIM (Perceptually-Weighted)
frequency analysis
frequency analysis
decoded
image
source
image
SSIM error calculation
frequency sensitivity
spatial pooling
perceptual
error
masking
frequency pooling
region-based
calculation
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 25
Perceptual Quality Metrics
frequency analysis
frequency analysis
decoded
image
source
image
MSE error calculation
frequency sensitivity
spatial pooling
perceptual
error
masking
frequency pooling
point-by-point
calculation
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 26
SSIMs – Grayscale
Compare local
image statistics
Point-by-point
Based on papers by Z. Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 27
No point-by-point comparisons
– Drop structure term
Local image statistics
– Mean and variance
– First order correlation coefficients
– Crossband correlations
Texture synthesis [Portilla&Simoncelli’00]
Structural Texture Similarity Metrics
Grayscale
J. Zujovic, T.N. Pappas, D.N. Neuhoff, T-IP’13
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 29
Portilla and Simoncelli’00
Universal parametric statistical model Necessary and sufficient parameters
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 30
STSIM-2: Subband Statistics
To compare images and : For each subband and find: Means and standard deviations Horizontal autocorrelations
Vertical autocorrelations Crossband correlations
J. Zujovic, T.N. Pappas, D.N. Neuhoff, T-IP’13
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 31
STSIM-2: Crossband Correlations
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 33
STSIM-2: Comparing Statistics
J. Zujovic, T.N. Pappas, D.N. Neuhoff, T-IP’13
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 34
STSIM-2: Pooling
J. Zujovic, T.N. Pappas, D.N. Neuhoff, T-IP’13
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 35
For each image, form feature vector consisting of all statistics for all subbands, including cross-correlations:
FX = (f1,x, f2,x, …, fM,x) , FY = (f1,y, f2,y, …, fM,y), M = 82
Compute Mahalonobis distance
where is the (overall or intra-class) variance of i th statistic across all images in the database.
STSIM: Mahalanobis distance
QSTSIM-M (x,y) =( f ix - f iy )2
s fi
2
i=1
M
å = fxTM fy
2if
J. Zujovic, T.N. Pappas, D.N. Neuhoff, T-IP’13
M. Maggioni, G. Jin, A. Foi, T.N. Pappas, ICIP’14
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 36
Local versus Global
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 37
Traditional methods • Raw color histogram comparisons
Our approach • Remove unnecessary color detail
— Extract dominant colors — Using adaptive clustering [Pappas’92]
• Use more sophisticated distance metric — EMD [Rubner’00], OCCD [Mojsilovic’02]
• Use “perceptually uniform” color space (L*a*b*)
Color Composition Similarity
Zujovic, Pappas, Neuhoff, ICIP’09
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 38
Color Composition Similarity
Original images
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 39
Color Composition Similarity
ACA Local Averages
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 40
Color Composition Similarity
ACA Local Averages
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 41
Color Composition Similarity
ACA Local Averages plus K-means
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 42
Minimum cost graph matching problem Quantize percentages of colors into “units” Example: 5% units = 20 units total
Optimal Color Composition Distance
Image x
Image y
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 43
Texture Similarity Metric Evaluation
Poor agreement among subjects (ICC = 0.66) – Rank correlation?
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 44
Testing Domains for Texture Similarity
monotonic distortion identical
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 45
Testing Domains for Texture Similarity
dissimilar similar identical
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 46
Testing Domains for Texture Similarity
Limitations/Capabilities of Human Perception Application Requirements Testing Domains
• Quantify (perceptually) small amounts of distortion • Similar vs. dissimilar • Retrieval of “identical” textures
Absolute scale/threshold?
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 47
Desired Texture Similarity Metric
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
similar
dissimilar
Me
tric
va
lue
s
Subjective scores of similarity between pairs of texture images
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 48
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Desired Texture Similarity Metric
similar
dissimilar
Me
tric
va
lue
s
Subjective scores of similarity between pairs of texture images
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 49
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Desired Texture Similarity Metric
similar
dissimilar
Me
tric
va
lue
s
Subjective scores of similarity between pairs of texture images
monotonic
identical
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 50
Color Analogy: MacAdam Ellipses
Color: • JNDs • Cannot quantify large
perceptual distances Texture:
• JNDs can be obtained by existing perceptual quality metrics (solid)
• “Ellipses” of similar textures (dashed)
• “Ellipses” of identical textures (dotted)
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 51
Testing Domains for Texture Similarity
Different domains require • Different metric evaluation criteria • Different subjective and objective tests • Different texture similarity metrics?
Retrieval of “identical” textures • Known-item search
Similar vs. dissimilar textures Quantify (perceptually) small amounts of
distortion J. Zujovic, T.N. Pappas, D.N. Neuhoff, H. de Ridder, R. van Egmond JOSAA’15
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 52
Building The Database
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 53
Precision at One
Measures how many times the first retrieved texture was the correct one
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 54
Mean Reciprocal Rank (MRR)
Measures the average inverse rank of the first correct retrieved image
… RR = 1
… RR = 1
… RR = 0.33
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 55
Mean Average Precision (MAP)
Measures average precision when cutoff is made at 1st, 2nd,…, Nth retrieved image
precision=1 precision=0.5
precision=0.66
precision=0.5
…
AP = 0.5*(1*1 + 0.5*0 + 0.66*1 + 0.5*0 + …) = 0.83
two relevant documents in database
precision after first retrieved document
indicator that document was relevant
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 56
Information Retrieval Statistics
Precision at one Mean Reciprocal Rank Mean Average Precision0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Pearson's r Spearman's rho0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
PSNR
SSIM
CWSSIM
CWSSIM global
STSIM-1
STSIM-1 global
STSIM-2
STSIM-2 global
STSIM-M
Ojala et al.
Do and Vetterli
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 57
Statistical Validation
P@1: Cochrane’s Q test – Applied to each pair of metrics to determine statistical significance
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 58
Receiver Operating Characteristic –
ROC
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 59
Receiver Operating Characteristic –
ROC
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 60
Receiver Operating Characteristic –
ROC
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 61
Testing Domains for Texture Similarity
Different domains require • Different metric evaluation criteria • Different subjective and objective tests • Different texture similarity metrics?
Retrieval of “identical” textures • Known-item search
Similar vs. dissimilar textures Quantify (perceptually) small amounts of
distortion
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 62
Goal: find clusters of similar textures • Similar within clusters • Dissimilar across clusters
Relatively large database • Difficult to see and compare all images at once
ViSiProG: Visual Similarity by Progressive Grouping • Build similarity groups one at a time • Build each group in a step-by-step fashion • Each user builds multiple clusters • Combine results from different users
Finding Clusters of Similar Textures
Zujovic, Pappas, Neuhoff, de Ridder, van Egmond, JOSAA’15
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 63
ViSiProG – Grayscale
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 64
ViSiProG – Grayscale
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 65
ViSiProG – Grayscale
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 66
ViSiProG – Grayscale
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 67
ViSiProG – Grayscale
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 68
ViSiProG – Grayscale
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 69
246 grayscale images Subjects asked to form groups of 9 similar images Formed similarity matrix
• Only 134 images were selected in a group Used spectral clustering to analyze results
• Cluster the data based on human similarity scores
Finding Clusters of Similar Textures
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 70
Similarity Clusters Examples
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 71
Information Retrieval Statistics
Precision at One Mean Reciprocal Rank Mean Average Precision0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Pearson's r Spearman's rho0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
PSNR
SSIM
CWSSIM
CWSSIM global
STSIM-1
STSIM-1 global
STSIM-2
STSIM-2 global
STSIM-M
Ojala et al.
Do and Vetterli
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 72
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
False positive rate
Tru
e p
osi
tive
ra
te
ROC space curves
PSNR
SSIM
CWSSIM
STSIM global
STSIM2 global
Receiver Operating Characteristic –
ROC
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 73
ViSiProG – Color Composition
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 74
Similarity Clusters Examples
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 75
Testing Domains for Texture Similarity
Different domains require • Different metric evaluation criteria • Different subjective and objective tests • Different texture similarity metrics?
Retrieval of “identical” textures • Known-item search
Similar vs. dissimilar textures Quantify (perceptually) small amounts of
distortion
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 76
Distortion Quantification
Subjects asked to rank the distortions from “best” to “worst”
Original
Low Medium High
Rotations
Shifts
Warps
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 77
Original Database
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 78
Analyzing the Results
Subjective similarity scores: • Average ranks (Borda’s rule) • Thurstonian scaling • Multidimensional scaling
Qualitatively similar results
Correlate with objective (metric) scores
78
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 79
Analyzing the Results
Pearson's r Spearman's rho0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Pearson's r Spearman's rho0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
PSNR
SSIM
CWSSIM
CWSSIM global
STSIM-1
STSIM-1 global
STSIM-2
STSIM-2 global
STSIM-M
Ojala et al.
Do and Vetterli
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 80
Material Properties
Texture appearance depends on • Material (reflectance, transmittance) • Surface geometry • Lighting (color, direction, …) • Viewing angle
Difficult to separate • “Inverse Optics” approach • Computationally intensive
Rely on natural texture statistics • Ecological approach • Fast • Works most of the time, but … • Can make errors (illusions)
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 81
Material Properties
Rely on natural image statistics to estimate specific attributes • Roughness • Glossiness • Directionality • Regularity • Scale
Can be estimated/compared outside quantitative range of STSIMs
Provide strong clues about material properties
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 82
Material Properties
Texture appearance depends on material, surface geometry, and lighting
Difficult to separate Rely on image statistics to estimate
specific attributes • Roughness • Glossiness • Directionality • Regularity • Scale
Can be estimated/compared outside quantitative range of STSIMs
Provide strong clues about material properties
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 83
Manipulation on Statistics
Input
Texture
Statistical
analysis
Statistics
manipulation
Gloss
manipulation
Negative skewness
Positive skewness
Example: Skewness hypothesis (Motoyoshi et al., 2007)
Example: λ-curve transformation (Wijntjes & Pont, 2010)
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 84
𝑌 =𝑋
𝑋2 + 𝜆(1 − 𝑋2)
Input, output values
Stretch degree in relief
depth
Stretches a Lambertian surface in depth; affects skewness of the luminance histogram
Lambertian surface
λ-curve Transformation
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 85
𝑌 =𝑋
𝑋2 + 𝜆(1 − 𝑋2)
Input, output values
Stretch degree in relief
depth
Stretches a Lambertian surface in depth; affects skewness of the luminance histogram
Natural surface
Glossier?
λ-curve Transformation
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 86
Manipulation on Image Cues
Input
Texture
Gloss
manipulation
Image cue
manipulation
Image cues: specular coverage, specular contrast, specular sharpness
Alternative approach (Marlow and Anderson, 2013):
specular coverage specular contrast specular sharpness
Synthetic images: hard to do on natural textures
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 87
Motivation
Even though we have multiple gloss related attributes:
manipulation of gloss is constrained by surface geometry and illumination direction it is difficult to control these attributes at the perceptual level
Goal
Transformation method to manipulate visual gloss of natural textures Without constraints on surface geometry and illumination conditions Investigate the relation between perceived gloss and perceived contrast
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 88
Method
Subband
Decompositio
n
Subband S-
curve
Transformatio
n
Output
Image Original
Image
Subjective experiments
Test the relation between perceived gloss and perceived contrast as you apply the S-curve transformation Test whether contrast adjustment could compensate for the gloss difference generated by illumination directions
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 89
Stimuli
Collection of natural and synthetic textures (256x256) Corbis website (natural, color) Pictures of black and white spaghetti (natural, color) CUReT texture database (natural, color and grayscale)
- Illumination: 0.196 radians and 0.589 radians in polar angle Synthesized Lambertian surfaces: Rendered Brownian surfaces (grayscale)
- Illumination: 0 and 50 degrees in polar angle
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 90
Examples
0.196 radians 0.589radians
CUReT
Lambertian
0 degrees 50 degrees
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 91
Image decomposition
1 cycles/pic
2 cycles/pic 4 cycles/pic 8 cycles/pic
16 cycles/pic
32 cycles/pic
64 cycles/pic
128 cycles/pic
original
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 92
S-curve Transformation
𝑌 = 𝜇 −𝜇 − 𝑋
𝛼2 𝜇 − 𝑋 2 1 − 1/𝑠2 + 1/𝑠2
Input value, output value
Mean of input values
s
S-curve with different 𝑠 values
𝑠 is the sole control parameter controlling the transformation
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 93
Haun & Peli, 2013: How do different spatial frequencies contribute to the overall perceived contrast? Weighting scheme for overall perceptual effect on contrast: Spatial frequencies around the peak of CSF (1-6 cycles/degree) contribute most to contrast perception, low and high frequency bands contribute less.
Perceived Contrast Weighting Scheme
.1 .3 .5 1 2 4 8 16 Spatial frequency(cycles/degree)
Decision weight
Apply the S-curve transformation with slope S to all frequency bands, except the low and high bands For low and high bands: Use slope 2 S when S > 1 Use slope .5 S when S < 1
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 94
Lambertian
CUReT-028
Pasta
S-curve Transformed Images
S = 0.25 S = 2 original S = 0.5 S = 4
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 95
λ = 0.25 λ = 2 original λ = 0.5 λ = 4
Lambertian
CUReT-028
Pasta
λ-curve Transformed Images
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 96
Graphical User Interface: Experiment I
Session 1: Arrange images in order of decreasing gloss Session 2: Arrange images in order of decreasing contrast
Each trial: Original and six S-curve or λ-curve transformed images in random order (Use one transformation, S or λ, in each trial) Random order of curves, random order of images
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 97
CorrelationRelation between Perceived Gloss and Contrast
Pearson Correlation 20 subjects
• Strong positive correlation between perceived contrast and slope of S-curve
Correlation between S-Curve and Perceived
Contrast
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 98
Pearson Correlation
• Positive correlation between perceived gloss and slope of S-curve • But larger variation than contrast • Perceived contrast and perceived gloss are closely related • Do people respond to systematic changes rather than gloss or contrast?
Correlation between S-Curve and
Perceived Gloss
20 subjects
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 99
Relation between Perceived Gloss and Perceived
Contrast
Average rankings between contrast and gloss in s-curve transformation
• Within the S-curve transformation, perceived gloss is positively correlated with perceived contrast across different types of textures.
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 100
Pearson Correlation
• Very little correlation between perceived contrast and slope of λ-curve • Except for synthetic Lambertian surfaces
Correlation between λ-curve and Perceived
Contrast
20 subjects
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 101
• Very little correlation between perceived contrast and slope of λ-curve
• Except for synthetic Lambertian surfaces • Controlling histogram skewness, the λ-curve is not sufficient to
manipulate the perceived gloss of natural textures
Correlation between λ-curve and Perceived
Gloss
Pearson Correlation 20 subjects
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 103
Gloss matching: Pairwise comparison Each trial: one original image in oblique illumination direction and one S-curve transformed version in near-frontal illumination
Graphical User Interface: Experiment II
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 104
Probability that frontal illuminated texture was selected as
glossier
Experimental Results
CUReT_35 CUReT_10 Lambertian_09 Lambertian_11
CUReT_28 CUReT_53
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 105
Conclusions
We proposed a novel transformation method to manipulate the perceived gloss of natural textures with unknown geometry and illumination field.
Natural textures behave differently than synthesized Lambertian surfaces.
There is a strong positive correlation between perceived gloss and perceived contrast across different types of images including Lambertian surface.
Contrast modification could compensate for gloss difference generated due illumination directions, within a certain range of directions
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 106
Material Properties
Texture appearance depends on material, surface geometry, and lighting
Difficult to separate Rely on image statistics to estimate
specific attributes • Roughness • Glossiness • Directionality • Regularity • Scale
Can be estimated/compared outside quantitative range of STSIMs
Provide strong clues about material properties
Lawrence Livermore National Laboratory LLNL-PRES-xxxxxx 107
Questions?
Thank you!
107
top related