Top Banner
Pierre Bénard Supervised by François Sillion & Joëlle Thollot 07/07/2011
158
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Pierre Bénard Ph.D. defense, 2011/07/07

Pierre Bénard Supervised by François Sillion & Joëlle Thollot

07/07/2011

Page 2: Pierre Bénard Ph.D. defense, 2011/07/07

• Expressive rendering

An alternative to photorealism

Tools for visual communication

• Interactive techniques

Direct user feedback

Dynamic visualization, video games…

[Herz98]

[GTDS04]

[CAS+97]

2

Page 3: Pierre Bénard Ph.D. defense, 2011/07/07

Okami Okami

Prince of Persia Prince of Persia

Team Forteress 2 Team Forteress 2

Jet Set Radio

3

Page 4: Pierre Bénard Ph.D. defense, 2011/07/07

Texture painting

McGuire et al., Stylized Rendering in Games, SIGGRAPH 2010 Course

4

Page 5: Pierre Bénard Ph.D. defense, 2011/07/07

McGuire et al., Stylized Rendering in Games, SIGGRAPH 2010 Course

Improved Sobel filter

5

Page 6: Pierre Bénard Ph.D. defense, 2011/07/07

McGuire et al., Stylized Rendering in Games, SIGGRAPH 2010 Course

Final compositing (textures + edges + fog)

6

Page 7: Pierre Bénard Ph.D. defense, 2011/07/07

McGuire et al., Stylized Rendering in Games, SIGGRAPH 2010 Course

Original Stylized

7

Page 8: Pierre Bénard Ph.D. defense, 2011/07/07

Temporal incoherencies

8

Page 9: Pierre Bénard Ph.D. defense, 2011/07/07

© Patrick Martin

Spatial Spatial

Attributes Attributes

Primitives Primitives Marks Marks 3D scene 2D image

perspective projection

2D lines and regions

color, shading, curvature…

pastel strokes

[Willats 97, Durand 02]

9

Page 10: Pierre Bénard Ph.D. defense, 2011/07/07

• Marks medium and pattern

Marks Marks

10

Page 11: Pierre Bénard Ph.D. defense, 2011/07/07

vs. Temporal continuity

« Il p

leut b

ergère », Jérém

y Dep

uyd

t (200

5)

11

Random changes Popping / Flickering

Page 12: Pierre Bénard Ph.D. defense, 2011/07/07

Static marks Shower-door effect

12

vs. Coherent Motion

Page 13: Pierre Bénard Ph.D. defense, 2011/07/07

Texture mapping 3D appearance

13

vs. Flatness

Page 14: Pierre Bénard Ph.D. defense, 2011/07/07

Motion coherence

Temporal continuity

Flatness

14

Page 15: Pierre Bénard Ph.D. defense, 2011/07/07

Conflicting goals compromise

Motion coherence

Temporal continuity

Flatness

Texture mapping

Static marks Random changes

15

Ideal solution

Page 16: Pierre Bénard Ph.D. defense, 2011/07/07

• REGIONS

Dynamic Solid Textures P. Bénard, A. Bousseau, J. Thollot, I3D 2009

NPR Gabor Noise

P. Bénard, A. Lagae, P. Vangorp, S. Lefebvre, G. Drettakis, J. Thollot, EGSR 2010

• LINES

Self-Similar Line Artmap P. Bénard, F. Cole, A. Golovinskiy, A. Finkelstein, NPAR 2010

Active Strokes work in progress

16

Temporally coherent bricks for stylized animations

Page 17: Pierre Bénard Ph.D. defense, 2011/07/07

Previous Work

Dynamic Solid Textures

NPR Gabor Noise

Perceptual Evaluation

Mapping Policies

Self Similar Line Artmap

Snakes-based Tracking and Parameterization

17

Page 18: Pierre Bénard Ph.D. defense, 2011/07/07
Page 19: Pierre Bénard Ph.D. defense, 2011/07/07

Previous Work

Dynamic Solid Textures

NPR Gabor Noise

Perceptual Evaluation

Mapping Policies

Self Similar Line Artmap

Active Strokes

19

Page 20: Pierre Bénard Ph.D. defense, 2011/07/07

• Distribution of marks

• Textures

[VBTS07] [VBTS07]

[CTP+03] [CTP+03]

20

Page 21: Pierre Bénard Ph.D. defense, 2011/07/07

• Distribution of marks

Few-marks

Many-marks

• Textures

[VBTS07] [VBTS07]

[CTP+03] [CTP+03]

21

Page 22: Pierre Bénard Ph.D. defense, 2011/07/07

Few-marks [Mei96,Dan99,HE04,

VBTS07,LSF10]

Motion coherence

Temporal continuity

Flatness

[Meie

r96]

3D distribution of anchor points

2D stroke texture

Painterly rendering

22

[VBTS07]

Page 23: Pierre Bénard Ph.D. defense, 2011/07/07

Good compromise: 2D strokes vs. 3D motion

Clutter / holes

Popping / flickering

[Meier96] [Meier96]

or

[LSF10] [LSF10]

[HE04] [HE04]

[Daniels99]

23

Page 24: Pierre Bénard Ph.D. defense, 2011/07/07

Many-marks [KC05,BKTS06]

24

Motion coherence

Temporal continuity

Flatness

Page 25: Pierre Bénard Ph.D. defense, 2011/07/07

Bridge the gap between texture and marks

Strong 2D appearance

Local motion and/or popping

Specific patterns (canvas fiber, watercolor pigments)

[KC05] [KC05]

[BKTS06]

25

Page 26: Pierre Bénard Ph.D. defense, 2011/07/07

• Distribution of marks

• Textures

In image space: local or global

In objet space

[VBTS07] [VBTS07]

[CTP+03] [CTP+03]

26

Page 27: Pierre Bénard Ph.D. defense, 2011/07/07

Local image space [BNTS07]

[BNTS07]

27

Motion coherence

Temporal continuity

Flatness

Page 28: Pierre Bénard Ph.D. defense, 2011/07/07

Strong 2D appearance

Regeneration artifacts

Post-production

[BNTS07]

28

Page 29: Pierre Bénard Ph.D. defense, 2011/07/07

Global image space [CTP+03,CDH06,

BSM+07]

+ +

[CTP+03]

29

Motion coherence

Temporal continuity

Flatness

Page 30: Pierre Bénard Ph.D. defense, 2011/07/07

Strong 2D appearance

Infinite zoom mechanism

3D motion approximated sliding

[CDH+06] [CDH+06]

[BSM+06] [BSM+06]

30

Page 31: Pierre Bénard Ph.D. defense, 2011/07/07

Object space [KLK+00,PHWF01,FMS01]

[PHWF01]

“Tonal Artmaps” 31

Motion coherence

Temporal continuity

Flatness

Page 32: Pierre Bénard Ph.D. defense, 2011/07/07

Exact motion

MIP-mapping

Perspective distortion

Limited styles [FMS01]

[KLK+00]

32

Page 33: Pierre Bénard Ph.D. defense, 2011/07/07

33

Local image space

Motion coherence

Temporal continuity

Flatness

Global image space

Motion coherence

Temporal continuity

Flatness

Object space

Motion coherence

Temporal continuity

Flatness

Few-marks

Motion coherence

Temporal continuity

Flatness

Many-marks

Motion coherence

Temporal continuity

Flatness

Page 34: Pierre Bénard Ph.D. defense, 2011/07/07

Previous Work

Dynamic Solid Textures

NPR Gabor Noise

Perceptual Evaluation

Mapping Policies

Self Similar Line Artmap

Active Strokes

Motion coherence

Temporal continuity

Flatness

Dynamic Solid Textures

34

Page 35: Pierre Bénard Ph.D. defense, 2011/07/07

• Real-time texture-based approach

• Accurate 3D motion object space approach

• Infinite zoom Dynamic Canvas [CTP+03]

• Easy parameterization solid textures

35

Page 36: Pierre Bénard Ph.D. defense, 2011/07/07

P. Bénard, A. Bousseau, J. Thollot, Dynamic Solid Textures for Real-Time Coherent Stylization. I3D 2009

36

Page 37: Pierre Bénard Ph.D. defense, 2011/07/07

• Texture “Fractalization”

1 solid texture : 4 octaves

octave 2 octave 1 octave 3 octave 4 weighted sum

freq. x2 freq. x2 freq. x2

37

Page 38: Pierre Bénard Ph.D. defense, 2011/07/07

• Zoom cycle

octave 2 octave 1 octave 3 octave 4 weighted sum

38

Page 39: Pierre Bénard Ph.D. defense, 2011/07/07

39

Page 40: Pierre Bénard Ph.D. defense, 2011/07/07

• One solid texture + 3D coordinates

• Procedural textures [Per85, Ola05]

or synthesized from 2D exemplars [KFCO+07]

• Ogre3D rendering engine

Additional cost: 10% compared to Gouraud shading

40

Page 41: Pierre Bénard Ph.D. defense, 2011/07/07

Homogeneity

Dynamic Solid Textures Standard 3D textures

41

Page 42: Pierre Bénard Ph.D. defense, 2011/07/07

42

Shot in Ogre3D

Page 43: Pierre Bénard Ph.D. defense, 2011/07/07

• Trade-off: Medium fidelity vs. Temporal continuity

• Linear blending new frequencies contrast loss

43

Page 44: Pierre Bénard Ph.D. defense, 2011/07/07

Statistical Analysis

-1.5

Z-Scores

-1.0 -0.5 0.0 0.5 1.0

Grid Dots Hatching Paper Noise Paint Pigments Regular patterns

Irregular patterns

Cross- hatching

-1.5

Z-Scores

-1.0 -0.5 0.0 0.5 1.0 1.5

Grid Dots Hatching Paper Noise Paint Pigments Near-regular

patterns Irregular patterns

Cross- hatching

S1

S2

P. Bénard, J. Thollot, F. Sillion, Quality Assessment of Fractalized NPR Textures. APGV 2009

44

Page 45: Pierre Bénard Ph.D. defense, 2011/07/07

Previous Work

Dynamic Solid Textures

NPR Gabor Noise

Perceptual Evaluation

Mapping Policies

Self Similar Line Artmap

Active Strokes

Motion coherence

Temporal continuity

Flatness

NPR Gabor Noise

45

Page 46: Pierre Bénard Ph.D. defense, 2011/07/07

P. Bénard, A. Lagae, P. Vangorp, S. Lefebvre, G. Drettakis, J. Thollot, A Dynamic Noise Primitive for Coherent Stylization. EGSR 2010

46

Page 47: Pierre Bénard Ph.D. defense, 2011/07/07

• Sparse convolution [Lewis 84,89]

• Spot Noise [van Wijk 91]

• Gabor Noise [LLDD09]

Offers significant spectral control

Support anisotropy

Our solution: NPR Gabor Noise

47

Page 48: Pierre Bénard Ph.D. defense, 2011/07/07

• Definition

Sum of randomly positioned and weighted kernels

Gabor kernel

noise

random positions and weights

48

anisotropic isotropic 𝐺 𝐾, 𝑎 × cos(𝐹0, 𝜔0)

Page 49: Pierre Bénard Ph.D. defense, 2011/07/07

• Flatness

Noise parameters in image space

Evaluation in image space

49

Page 50: Pierre Bénard Ph.D. defense, 2011/07/07

• Flatness

• Coherent motion

Point distribution on the surface of the 3D model

50

Page 51: Pierre Bénard Ph.D. defense, 2011/07/07

• Flatness

• Coherent motion

51

Page 52: Pierre Bénard Ph.D. defense, 2011/07/07

• Flatness

• Coherent motion

• Temporal continuity

Smooth LOD mechanism

52

Page 53: Pierre Bénard Ph.D. defense, 2011/07/07

• Sample 3D triangles (GPU)

2D Poisson distribution with constant screen space density

Far • small screen area • less points

Close • large screen area • more points

53

Page 54: Pierre Bénard Ph.D. defense, 2011/07/07

• Generate 2D point sprites

Point distribution Texture sprites

54

Page 55: Pierre Bénard Ph.D. defense, 2011/07/07

• Blending scheme using statistical properties

Reduce popping

Preserve noise appearance

Far Close

visibility = 0 visibility = 1

55

Page 56: Pierre Bénard Ph.D. defense, 2011/07/07

56

Page 57: Pierre Bénard Ph.D. defense, 2011/07/07

57

Page 58: Pierre Bénard Ph.D. defense, 2011/07/07

58

Page 59: Pierre Bénard Ph.D. defense, 2011/07/07

59

Page 60: Pierre Bénard Ph.D. defense, 2011/07/07

60

Page 61: Pierre Bénard Ph.D. defense, 2011/07/07

• Interactive scheme: remaining popping

“Procedural” approach

Slower, but should avoid popping

Useful for high quality offline rendering

• Gabor kernel

Temporally coherent spot noise

61

Page 62: Pierre Bénard Ph.D. defense, 2011/07/07

Previous Work

Dynamic Solid Textures

NPR Gabor Noise

Perceptual Evaluation

Mapping Policies

Self Similar Line Artmap

Active Strokes

62

Page 63: Pierre Bénard Ph.D. defense, 2011/07/07

• Evaluate success of various solutions according to:

Motion

coherence Temporal continuity

Flatness

P. Bénard, A. Lagae, P. Vangorp, S. Lefebvre, G. Drettakis, J. Thollot, A Dynamic Noise Primitive for Coherent Stylization. EGSR 2010

63

Page 64: Pierre Bénard Ph.D. defense, 2011/07/07

• Methodology

15 naïve subjects, ~ 20-30 minutes

• Ranking tasks “Rank the images/videos according to … ”

64

Page 65: Pierre Bénard Ph.D. defense, 2011/07/07

Extreme cases

Local image space Global image space Object space

Many-marks

Adv D2D DST

nprGN SD TM

65

Page 66: Pierre Bénard Ph.D. defense, 2011/07/07

Adv D2D DST

nprGN SD TM

66

Page 67: Pierre Bénard Ph.D. defense, 2011/07/07

• Image space methods more flat for simple scene

• Many 3D cues flatness hard to perceive in complex scene

Flatness

Coherent motion

Temporal continuity

Pleasantness

Shower Door

Texture Mapping

Object space

Local image space

Global image space

Many-marks /

67

Page 68: Pierre Bénard Ph.D. defense, 2011/07/07

• Object space methods more coherent

Flatness Coherent motion

Temporal continuity

Pleasantness

Shower Door

Texture Mapping

Object space

Local image space /

Global image space /

Many-marks / /

68

Page 69: Pierre Bénard Ph.D. defense, 2011/07/07

• High variance

• Advection and NPR Gabor Noise produce more changes: organic motion

Flatness Coherent motion

Temporal continuity

Pleasantness

Shower Door

Texture Mapping

Object space

Local image space /

Global image space / /

Many-marks / / /

69

Page 70: Pierre Bénard Ph.D. defense, 2011/07/07

• Object space approaches more pleasant

• Strong correlation with “motion coherence”

most important criteria to preserve

Flatness Coherent motion

Temporal continuity

Pleasantness

Shower Door

Texture Mapping

Object space

Local image space / /

Global image space / /

Many-marks / / / /

70

Page 71: Pierre Bénard Ph.D. defense, 2011/07/07

• First formal evaluation

• Intrinsic limitations

Hatching

3D scenes

Naïve users

71

Page 72: Pierre Bénard Ph.D. defense, 2011/07/07

• Two new solutions

Object space: Dynamic Solid Texture

Many-marks: NPR Gabor Noise

72

Motion coherence

Temporal continuity

Flatness

NPR Gabor Noise

Motion coherence

Temporal continuity

Flatness

Dynamic Solid Textures

Page 73: Pierre Bénard Ph.D. defense, 2011/07/07
Page 74: Pierre Bénard Ph.D. defense, 2011/07/07

• Simulation of brush strokes

Parameterization

0

l

Brush path

Line texture

74

Page 75: Pierre Bénard Ph.D. defense, 2011/07/07

Previous Work

Dynamic Solid Textures

NPR Gabor Noise

Evaluation

Mapping Policies

Self Similar Line Artmap

Active Strokes

75

Page 76: Pierre Bénard Ph.D. defense, 2011/07/07

• Stretching

• Tiling

76

Page 77: Pierre Bénard Ph.D. defense, 2011/07/07

77

Page 78: Pierre Bénard Ph.D. defense, 2011/07/07

• Stretching Temporally coherent

Stretch or compress the texture

• Tiling

Motion coherence

Temporal continuity

Flatness

Stretching

78

Page 79: Pierre Bénard Ph.D. defense, 2011/07/07

79

Page 80: Pierre Bénard Ph.D. defense, 2011/07/07

• Stretching Temporally coherent

Stretch or compress the texture

• Tiling Preserve the texture characteristics

Incoherent sliding

• Fading / Artmap

Motion coherence

Temporal continuity

Flatness

Tiling

80

Page 81: Pierre Bénard Ph.D. defense, 2011/07/07

81

Page 82: Pierre Bénard Ph.D. defense, 2011/07/07

• Stretching Temporally coherent

Stretch or compress the texture

• Tiling Preserve the texture characteristics

Incoherent sliding

• Fading / Artmaps No sliding

Blending artifacts Motion

coherence Temporal continuity

Flatness

Fading

82

Page 83: Pierre Bénard Ph.D. defense, 2011/07/07

Previous Work

Dynamic Solid Textures

NPR Gabor Noise

Evaluation

Mapping Policies

Self Similar Line Artmap

Active Strokes Motion coherence

Temporal continuity

Flatness

SLAM

83

Page 84: Pierre Bénard Ph.D. defense, 2011/07/07

P. Bénard, F. Cole, A. Golovinskiy, A. Finkelstein Self-Similar Texture for Coherent Line Stylization. NPAR 2010

84

Page 85: Pierre Bénard Ph.D. defense, 2011/07/07

Lmax / 2

Lmax

85

Page 86: Pierre Bénard Ph.D. defense, 2011/07/07

1) Density 2) Smooth variation

86

Page 87: Pierre Bénard Ph.D. defense, 2011/07/07

3) Self-Similarity

87

Page 88: Pierre Bénard Ph.D. defense, 2011/07/07

3) Self-Similarity

88

Page 89: Pierre Bénard Ph.D. defense, 2011/07/07

89

Page 90: Pierre Bénard Ph.D. defense, 2011/07/07

• Parametric texture synthesis [PS00]

• Property: (empirical observation, no formal proof)

small change in noise seed small change in output texture [PS00] Portilla and Simoncelli,

A Parametric Texture Model based on Joint Statistics of Complex Wavelet Coefficients, 2000

90

Exemplar

Synthesized Texture

White Noise Seed

[PS00]

Page 91: Pierre Bénard Ph.D. defense, 2011/07/07

=

Wsource Wf

Seed Pyramid

Wf

[WfWf]

0.5

1

α Wf + (1-α) [WfWf]

Lmax

Lmax

Lmax / 2

91

Page 92: Pierre Bénard Ph.D. defense, 2011/07/07

92

Page 93: Pierre Bénard Ph.D. defense, 2011/07/07

Synthesized textures Examples

93

Page 94: Pierre Bénard Ph.D. defense, 2011/07/07

94

Page 95: Pierre Bénard Ph.D. defense, 2011/07/07

95

Page 96: Pierre Bénard Ph.D. defense, 2011/07/07

96

Page 97: Pierre Bénard Ph.D. defense, 2011/07/07

• Input

Silhouettes

Suggestive contours

Apparent ridges

• Arc-length artifacts

Coherent parameterization

97

Page 98: Pierre Bénard Ph.D. defense, 2011/07/07

Previous Work

Dynamic Solid Textures

NPR Gabor Noise

Evaluation

Mapping Policies

Self Similar Line Artmap

Active Strokes

Work in progress in collaboration with C. Lu, F. Cole and A. Finkelstein 98

Page 99: Pierre Bénard Ph.D. defense, 2011/07/07

• Flatness

Remain linear in screen-space

• Motion coherence

Evolve according to the motion of the objet

• Temporal continuity

Adapt to the topological events

99

Page 100: Pierre Bénard Ph.D. defense, 2011/07/07

Coherent Stylized Silhouettes, Kalnins et al., SIGGRAPH 2003

Optimization between 2D and 3D

Multiple brush strokes per line

Dependent on the input connectivity

100

Page 101: Pierre Bénard Ph.D. defense, 2011/07/07

Kalnins et al., Coherent Stylized Silhouettes, SIGGRAPH 2003

101

Page 102: Pierre Bénard Ph.D. defense, 2011/07/07

Snakes

Snakes

3D scene

Lines extraction

Lines extraction

Feature samples

Line drawing

Brush Paths

Brush Paths

Parameterization Parameterization

Stylization Stylization Cleaning Vectorization

Cleaning Vectorization

Tracking Tracking

102

Page 103: Pierre Bénard Ph.D. defense, 2011/07/07

Feature samples

Line drawing

• Position • Local tangent • Velocity

Snakes

Snakes

3D scene

Lines extraction

Lines extraction

Feature samples

Line drawing

Brush Paths

Brush Paths

Parameterization Parameterization

Stylization Stylization Cleaning Vectorization

Cleaning Vectorization

Tracking Tracking

103

Page 104: Pierre Bénard Ph.D. defense, 2011/07/07

3D scene

Lines extraction

Lines extraction

Feature samples

Line drawing

Snakes

Snakes

Brush Paths

Brush Paths

Parameterization Parameterization

Stylization Stylization Cleaning Vectorization

Cleaning Vectorization

Tracking Tracking

104

• Polylines • In image space • Persistent

Page 105: Pierre Bénard Ph.D. defense, 2011/07/07

• Advection

• Relaxation

105 Frame f Frame f+1

Page 106: Pierre Bénard Ph.D. defense, 2011/07/07

• Advection

• Relaxation

106

Page 107: Pierre Bénard Ph.D. defense, 2011/07/07

107

Page 108: Pierre Bénard Ph.D. defense, 2011/07/07

108

Page 109: Pierre Bénard Ph.D. defense, 2011/07/07

3D scene

Lines extraction

Lines extraction

Feature samples

Line drawing

Snakes

Snakes

Brush Paths

Brush Paths

Parameterization Parameterization

Stylization Stylization Cleaning Vectorization

Cleaning Vectorization

Tracking Tracking

109

Page 110: Pierre Bénard Ph.D. defense, 2011/07/07

• Linear screen space parameterization

110

Page 111: Pierre Bénard Ph.D. defense, 2011/07/07

• Segments fitting

111

Page 112: Pierre Bénard Ph.D. defense, 2011/07/07

• Arcs fitting

112

Page 113: Pierre Bénard Ph.D. defense, 2011/07/07

113

Page 114: Pierre Bénard Ph.D. defense, 2011/07/07

114

Page 115: Pierre Bénard Ph.D. defense, 2011/07/07

115

Page 116: Pierre Bénard Ph.D. defense, 2011/07/07

116

Page 117: Pierre Bénard Ph.D. defense, 2011/07/07

Robustness toward noise / outliers

Temporal smoothing

117

Input samples Active strokes

Page 118: Pierre Bénard Ph.D. defense, 2011/07/07
Page 119: Pierre Bénard Ph.D. defense, 2011/07/07

• 4 new bricks for time coherent stylization

Dynamic Solid Textures

NPR Gabor Noise

Self-Similar Line Artmap

Active Strokes

• Formal evaluation

Formulation into 3 goals

2 perceptual studies

1 objective quality metric

119

Page 120: Pierre Bénard Ph.D. defense, 2011/07/07

• 2D Animation

Ongoing work with Antoine Boellinger (Master 1)

120

Page 121: Pierre Bénard Ph.D. defense, 2011/07/07

• 2D Animation

Ongoing work with Antoine Boellinger (Master 1)

• Evaluation & Objective metrics

Optical flow analysis

• Stylization by optimization

Versatile system to explore the space of trade-offs

121

Page 122: Pierre Bénard Ph.D. defense, 2011/07/07

122

Page 123: Pierre Bénard Ph.D. defense, 2011/07/07

123

Page 124: Pierre Bénard Ph.D. defense, 2011/07/07

124

Page 125: Pierre Bénard Ph.D. defense, 2011/07/07

125

Page 126: Pierre Bénard Ph.D. defense, 2011/07/07

• Simple

• Strong motion coherence and temporal continuity

• More suitable for unstructured patterns

• Real-time animation

126

Page 127: Pierre Bénard Ph.D. defense, 2011/07/07

• Input: pairs of 2D texture

• Definition: texture distortion = visual dissimilarity between original and transformed textures

• Goal: define a quantitative metric of this distortion

original transformed

P. Bénard, J. Thollot, F. Sillion, Quality Assessment of Fractalized NPR Textures. APGV 2009

127

Page 128: Pierre Bénard Ph.D. defense, 2011/07/07

• Procedure:

User study ranking of texture pairs according to their distortion

Statistical analysis scale of perceived quality

Correlation investigation objective metric

• Restrictions:

No texture mapping

Static images

128

Page 129: Pierre Bénard Ph.D. defense, 2011/07/07

• Ordinal scale

S1

S2

Statistical Analysis

129

Page 130: Pierre Bénard Ph.D. defense, 2011/07/07

• Thurstone’s law of comparative judgment [Tor58]

Statistical Analysis

-1.5

Z-Scores

-1.0 -0.5 0.0 0.5 1.0

Grid Dots Hatching Paper Noise Paint Pigments Regular

patterns

Irregular

patterns

Cross-

hatching

-1.5

Z-Scores

-1.0 -0.5 0.0 0.5 1.0 1.5

Grid Dots Hatching Paper Noise Paint Pigments Near-regular

patterns

Irregular

patterns

Cross-

hatching

S1

S2

130

Page 131: Pierre Bénard Ph.D. defense, 2011/07/07

-1.5

Z-Scores

-1.0 -0.5 0.0 0.5 1.0 1.5

Grid Dots Hatching Paper Noise Paint Pigments Near-regular

patterns Irregular patterns

Cross- hatching

-1.5

Z-Scores

-1.0 -0.5 0.0 0.5 1.0

Grid Dots Hatching Paper Noise Paint Pigments Regular patterns

Irregular patterns

Cross- hatching

S1

S2

131

Page 132: Pierre Bénard Ph.D. defense, 2011/07/07

Statistical Analysis

S1

S2

-1.5

Z-Scores

-1.0 -0.5 0.0 0.5 1.0

Grid Dots Hatching Paper Noise Paint Pigments Regular patterns

Irregular patterns

Cross- hatching

-1.5

Z-Scores

-1.0 -0.5 0.0 0.5 1.0 1.5

Grid Dots Hatching Paper Noise Paint Pigments Near-regular

patterns Irregular patterns

Cross- hatching

132

Page 133: Pierre Bénard Ph.D. defense, 2011/07/07

Overall contrast of patterns Feature shapes

Statistical Analysis

S1

S2

-1.5

Z-Scores

-1.0 -0.5 0.0 0.5 1.0

Grid Dots Hatching Paper Noise Paint Pigments Regular patterns

Irregular patterns

Cross- hatching

-1.5

Z-Scores

-1.0 -0.5 0.0 0.5 1.0 1.5

Grid Dots Hatching Paper Noise Paint Pigments Near-regular

patterns

Irregular

patterns

Cross-

hatching

Paper Noise Paint Pigments

133

Page 134: Pierre Bénard Ph.D. defense, 2011/07/07

Overall contrast of patterns

Feature shapes

Statistical Analysis

S1

S2

-1.5

Z-Scores

-1.0 -0.5 0.0 0.5 1.0

Grid Dots Hatching Paper Noise Paint Pigments Regular

patterns

Irregular

patterns

Cross-

hatching

-1.5

Z-Scores

-1.0 -0.5 0.0 0.5 1.0 1.5

Grid Dots Hatching Paper Noise Paint Pigments Near-regular

patterns

Irregular

patterns

Cross-

hatching

Irregular patterns Cross-hatching Dots Hatching

134

Page 135: Pierre Bénard Ph.D. defense, 2011/07/07

• Gray level co-occurrence matrix (GLCM) [HSD73]

Texture descriptor [TJ93]

Local image property

Match certain levels of human perception [JGSF76]

Linked to density and pattern coherence criteria

Parameters selection

• Average Co-occurrence Error [CRT01]

High correlation with the perceptual interval scale (Person’s correlation: 0.953 for S1 and 0.836 for S2 )

Correlation with Objective Metrics

135

Page 136: Pierre Bénard Ph.D. defense, 2011/07/07

ACE relevant estimator of the distortion

S1

S2

-1.5

Z-Scores

-1.0 -0.5 0.0 0.5 1.0

Grid Dots Hatching Paper Noise Paint Pigments Regular patterns

Irregular patterns

Cross- hatching

-1.5

Z-Scores

-1.0 -0.5 0.0 0.5 1.0 1.5

Grid Dots Hatching Paper Noise Paint Pigments Near-regular

patterns Irregular patterns

Cross- hatching

AC

E

1.2

1.0

0.8

0.6

0.4

AC

E

1.4

1.2

1.0

0.8

0.6

r² = 0.6992

r² = 0.9075

136

Page 137: Pierre Bénard Ph.D. defense, 2011/07/07

• More suitable for low-contrast media

• Simple

• Strong motion coherence and temporal continuity

• Real-time animation

137

Page 138: Pierre Bénard Ph.D. defense, 2011/07/07

l=0.5

l=0

l=1

Lmax

138

Page 139: Pierre Bénard Ph.D. defense, 2011/07/07

Input: white noise seed

Parametric Texture Synthesis [PS00]

Seed Pyramid

Input: texture example

Output: SLAM

139

Page 140: Pierre Bénard Ph.D. defense, 2011/07/07

Self-Similar Texture for Coherent Line Stylization, Bénard et al. 2010

2D Infinite zoom: Self-Similar Line Artmap (SLAM)

Propagation using a 2D buffer aliasing ambiguities

One brush stroke per line

140

Page 141: Pierre Bénard Ph.D. defense, 2011/07/07

141

Page 142: Pierre Bénard Ph.D. defense, 2011/07/07

• Linear screen-space parameterization using Self-similar Line Artmaps

• Multiple brush strokes per feature line

• New propagation mechanism

142

Page 143: Pierre Bénard Ph.D. defense, 2011/07/07

• Feature samples – lines extracted in 2D or 3D

3D position

Local tangent

2D velocity

143

Page 144: Pierre Bénard Ph.D. defense, 2011/07/07

Snakes

Snakes

Tracking Advection Advection

3D scene

Lines extraction

Lines extraction

Feature samples

Relaxation Relaxation

Vectorization

Coverage Coverage Connectivity Connectivity

Line drawing

Brush Paths

Brush Paths

Geometry Geometry

Parameterization Parameterization

Stylization Stylization

144

Page 145: Pierre Bénard Ph.D. defense, 2011/07/07

• 2D polylines active contours (Kass et al. 1988)

• Goals

Coherence continuous evolution across frames

Accuracy faithfully represent shape

Coverage Level of Detail

Simplicity simple topology

Length stylization freedom

Trac

kin

g V

ect

ori

zati

on

145

Page 146: Pierre Bénard Ph.D. defense, 2011/07/07

• Directional coverage radius

• Local vectorization operators

Trim

Extend

Merge

Split

Coverage Coverage

Connectivity Connectivity

146

Page 147: Pierre Bénard Ph.D. defense, 2011/07/07

Coverage Coverage

Connectivity Connectivity

147

Page 148: Pierre Bénard Ph.D. defense, 2011/07/07

• Directional coverage radius

• Local vectorization operators

Trim

Extend

Merge

Split

• Applied sequentially in a greedy fashion

Coverage Coverage

Connectivity Connectivity

148

Page 149: Pierre Bénard Ph.D. defense, 2011/07/07

• Reprojection similar to Kalnins et al. 2003

• Computed for feature lines only connection between contour and feature sample needed

Advection Advection

149

Page 150: Pierre Bénard Ph.D. defense, 2011/07/07

• Minimize the energy:

• Re-sampling [DM00]

Relaxation Relaxation

External – Attraction by the features – Mass-spring forces prevents shrinking

Internal – Continuity – Smoothness

150

Page 151: Pierre Bénard Ph.D. defense, 2011/07/07

Snakes

Snakes

Tracking Advection Advection

3D scene

Lines extraction

Lines extraction

Feature samples

Relaxation Relaxation

Vectorization

Coverage Coverage Connectivity Connectivity

Line drawing

Brush Paths

Brush Paths

Geometry Geometry

Parameterization Parameterization

Stylization Stylization

151

Page 152: Pierre Bénard Ph.D. defense, 2011/07/07

• Consistently increasing subset of a snake

152

Page 153: Pierre Bénard Ph.D. defense, 2011/07/07

• Uniform in screen-space linear arc-length parameterization :

avec = slope = phase = arc-length

• Evolve according to the motion and topology of the contours parameterization at each vertex

Parameterization Parameterization

153

Page 154: Pierre Bénard Ph.D. defense, 2011/07/07

• Propagation at each vertex

Parameterization stored between two frames

Parameterization Parameterization

154

Page 155: Pierre Bénard Ph.D. defense, 2011/07/07

• Propagation at each vertex

• Linearization (in the least-square sense) p

aram

eter

izat

ion

arc-length

Parameterization Parameterization

155

Page 156: Pierre Bénard Ph.D. defense, 2011/07/07

• Propagation at each vertex

• Vectorization events

Split

Extend

Trim

Merge: mechanism to avoid parameterization discontinuities

• Linearization

propagated directly

Parameterization Parameterization

156

Page 157: Pierre Bénard Ph.D. defense, 2011/07/07

• Only if the slope and phase match

• Leveling mechanism pushes the 2 parameterizations to their mid-value:

with

par

amet

eriz

atio

n

arc-length

Parameterization Parameterization

157

Page 158: Pierre Bénard Ph.D. defense, 2011/07/07

• Feature lines dependency

Robustness toward noise / outliers

Temporal smoothing

158