Page 1
Learning hatching for pen-and-ink illustrations of surfaces
Evangelos Kalogerakis1,2, Derek Nowrouzehahrai1,3,4,Simon Breslav1,5, Aaron Hertzmann1
1University of Toronto, 2Stanford University, 3Disney Research Zurich, 4University of Montreal, 5Autodesk Research
Page 2
Exemplar shape Artist’s illustration
Goal: Synthesis of hatching illustrations
Page 3
Exemplar shape Artist’s illustration
Learnedmodel of hatching
Goal: Synthesis of hatching illustrations
Page 4
Exemplar shape Artist’s illustration
Learnedmodel of hatching
Goal: Synthesis of hatching illustrations
Input shape Synthesized illustration
Page 5
Challenge: understanding hatching styles
Page 6
Related work: hatching smooth surfacesIso-parametric curves [Saito and Takahashi 1990, Winkenbach and Salesin 1996]
Page 7
Related work: hatching smooth surfacesIso-parametric curves [Saito and Takahashi 1990, Winkenbach and Salesin 1996]
Smooth curvature directions and shading-based tone [Elber 1998, Hertzmann and Zorin 2000]
Page 8
Related work: hatching smooth surfacesIso-parametric curves [Saito and Takahashi 1990, Winkenbach and Salesin 1996]
Smooth curvature directions and shading-based tone [Elber 1998, Hertzmann and Zorin 2000]
Shading gradients[Singh and Schaefer 2010]
Page 9
Related work: hatching smooth surfacesIso-parametric curves [Saito and Takahashi 1990, Winkenbach and Salesin 1996]
Smooth curvature directions and shading-based tone [Elber 1998, Hertzmann and Zorin 2000]
Shading gradients[Singh and Schaefer 2010]
Real-time hatching[Praun et al. 2001, Kim et al. 2008]
Page 10
Related work: hatching smooth surfaces
Artist’s illustration
Smoothed curvature directions Smoothed image gradients[Hertzmann and Zorin 2000] [Singh and Schaefer 2010]
Page 11
Related work: where do people draw lines?
Average images composed of artists’ drawings
Predicted line drawing
[Cole et al. 2008]
Page 12
Our approach
Learns a model of hatching style from a single artist’s drawing of an input shape
Page 13
Our approach
Learns a model of hatching style from a single artist’s drawing of an input shape
Can transfer the hatching style to different views of the exemplar shape as well as different shapes
Page 14
Our approach
Learns a model of hatching style from a single artist’s drawing of an input shape
Can transfer the hatching style to different views of the exemplar shape as well as different shapes
The hatching style is determined by hatching properties related to hatching tone and orientations
Page 15
Hatching properties
Hatching level
No hatchingHatchingCross‐hatching
Page 16
Hatching properties
Hatching levelStroke thickness
Page 17
Hatching properties
Hatching levelStroke thicknessStroke spacing
Page 18
Hatching properties
Hatching levelStroke thicknessStroke spacingStroke length
Page 19
Hatching properties
Hatching levelStroke thicknessStroke spacingStroke lengthStroke intensity
Page 20
Hatching properties
Hatching levelStroke thicknessStroke spacingStroke lengthStroke intensityHatching orientations
Page 21
Hatching properties
Hatching levelStroke thicknessStroke spacingStroke lengthStroke intensityHatching orientations
Artist’s illustrationComputer‐generated
illustration
Page 23
Learning stage
Thickness Spacing Intensity
Length OrientationsHatching level
Page 24
Learning stage
Shape features
Image‐space features
Thickness Spacing Intensity
Length OrientationsHatching level
Page 25
Learning stage
Hatching properties
y
Shape and image descriptors
x
Hatching properties
y
Shape and image features
x
Page 26
Learning stage
Hatching properties
y
Shape and image features
x
y = f(x)
Page 27
Linear model expressing hatching orientations as a weighted sum of selected orientation features.
Learning hatching orientations
Page 28
Linear model expressing hatching orientations as a weighted sum of selected orientation features.
Learning hatching orientations
Page 29
Learning hatching orientations
Artist’s illustration Fitting a single model across the illustration
Page 30
Learning orientation fields
Artist’s illustration
Page 31
Mixture of experts model
Simultaneous segmentation & model fitting for each segment
Page 32
Learning stroke properties
Map features to thickness, intensity, spacing, length
Page 33
Map features to thickness, intensity, spacing, length
Learning stroke properties
Page 34
Map features to thickness
Learning stroke properties
Extracted thickness Learned thickness
Page 35
Map features to intensity
Learning stroke properties
Extracted intensity Learned intensity
Page 36
Map features to spacing
Learning stroke properties
Extracted spacing Learned spacing
Page 37
Map features to length
Learning stroke properties
Extracted length Learned length
Page 38
Learning hatching level and segment labels
Map features to discrete values with Joint Boosting + CRF
Extracted hatching level Learned hatching level
No hatching
Hatching
Cross‐hatching
Page 40
Synthesis stage
Thickness Spacing Hatching level
Intensity Length Orientations
Page 41
Synthesis stage
Thickness Spacing Hatching level
Intensity Length Orientations
Page 42
Artist’s illustration
Page 43
Artist’s illustration
Page 44
Artist’s illustration
Page 45
Artist’s illustration
Page 46
Artist’s illustration
Page 47
Artist’s illustration
Page 48
Artist’s illustration
Page 49
Artist’s illustration
Page 50
Artist’s illustration
Page 51
Artist’s illustration
Page 52
Artist’s illustration
Page 53
Artist’s illustration
Page 54
Artist’s illustration
Page 55
Artist’s illustration
Page 56
Artist’s illustration
Page 57
Artist’s illustration
Page 58
Artist’s illustration
Page 59
Artist’s illustration
Page 60
Artist’s illustration
Page 61
Artist’s illustration
Page 62
Analysis of features used
Orientation features:
Page 63
Analysis of features used
Orientation features:• Principal curvatures
and local symmetry axes dominate
Page 64
Analysis of features used
Orientation features:• Principal curvatures
and local symmetry axes dominate
• Also orientations aligned with feature lines are also important
Page 65
Analysis of features usedHatching level: image intensity, shading featuresStroke thickness: shape descriptors, curvature, shading features, image gradients, location of feature lines, depthSpacing: shape descriptors, curvature, derivatives of curvature, shading featuresIntensity: shape descriptors, image intensity, shading features, depth, location of feature linesLength: shape descriptors, curvature, radial curvature, shading feature, image intensity, image gradientSegment label: shape descriptors
Page 66
Summary
• An algorithm that learns hatching styles
Page 67
Summary
• An algorithm that learns hatching styles
• Learns from a single drawing
Page 68
Summary
• An algorithm that learns hatching styles
• Learns from a single drawing
• Synthesizes hatching illustrations in the input artist’s style for novel views and shapes
Page 69
Limitations
• We do not always exactly match the artist’s illustration -aspects of hatching style are lost
Page 70
Limitations
• We do not always exactly match the artist’s illustration -aspects of hatching style are lost
• Pre-processing stage relies on thresholds to robustly extract hatching properties.
Page 71
Limitations
• We do not always exactly match the artist’s illustration -aspects of hatching style are lost
• Pre-processing stage relies on thresholds to robustly extract hatching properties.
• Computation time is large (5h-10h for training, 0.5-1h for synthesis)
Page 72
Future Work
• Analyze larger set of drawings
Page 73
Future Work
• Analyze larger set of drawings
• Extend our framework to analyze other forms of art
Page 74
Future Work
• Analyze larger set of drawings
• Extend our framework to analyze other forms of art
• Applications to field design on surfaces
Page 75
Thank you!
Acknowledgements:Seok-Hyung Bae, Patrick Coleman, Vikramaditya Dasgupta, Mark Hazen,
Thomas Hendry, Olga Vesselova, Olga Veksler, Robert Kalnins, Philip Davidson, David Bourguignon, Xiaobai Chen, Aleksey Golovinskiy,
Thomas Funkhouser, Andrea Tagliasacchi, Richard Zhang, Aim@Shape, VAKHUN, Cyberware repositories