Orientable Textures for Image-Based Pen-And-Ink Illustration Michael P. Salisbury Michael T. Wong John F. Hughes David A. Salesin SIGGRAPH 1997 Andrea Rowan January 23, 2001
Orientable Textures for Image-Based Pen-And-Ink Illustration
Michael P. Salisbury
Michael T. Wong
John F. Hughes
David A. Salesin
SIGGRAPH 1997
Andrea Rowan
January 23, 2001
Outline
• Introduction to Image-Based Rendering• Difference Image Algorithm
– Foundations– Interactive System– Rendering
• Results• Problems• Future Work
Introduction
Geometry-Based Systems• Reads in 3-D geometry of
scene
• Slow for complex objects
• Faster for walkthrough of scene / manipulation of objects
Image-Based Systems• Reads in a 2-D grayscale
image
• Same speed for complex objects
• Slower for walkthrough of scene / manipulation of objects
vertices: 11159faces: 13352v -383.118 595.707 143.442 0.351793 0.169915 0.920527v -376.05 577.648 143.533 0.294138 0.14277 0.94504v -359.736 601.004 135.433 0.235886 0.183209 0.954354v -404.41 671.841 143.49 0.757324 0.31507 0.572006v -405.197 671.996 146.182 0.655538 -0.71832 0.232993v -427.467 652.795 149.643 0.123845 -0.135311 0.983033v -388.729 670.429 94.5347 0.537561 0.830288 0.147142
This paper’s algorithm usesan image-based system!
Algorithm - Foundations
• Target (Tone) Image– Defines the tones at every point in the grayscale
image– 0.0 (white) to 1.0 (black)
0.0
1.01.0
1.00.5
0.3
Algorithm - Foundations
• Direction Field– Defines the desired orientation of strokes at
each region of the illustration
Algorithm - Foundations
• Stroke Example Set– Set of strokes that will be used to fill in tone
areas– One stroke randomly chosen from the set each
time
Interactive System
• Editing tone - user can lighten or darken reference image
Interactive System
• Editing direction - user can modify the direction image– comb - changes “direction” to match motion of
cursor– blending tool - smooth between regions of
different direction– region-filling tools
Source tool Constant Direction Fill Interpolated Fill
Interactive System
• Applying stroke – Vertical vector of stroke sample matches
direction vector in direction image– Strokes placed dynamically (extra strokes for
diverging field, strokes bent with direction)
Direction image Static strokes Dynamic strokes
Rendering
• Previous Steps were User-controlled
• Rendering is entirely automated
• importance - the fraction of darkness that has not been added to a section of the image.– Density of strokes is directly related to darkness
in tone image.
Rendering - Steps
• Making illustration match tone image– Illustration is b/w, and tone image is grayscale– Divide screen space into regions
• Size of region depends on color in tone image (Larger regions for lighter areas of tone image)
Regions Varying RegionSize
Constant RegionSize
Rendering - Steps
• Making illustration match tone image– When adding a stroke, add blurred stroke to
region– Difference image =
tone image value
- blurred version of illustration value
– Importance image =
current difference value/initial difference value
Rendering - Steps
Example:
Consider a pixel with initial tone value of 0.2, initial illustration value (as for all regions) is 0.
At the start:
difference image = tone - blurred illustration = 0.2 - 0.0 = 0.2
importance image = current difference/initial difference =
0.2/0.2 = 1. Want this to approach zero or some min. threshold
Add a stroke, which blurred, adds .15 to the value
difference image = 0.2 - 0.15 = 0.05
importance image = .05/.2 = 0.025
Importance value decreases with each stroke.
Rendering - Steps
• Drawing next Stroke in the Right Place– When a stroke is drawn, pixels in the area of the
stroke lose their importance– Quadtree keeps track of most important pixel or
region of pixels– Next stroke is drawn at most important pixel
Rendering - Steps
• When to Stop the Illustration– Importance values get closer to zero with each
stroke– Exact match of zero is difficult– Illustration stops when some minimum importance
value is attained for each pixel
Rendering - Approximations
• Assumption 1: Blurred version of multiple strokes is same as sum of blurred versions of independent strokes
difference image = tone - blurred illustration
OR
difference image = tone - (blurred illustrationold + blurred strokenew)
difference image = difference imageold - blurred strokenew
Rendering - Approximations• Assumption 1: Cont’d
– Depends on strokes not overlapping– Points where strokes cross will be counted as darkened twice – Illustration is in black & white, so two strokes crossed is the
same darkness as one stroke– Solution is hacked with lightening factor stored in a
“darkness-look-up-table” – Example:
• If region is 50% gray (0.5), 90% of pixels drawn are visible
• Reduce darkness of blurred strokes to 90% before adding blurred value to illustration
Rendering - Approximations
• Assumption 2: Simplified filtered image of stroke for computations– Render control hull as blurry line– Width = 2h/t
• h = stroke thickness (mm)• t = desired tone value (0.0=white to 1.0=black)
• Clamped from 1 to 10 mm.
Drawing a Stroke
• Orienting and Bending– Control hull - frame around each stroke– Broken into parts and mapped to direction image
Pi
Pi+1
Pi+2
Pi+3
Stroke Direction image Illustration
Pi
Pi+1
Pi+2
Pi+3
Drawing a Stroke
• Clipping– When direction field changes rapidly– When stroke crosses into a region that is already
dark enough
Output Enhancements
• Variable Width– Pen and ink pressure can vary from start of stroke
to end– Each stroke has 3 widths
• Start width
• Middle width
• End width
– Different ratios if drawing hair vs. shadows
Output Enhancements
• “Wiggles”– Artists don’t draw with rulers!– For strokes > 5 mm– Add control points to control hull– Random perturbation within range (-0.15 to 0.15
mm)
Results
• Stroke density adjusts for different sized images
Results
Performance (SGI 180MHz R500 processor)
Object Strokes Time (s)Books 16722 258Teapot small 2924 50Vectors 665 25Raccoon 55893 960
Other Work• Michael Kowalski et al., “Art-Based Rendering of Fur, Grass,
and Trees,” SIGGRAPH 1999
• Use off-screen grayscale rendering of scene as reference image
• Convert 2-D screen position to 3-D space for interactive geometry-based system
Accomplishments
• Textures appear attached to objects (this is difficult in image-based rendering)
• Good algorithm for stroke density in image-based rendering
Problems
• High degree of user-tweaking (direction image)– Direct user interaction with illustration causes
performance hit
• Poor performance (every pixel is looked at)
Future Work
• User interaction with pen-and-ink illustration, not direction image
• Support of coherent textures (bricks, fabrics, etc.)