Non-Photorealistic Rendering: Cross Hatching

Final Master Thesis Master of Innovation and Research in Informatics
Computer Graphics and Virtual Reality
Non-Photorealistic Rendering: Cross Hatching
Barcelona School of Informatics (FIB) of Polytechnic University Of Catalonia (UPC)
with support from
Abstract
Non-photorealistic rendering can be used to create abstracted images of real- life objects. Cross-hatching is a common style under the non-photorealistic rendering umbrella, where characteristics such as lighting and shape of objects are represented by layered patches of parallel lines.
The work in this master thesis explores the creation of cross-hatched ren- ders where lighting and shape can be implied by density and direction of the hatching lines. We choose to place the hatching lines following the curvature characteristics of the object. For this purpose, we describe a way to compute and interpolate principal curvature directions and an image-space algorithm to draw lines following such directions.
1
Contents
2.1.1 Stylization . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.1.2 Non-Photorealistic Lighting . . . . . . . . . . . . . . . . . 7 2.1.3 Outlines and Silhouette detection . . . . . . . . . . . . . . 8
2.2 Hatching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3 Overview 12 3.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 3.2 Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
4 Hatching direction field 14 4.1 Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 4.2 Identifying Principal Directions . . . . . . . . . . . . . . . . . . . 15
4.2.1 Numerical Approximation . . . . . . . . . . . . . . . . . . 15 4.3 Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
4.3.1 Smoothing the cross field . . . . . . . . . . . . . . . . . . 17 4.3.2 Error Analysis . . . . . . . . . . . . . . . . . . . . . . . . 18
5 Rendering 21 5.1 Line Drawing Algorithm . . . . . . . . . . . . . . . . . . . . . . . 21
5.1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 5.1.2 Input Image . . . . . . . . . . . . . . . . . . . . . . . . . . 21 5.1.3 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . 24 5.1.4 Line Integration . . . . . . . . . . . . . . . . . . . . . . . 25 5.1.5 Selecting hatching direction . . . . . . . . . . . . . . . . . 25 5.1.6 Seed Selection . . . . . . . . . . . . . . . . . . . . . . . . 26 5.1.7 Point validation . . . . . . . . . . . . . . . . . . . . . . . . 27
5.2 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 5.3 Unity integration . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
6 Results 30 6.1 Rendering examples . . . . . . . . . . . . . . . . . . . . . . . . . 30 6.2 Rendering times . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
7 Conclusions and Future Work 33
8 References 36
List of Figures
1 Examples of hatching. . . . . . . . . . . . . . . . . . . . . . . . . 4 2 Examples of NPR styles. . . . . . . . . . . . . . . . . . . . . . . . 6 3 Results from [EM17]. . . . . . . . . . . . . . . . . . . . . . . . . . 7 4 Examples of lookup texture use. . . . . . . . . . . . . . . . . . . 8 5 Hatching textures applied to sphere from [Pra+01]. . . . . . . . . 9 6 Resulting render from [Sal+97], preceded by user generated tone
and direction maps and stroke samples. . . . . . . . . . . . . . . 10 7 Detail from result from [GI13]. . . . . . . . . . . . . . . . . . . . 10 8 Jiffycrew Hatching example. . . . . . . . . . . . . . . . . . . . . . 11 9 Side-by-side results from hatching methods. . . . . . . . . . . . . 12 10 Normal curvature illustrated in plane section [Log]. . . . . . . . . 14 11 Neighbourhood of P. . . . . . . . . . . . . . . . . . . . . . . . . . 15 12 Smoothing results and artifacts. . . . . . . . . . . . . . . . . . . . 17 13 Torus highlighting circles of radius R1(pink) and R2(red) [Wik]. . 18 14 Histogram of error in principal direction approximation (1). . . 19 15 Histogram of error in principal direction approximation (2). . . 20 16 Line drawing scheme. . . . . . . . . . . . . . . . . . . . . . . . . . 21 17 Cross hatching result. . . . . . . . . . . . . . . . . . . . . . . . . 22 18 Input image and its channel breakdown. . . . . . . . . . . . . . . 23 19 Effects of WhiteOffset parameter. . . . . . . . . . . . . . . . . . . 23 20 Portion of field drawn with different dTests. . . . . . . . . . . . . 25 21 Example of long spiral. . . . . . . . . . . . . . . . . . . . . . . . . 25 22 Seed selection scheme. . . . . . . . . . . . . . . . . . . . . . . . . 26 23 Example of inconsistent directions given by high angle tolerances. 27 24 Different outline results. . . . . . . . . . . . . . . . . . . . . . . . 29 25 Final results on torus and teapot. . . . . . . . . . . . . . . . . . . 30 26 Final results on monkey head. . . . . . . . . . . . . . . . . . . . . 31 27 Hatching rendering in low poly tree model. . . . . . . . . . . . . 33
3
1 Introduction
Non-Photorealistic rendering (NPR) is an area of computer science that seeks to render digital images mimicking the abstraction normally associated with artist representations. There are many different styles to simulate, and the reasons for adapting the trappings of a style go beyond aesthetic preference.
Different art styles can be used to emphasize or under play some character- istics of the subject, which changes the how and what information is conveyed in the image. These decisions are normally made by an artist, therefore, in computer graphics, the challenge is not only to copy visual styles but to imitate decisions that would be made by a human.
The applications for NPR are as diverse as the styles themselves. They can be seen in cinema, games, and illustrations, to name a few. In some fields, such as technical illustration, the information needed to be conveyed is so well suited for NPR styles that visual realism can be counter productive [Puj11] (Figure 2d).
This work focuses on the implementation of one particular NPR style, hatch- ing.
Figure 1: Examples of hatching. (top) Part of the exhibition “Prehensility”, by Olivia Knapp. (left) Detail of Queens of Persia at the Feet of Alexander. Etching and engraving. [Hic18]
(right) Hatching Sketch [Kem20]
Hatching is a drawing technique that is characterized by patches of parallel lines. The direction and density of the patches can be used to imply different characteristics of the shape, such as tone, lighting, form, texture or movement.
4
If the hatching is composed of multiple crossing layers, it can also be called cross-hatching. This technique is commonly used with medium that does not allow for smooth color transitions, such as engraving and pen and ink.
Hatching lines can be drawn in several styles, from high detail etching to sketching. The density of the hatching is commonly used to imply tone or shadow, while the direction of the lines can be used in many different ways. It is common to have the lines follow the shape of the object, at least partially, as they can be a useful tool to efficiently imply shape. [PK19]
In this thesis we seek to develop a method of creating such images. We focus in particular in the method to orient the hatching lines, looking to create a picture similar to Figure 1. We have chosen to develop the process in Unity, thinking of exploring the potential of its rendering engine and the possibility of utilizing the style as part of the visual style in games.
5
2 Previous Work
2.1 Non-Photorealistic Rendering
There is a huge body of work exploring NPR. Despite the variety of topics that can be covered within this label, there are some recurring features worth exploring.
(a) Team fortress 2 [MFE07] (b) Legend of Zelda: The Wind Waker[Gam]
(c) Hatchig result from [Lak+00] (d) Results from [GSC99]
Figure 2: Examples of NPR styles.
2.1.1 Stylization
A number of papers and commercial solutions look into the creation of digital images with characteristic of specific artistic styles. It is hard to make definitive list of artistic styles and their characteristics, as each author will create their
6
own variations. In computer graphics, stylization may refer not only to the types of marks used to create the image but to the characteristics of the media used.
Some stylization can be achieved by applying image filtering algorithms di- rectly to a photograph or render, exemplified in [DLJ14], where the authors rely on the saliency map of an image to produce a Chinese ink painting version of the image.
Another example is [DS00], where the authors mimic pen and ink styles. They discuss the use of silhouettes and outlines, with some use of hatching, to render trees as if drawn by pen and ink.
Machine learning algorithms can also be used. The work in this field can be extremely versatile, as show in [EM17], where the authors present an improved version of an algorithm capable of adapting and transfer artistic styles, learned from one arbitrary image, to another (Figure 3).
Figure 3: Results from [EM17]. In order, the target image, the style source, and the resulting image.
2.1.2 Non-Photorealistic Lighting
Lighting is often an important aspect of an image. Besides conveying the scene’s light situation, it can be essential for conveying shape information. As such, NPR often includes some kind of lighting abstraction.
Most NPR lighting is still based in traditional lighting models. In [MFE07], for example, the authors discuss the contribution of a Lambert, Phong, Ambient and Rim light terms for the lighting model in the style of Team Fortress 2 (Figure 2a).
A common feature added to NPR models is cell shading (Figure 2b). These styles consist of using one or more color bands to represent shadows and high- lights. The goal is to eliminate the soft transitions between lit and shaded areas. This can be done by setting a threshold and setting all shades bellow it to dark and all shades above to bright. This approach can be adapted to include extra banding levels by setting multiple thresholds.
Alternatively, a lookup texture can be used. Normally a 1D texture is used, where the single dimension represents the brightness level. By manipulating the values at each lighting level it is easy to achieve the hard transitions, while keeping some anti-aliasing. This technique is described in [Lak+00] (Figure 2d).
7
Moreover, lookup textures are a common feature in many different NPR lighting models, and can be used to achieve a variety of effects. It offers better control over the light level transitions and the option to add a tint to light and dark areas. They can be used only as a lighting tint, as shown in [Tec], or to generate more subtle cell-shading-like effects, like in [MFE07]. In [BTM06] the authors expand on the idea and show how to use a 2D lookup-texture for the same purpose, with the second dimension being mapped to depth. Examples of lookup textures are shown in Figure 4.
(a) 1D lookup texture from [Tec] (b) 2D lookup texture from [BTM06]
Figure 4: Examples of lookup texture use.
A similar concept is to use the a set of textures instead of a single lookup texture. The level of light is used to decide which one of these textures will be sampled. Some hatching styles can be achieved with this method, with [Lak+00] showing a few examples.
2.1.3 Outlines and Silhouette detection
Outlines are a common aspect of many NPR rendering styles and can be one of the most effective ways of conveying shape (Figure 2d). In [Goo01] a few existing methods are discussed. One would be to use the difference in image space depth and normal to detect silhouette and internal outlines.
A second approach relies on the fact that at silhouette points the dot product of the normal and view direction will be 0. In [HZ00] this idea is expanded upon to create smooth outlines that compliment their hatching algorithm (Figure 9a).
Implementations based on both of these approaches are discussed further in Section 5.2.
In [CM02] a geometry based approach is introduced. The vertex renders the edges of the mesh as extra triangles when they are considered to be part of the outline. The condition for an edge to be considered an outline is that, for adjacent each faces, the dot product of the normal in camera space with the view vector have opposite signals. Occluded outlines are culled by the z test.
A common issue with these renderings is that the triangle mesh is often not a close representation of the smooth surface it is meant to depict, and this discrepancy becomes most clear at the edges, which are enhanced by the outlines. For this reason, outlines in low poly models can clearly display the lack of resolution. This issue can be avoided by increasing the resolution of the mesh, but this may not be achievable in some situations.
8
2.2 Hatching
In terms of hatching and cross hatching specifically, there are several different approaches, at times focusing on different aspects of the style.
The simplest implementation is to use a set of textures representing the different levels of hatching and use the light information to select which would be used. This idea can be expanded by using the surface characteristics to align the hatching textures. That is what is done in [Pra+01], where texture patches are placed and oriented such that the lines of the hatching textures are aligned with the principal directions of the objects. Once the texture patches are generated and properly oriented, the technique can easily run in real time. This paper also introduces the creation of custom mip-maps for the hatching textures presented, such that the size and distance of the hatches remains consistent regardless of the mesh’s distance from the camera. This allows for more complex scenes to be hatched more effectively. However, though there is great effort to mask the seams of the texture patches, the transitions and uv distortions still create visible artifacts (Figure 5).
Figure 5: Hatching textures applied to sphere from [Pra+01].
Several other works also suggest drawing the hatching lines following cur- vature information. A few defend the creation of a direction field based on principal directions accompanied by an integration method used to draw each hatching line following this field. Generally, a smoothing step is used to re- move noise and align the directions. That is the case in both [HZ00] (Figure 9a) and [Zan+04], the later also introducing different ways to style the lines drawn, such as shading and tapering. This method produces good looking re- sults, but requires that the hatching lines be re-drawn for each frame, making it too expensive for real time rendering.
Alternatively, the hatching lines can be drawn using the parametrization of the surface as a guide, as shown in [Win+96]. This method produces results on pair with the methods relying on principal curvatures, but introduces huge limitations, as it relies on a global parametrization of the models.
In [Sal+97], the direction of the hatches is drawn following a direction field, however, in this case the field must be defined by the user (Figure 6). They provide a tool that helps create and manipulate the flow of this field. The lines themselves are based on textures that can generate different hatching styles. This also creates great looking results with some versatility, but is more labor intensive and unsuitable for real time rendering, as the direction fields would
9
have to be created per frame.
Figure 6: Resulting render from [Sal+97], preceded by user generated tone and direction maps and stroke samples.
Other strategies include [SBB17], where the hatching orientations and style is driven solely by the…