Surface Modeling of Impasto Paintings

Surface Modeling of Impasto Paintings
Leon Demetriou
June 2010
Abstract
Computer generated art has come a long way since the inception of graphics back in the 1960s. Non-photorealistic rendering techniques have managed to simulate the look and feel of natural media such as paints and canvases to a great extent. Despite this, it is still very difficult to get the impasto touch of real paintings. This occurs when paint is laid onto the canvas in thick strokes forming a 3D texture. Without such texture information digital art lacks the required elements for a completely realistic rendition. The aim of this project is to enhance an existing tool developed at Imperial College with a viewer that is capable of taking depth information into account to produce a three- dimensional model of the painting surface. Once we have built a surface model of the paint we are able to experiment with numerous lighting models. To further enhance the experience for users of the system, we have incorporated head tracking using ordinary web-cameras to simulate changing lighting conditions as a person alters their position relative to the canvas. Our proposed solution can be extended for use in digital galleries for displaying computer generated art.
2
Acknowledgments
Firstly I would like to thank my supervisor, Dr. Simon Colton for giving me the oppor- tunity to work on such an interesting project together with all his help and constructive feedback along the way. I would also like to acknowledge my second supervisor, Dr Iain Phillips for his advice. Finally, I want to thank my friends and family for their continuous support over the years during my studies at Imperial College.
3
Contents
1 Introduction 7 1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.2 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.3 Report Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2 Background 11 2.1 Surface Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.1.1 2.5D Data Structures . . . . . . . . . . . . . . . . . . . . . . . . 11 2.1.2 Polygonal Models . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.1.3 Free-Form Surfaces . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.1.4 Texture Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.2 Shading Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.2.1 Phong Shading . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.2.2 Gouraud Shading . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.2.3 Flat Shading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 2.2.4 Cook-Torrance Shading . . . . . . . . . . . . . . . . . . . . . . . 21
2.3 Global Illumination Models . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.3.1 Ray Tracing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.3.2 Radiosity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.4 Viola-Jones Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.5 Edge Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 2.6 Software APIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.6.1 Java 3D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 2.6.2 OpenGL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 2.6.3 GLSL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 2.6.4 OpenCV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.7 NPR Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 2.7.1 ArtRage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 2.7.2 Fast Paint Texture . . . . . . . . . . . . . . . . . . . . . . . . . . 32 2.7.3 IMPaSTo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 2.7.4 The Painting Fool . . . . . . . . . . . . . . . . . . . . . . . . . . 33 2.7.5 Wet & Sticky . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
3.1.1 Internal Representation . . . . . . . . . . . . . . . . . . . . . . . 37 3.1.2 Stroke Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.2 Shading Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 3.2.1 Illumination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.3 Motion Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 3.3.1 Mouse Coordinates Tracking . . . . . . . . . . . . . . . . . . . . 43 3.3.2 Head Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.4 User Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 3.5 Programming Considerations . . . . . . . . . . . . . . . . . . . . . . . . 46 3.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
4 Implementation 48 4.1 System Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 4.2 Front End GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
4.2.1 Filter Panels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 4.2.2 Multi-Threading . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 4.2.3 Additional Features . . . . . . . . . . . . . . . . . . . . . . . . . 52
4.3 Paint Surface Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 4.3.1 Image Filter Framework . . . . . . . . . . . . . . . . . . . . . . . 52 4.3.2 Height Composition . . . . . . . . . . . . . . . . . . . . . . . . . 55
4.4 Lighting Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 4.4.1 Smooth Shader . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 4.4.2 Phong Shader . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
4.5 Viewer Motion Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 4.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
5 Experimental Design 64 5.1 Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
5.1.1 Computer Hardware . . . . . . . . . . . . . . . . . . . . . . . . . 64 5.1.2 Web Cameras . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 5.1.3 Monitors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
5.2 Performance Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 5.3 User Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
5.3.1 Questionnaire . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 5.3.2 Nielsen’s Heuristics . . . . . . . . . . . . . . . . . . . . . . . . . . 68
5.4 Robustness & Stability Testing . . . . . . . . . . . . . . . . . . . . . . . 68 5.4.1 Monkey Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 5.4.2 Stress Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 5.4.3 Motion Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
5
Contents
5.5 Aesthetics Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 5.6 Platform Compatibility . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 5.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
6 Results and Analysis 74 6.1 Performance Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
6.1.1 Filter Performance . . . . . . . . . . . . . . . . . . . . . . . . . . 74 6.1.2 Rendering Performance . . . . . . . . . . . . . . . . . . . . . . . 76
6.2 User Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 6.2.1 Questionnaire . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 6.2.2 Nielsen’s Usability Heuristics . . . . . . . . . . . . . . . . . . . . 78
6.3 Robustness & Stability Testing . . . . . . . . . . . . . . . . . . . . . . . 80 6.3.1 Monkey Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 6.3.2 Stress Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 6.3.3 Motion Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
6.4 Aesthetics Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 6.5 Platform Compatibility . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 6.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
7 Conclusion and Future Work 87 7.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 7.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 7.3 Closing Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
8 Appendix 92
1 Introduction
1.1 Motivation
Simulation of art materials and artistic styles is a very important part of NPR1. This is an area of computer graphics that focuses on using a range of expressive and creative ideas for use in digital art [23, pages 3-10]. Photorealistic computer graphics, on the other hand, tend to be involved with algorithms that aim to reproduce reality as far as possible. In many cases they attempt to reproduce physical laws to match the exact output of the camera.
During ancient times, the Egyptians used to portray people on papyrus or linen, in a manner that deviated from reality. Given this freedom, they were able to convey more powerful messages since their drawings contained more information for study. Even now, medical and technical diagrams enhance certain aspects of an image in order to highlight features that are more important within that context. NPR techniques have a number of goals ranging from artificial intelligence to enhancing legibility.
We are interested in taking digital art produced from a range of rendering systems and adding further texture information to enhance its appearance and level of realism. Although we aim to produce surface models of the canvas that are a close match…