Forensic facial reconstruction using computer modelling software

Davy, S.L., Gilbert, T., Schofield, D. and Evison, M.P. (2005). Forensic facial reconstruction using computer modelling software. In Clement, J.G. and Marks, M.K. (Eds.), Computer-Graphic Facial Reconstruction, New York: Elsevier Academic Press, pp 183-196. ISBN 9780124730519 (Print), 9780080454221 (Online) This text is the Accepted Manuscript only. The final volume can be found here.

Forensic Facial Reconstruction using Computer Modeling Software

Stephanie L. Davy1, Timothy Gilbert2, Damian Schofield3, Martin P. Evison1

1University of Sheffield

The Medico-Legal Centre

Department of Forensic

Pathology

Watery Street

Sheffield, S3 7ES, UK

2Aims Solutions Ltd.

PO Box 6345

Nottingham, NG7 2XN, UK

3University of Nottingham

School of ComputerScience & IT

University of Nottingham

University Park

Nottingham, NG7 2RD, UK

Contact: Steph Davy, [email protected], +44 (0)793 942 9983/+ 44 (0)114 273 2791

Currently, there is no single answer to the many challenges facing forensic facial

reconstruction. The process of completing a three-dimensional clay reconstruction can take

several days to complete. With the advent of user-friendly computer software and

methods, the time taken to produce a facial reconstruction process could potentially be

reduced to mere hours. As computer technology progresses and develops, computer

generated facial reconstruction techniques will improve. These developments could save

both time and money, as well as increasing the reliability of the technique.

Existing technologies such as 3-D scanners and digital cameras can be utilized to capture

the geometry of a skull, but the technologies to reconstruct the skull have yet to obtain

similar speed and efficiency improvements. The digital capture of a skull has several

benefits. Firstly, the need for casting of a skull is eliminated, reducing both the opportunity

Davy, S.L., Gilbert, T., Schofield, D. and Evison, M.P. (2005). Forensic facial reconstruction using computer modelling software. In Clement, J.G. and Marks, M.K. (Eds.), Computer-Graphic Facial Reconstruction, New York: Elsevier Academic Press, pp 183-196. ISBN 9780124730519 (Print), 9780080454221 (Online)

http://store.elsevier.com/Computer-Graphic-Facial-Reconstruction/John-Clement/isbn-9780080454221/

mailto:[email protected]

Davy et al, 2

for damage to an original specimen, as well as the cost of materials and time spent in a

laboratory. As with most similar computer technology, computer-generated reconstructions

can be more easily altered than can a clay version. With keystrokes or mouse clicks,

features may be altered, added, or removed altogether. When working on a physical skull,

there is no “undo” button.

Another benefit of computerized forensic facial reconstructions is that of reproducibility. If

twenty practitioners were given the same face to manually reconstruct from the same skull,

twenty different reconstructions are likely to result. This point was illustrated in the Green

River serial killer cases, in which multiple facial reconstructions of several victims were

created. The results were highly variable from practitioner to practitioner and met with

little success (Haglund and Reay, 1991). However, with a computerized program

constructed from the same data, using the same techniques, each practitioner should

produce the same basic reconstruction. Ideally, the process would work for confirming a

practitioner’s use of tissue depths and facial features much in the same way as the

FORDISC software package (Ousley and Jantz, 1996) works for confirming (rather than

determining) sex and ancestral affiliation. The software would suggest appropriate depths

and features, but the practitioner would have the ability to override them or adjust them to

accommodate for case-specific needs. Computer verification increases the accuracy and

objectivity of a reconstruction. In most cases, the only variations between practitioners

would be in the details that have not yet been fully scientifically established, such as ear

shape or lip shape (however, there are recognized formulae which provide guidelines).


Davy et al, 3

Since the time involved in completing a reconstruction will be greatly reduced with a

satisfactory method of computerization, so will the costs. Billable hours can be reduced,

and the only capital investment would be in the price of the software and an ordinary

desktop computer system. This would make the practice more accessible and available to

places that need the service most for cases of unidentifiable victims (i.e. typically under-

funded police departments and medical examiners offices). It is, however, recommended

that such departments enlist the aid of a forensic anthropologist or otherwise qualified

consultant to increase the chances of successful identification.

Additionally, computerized facial reconstructions could be implemented in situations

where traditional facial reconstructions cannot. Typically, forensic facial reconstruction

has not been employed in cases of mass graves or mass disasters due to time and cost

constraints associated with plasticine methods; computer-generated facial reconstructions

could eliminate both of these constraints. Forensic facial reconstruction could be a very

powerful tool in future mass human identification scenarios.

CURRENT LIMITATIONS

There are several limitations, both in currently available computer technology and in

reliable data, which prevent the development of a workable automated software product for

computerized facial reconstruction. While computer technology is advancing at an

astounding rate, there are still several avenues that prevent realistic looking reconstructions


Davy et al, 4

from being easily and rapidly produced. One of the problems is that the tissue-depth data

upon which reconstructions are based is seen as unreliable, and is the subject of ongoing

research.

At present, realistic skin and hair modeling within 3D modeling software packages is very

time consuming and far from ideal. Using programs such as FaceGen (Singular Inversions,

Inc., 2002) one can use fairly realistic-looking skin textures, but such a texture loses its

realism once stretched over a different skull. Issues associated with skin textures will be

discussed further later in this chapter. Also, specialist software for modelling realistic

human hair

Other limitations to skin modeling include age-related features, such as wrinkles.

Appropriate wrinkle modeling relies upon an interface of several disciplines, including

computer programming as well as human biology and psychology. It is important that not

only do wrinkles appear realistic in an aesthetic sense, but also that they are anatomically

appropriate. Individuals age at different rates as well as by developing wrinkles in different

areas. Such considerations must be made in order to ensure that the reconstructed

individual is perceived to be of a suitable age by potential identifying witnesses. Several

studies have been conducted regarding the perception of age in faces (Evison, 2001). This

has important implications for future work in computerized facial reconstruction. Burt and

Perret (1995) also found that shape and color may have an impact upon the interpretation

of age.


Davy et al, 5

WORK WITH 3-D MODELING SOFTWARE

In a joint project between the University of Sheffield Forensic Anthropology teams and

Aims Solutions Ltd., a University of Nottingham spin-out company, the authors have been

able to develop new techniques for computerized facial reconstruction and create several

reconstructions. The authors were able to undertake this project due a generous grant from

the Higher Education Innovation Fund (HEIF).

3ds max™ version 5 (Discreet™, 2002) is a program designed for computer modeling and

animation. It provides a high level of flexibility and a substantial number of varied

functions from which to choose. This seemed an ideal program for a project of this type,

particularly since the development team had extensive experience of using this software.

Since this small research project was experimental, it was possible to easily change the

approach as the methodology was developed.

The Egyptology Department at the Bolton Museum (United Kingdom) enlisted the authors

to attempt the computerized reconstruction of the face of a mummy using only

radiographs. The mummy was an unknown male, aged between 20 and 35 years. He had

placed in the coffin of a female mummy, apparently because grave robbers prefer to sell

mummies and sarcophagi in sets. Although the identity of the mummy is unknown, it was

known that the body was found in an area used for the burial of priests and royalty.


Davy et al, 6

Figure 1: The laser-scanned skull with x-rays

This facial reconstruction was successfully completed and is described in this chapter. Two

additional cases (one archaeological and one forensic) were also completed using the

techniques developed during this project.

Capturing the Skull

In a “normal” facial reconstruction, the practitioner generally has the skull (or fragments

thereof) in his/her physical possession. However, in this particular project only frontal and

lateral radiographs were available for use. The authors decided that the most efficient

solution to this problem was to use an existing skull from the lab that was of a similar

sex/age/ethnic affiliation and “morph” it to match the given radiographs. The skull was

scanned using a Cyberware 3030 color laser scanner, Echo software was then used to

create a polygonal model.


Davy et al, 7

Figure 2: The skull after smoothing

The skull model was converted into an editable mesh, which converts the surface into

editable polygons (Figure 1). Then, the radiographs were imported into the software as

images and set up as textures on solid 3D geometry so that they were visible in the lateral

and frontal views. The skull was then lined up with the radiographs and the polygons were

manipulated to match the shape of the radiograph in perpendicular views (Figure 2). 3ds

max™ allows the viewer to see the “world” in which s/he is working from multiple angles

and in layers, which enabled the authors to easily ensure that all vertices making up the


Davy et al, 8

virtual skull were correctly located. Forensic cases are often more straightforward because

the actual skull is usually available and the geometry produced by the 3D scanner can be

directly imported into the 3ds max™ software.

Placing the Landmarks

Small pyramids were used to represent the traditional tissue depth markers utilized in clay

reconstructions (Figure 3). The square

bases were placed perpendicularly to

the bone surface at the appropriate

craniometric points on the skull. The

height of the pyramids was input in

millimeters to the measurements

specified in the literature (Rhine &

Campbell, 1980). For clarity, each

pyramid was individually renamed for

the point it represented (i.e. nasion-

right or glabella-left). For the regions

in which tissue depth data was

lacking, additional pyramids were

created in a second color (for

differentiation purposes) at mathematically calculated intermediary points. (this is

repeated overleaf)

Figure 3: The skull with landmarks. Red

denotes a height taken from literature, whereas

blue denotes an interpolated value.


Davy et al, 9

To calculate the size of the interpolated points, the spaces between the existing landmarks

were broken down into a “grid” covering the entire surface, and the mid-point of any two

locations on the same line of this grid added as a marker of an averaged height. For

example, the gonions and the mental eminence were deemed to be on the same horizontal

contour as each other. Hence, an interpolated point was added on the bottom edge of the

mandible halfway between the two points and the height taken as an average of the two.

As this distance was relatively large in comparison to others, additional points were added

between this new point and the gonion, and between the point and the mental eminence.

Again, averaging the points on either side provided the heights.

Although this method gives approximate values, the authors believe that it is of

comparable accuracy to the existing method of adding clay strips to join the landmarks

together. It could be postulated that the accuracy may be improved by taking into account

the heights from more of the surrounding landmarks. There are a variety of mathematical

methods that have been used to interpolate the size of intermediary landmark sites for

cranio-facial reconstruction (Albrecht et al, 2003; Attardi et al, 2001; Cairns,

1999).However, the authors believe that the discrepancy between the different point

heights calculated using these different mathematical techniques is minimal in camparison

with the assuptions made during the reconstruction process. Figure 3 shows the final

result of this process, complete with landmarks for the eyes, nose and mouth. The creation

of these parts is discussed later.


Davy et al, 10

Initially, the computer work was a time consuming process. Development of a suitable

technique for the placement of the landmarks took several days for the animator to

accomplish under the guidance of the forensic anthropologist. However, the time taken for

this process was greatly reduced in subsequent cases because the pyramids could be saved

independently of the skull. Each group of landmark data, such as Caucasian females or

Negroid males, was saved individually with the appropriate measurements. These can now

be imported into future cases as a “cloud” of craniometric landmark points that are in the

approximately correct location, but already have the correct height data applied. These can

then be spatially adjusted until they are in the correct craniometric positions.

Creating the eyes

Creating the eyeballs was possibly one of the simplest of the reconstruction tasks. Two

spheres were created to the dimensions recommended in the literature. Using a wire frame

polygon view, portions of the spheres could be selected and appropriately colored using

the materials editor function of 3ds max™ to create convincing textures for the pupils and

irises. The whites of the eyes were slightly more difficult because they are not a true white

in living subjects, so a gradient texture was used to redden the eyes towards the lids and

corners. The eyeballs were then positioned in the sockets using the data detailed in

research by Stephan (2002). The protrusion was carefully measured using the tape measure

feature provided in 3D ds max.


Davy et al, 11

Forming the nose

The size of the nose was determined by using calculations based on Macho’s research

(1986). The software dimensioning features were then used to create the proper dimensions

and depth markers were placed in the appropriate areas. Previously created, generic nose

geometry was merged into the scene and non-uniformly scaled to match the calculated

sizes, and appropriate landmarks . This method was also used for the other facial features.

Forming the Tissue

SPLINES AND SURFACE MODIFIERS

In the early stages of the project, the authors decided to utilize splines to connect the tissue

depth markers. Splines are lines that can be interpolated and curved. They were used to

provide a base surface over which a skin layer could be fitted. A number of techniques

were attempted to construct the “spline cage”, but this proved frustrating on several fronts.

First, the process was very time consuming and labor intensive, requiring each spline to be

connected continuously over the tissue depth markers. The final curvature of the face was

ultimately dependant upon the layout of the splines; the process became tedious as sections

of the head were built and re-built to find the most effective structure. After the skin layer

was applied using the 3ds max™ surface modifier function (a tool used to patch together a

3D polygonal surface (mesh) based on the contours of a spline network) it was evident that

the number of interpolated points used led to a face that had a relatively low polygon

count. The resulting reconstruction was deemed jagged and rough, especially in the cheek

regions.


Davy et al, 12

In this case the problem was solved by smoothing the skin by applying the mesh smooth

modifier tool in 3ds max™. In the future this problem could be overcome by adding more

interpolated points, thereby creating a more detailed network of splines. This would

obviously lengthen an already time-consuming process if carried out by hand, but the

development of a semi-automated system to create the soft tissue could alleviate this. The

development of such a system is seen by the authors as a non-trivial problem since the way

the landmark sites are linked can alter the look and feelof a reconstruction.

ANATOMICAL RECONSTRUCTION AND NURBS


Davy et al, 13

The authors were interested in creating a

more advanced method for reconstruction that

incorporated the anatomically-based method

favored by many physical reconstructionists

(Prag and Neave, 1997; Gerasimov, 1971).

This method particularly wanted to examine

the use of the underlying facial musculature

in the modelling of the skin on the facial model. This was accomplished by the application

of NURBS (Non-Uniform Rational B-Splines). NURBS CV-curves (Control Vertex-

curves) are similar to splines in that they are curves that can be manipulated, but in this

case each section of the curve is the average line between three weighted control points

(Figure 4), whereas a spline is made up of vertices that are positioned on the line itself

with individual orientations and weightings. The lofted surfaces generated from NURBS

CV-curves are generally smoother than the spline networks used earlier and have the added

advantage that they can be manipulated quickly by altering the position of the CV-curves.

This makes the creation of complex shapes relatively simple to achieve, and the resulting

muscles can be transferred to new skulls and fitted with a minimum of effort.

Figure 4: The basic construction of a

NURBS CV-Curve


Davy et al, 14

NURBS CV-curves were used to create elliptical cross-sections, which were then lofted

into surfaces replicating the gross

musculature of the head (Figure

5). The cross sections were

correlated with established

diagrams of craniofacial muscles,

along with the tissue depth

landmarks, to indicate the depth of

the soft tissue over the surface of

the skull.

The contours of the face were then

built using more CV-curves,

following the shape of the

underlying muscular tissue structure, then lofted into an approximate reconstruction of the

subject’s face. The resulting mesh was carefully altered to provide added definition to

details such as the eyes and the lips, and any imperfections in the surface corrected. This

method undoubtedly created a more contoured and aesthetically pleasing face, although it

was more time-consuming than the earlier methods developed. While the muscle “texture”

was irrelevant to the final reconstruction, a pinkish color and striated texture was added to

the muscles to give them a more realistic appearance.

Figure 5: The skull with NURBS muscles


Davy et al, 15

How about adding an image of the wire frame skin mesh here – before texturing ?

TEXTURING THE FACE

One advantage that computer-generated facial reconstruction has over traditional methods

is its ability to texture the faces realistically. In3D computer modelling, texturing is a

process akin to applying wallpaper to a flat surface, where a pattern or image is draped

over the solid object. A similar technique was also possible with a clay head, but it

required a fine-artist to paint directly onto the model. With 3ds max™ it is possible to

drag-and-drop a texture onto the generated face with a minimum of effort.

In addition to the standard XYZ cartesian coordinates, each vertex also has a UVW

coordinate, which corresponds to its texture as opposed to its position in virtual space.

Once a texture is applied to a mesh, a UVW wrap modifier is added to roughly fit the face

onto the mesh (for example, applying the UVW wrap in the form of a cylinder would be

similar to stretching a rubber tube with the face painted on over a clay head; the features

might not align properly, but it gives a starting point for finer adjustment).

Once the face is visible on the head model it can then be fine-tuned using the UVW

Unwrap modifier feature in 3ds max™. This gives a two-dimensional representation of the

UVW coordinates overlaid onto the face texture. The vertices can be moved in these two

dimensions to fit the texture onto the head correctly. This stage can again be quite time-

consuming – especially if the head is complex and has a large number of vertices.


Davy et al, 16

However, once the process is complete the software remembers the individual UVW

coordinates and new textures can be quickly applied onto the head. This is of great value

when examining the effects of aging and ethnicity on a face. A texture designed to look

older or darker, for example, can be quickly applied and its effect on the “look” of the face

evaluated.

For the face shown throughout this chapter, a texture generated by the FaceGen Modeller

software package was used (op cit.), as this software was already in use in other areas and

provides good-quality textures manipulated by the adjustment of a range of facial variables

and parameters. The texture could be produced using a wide variety of methods, including

digital photographs of a real face or hand-painted artwork. The exact approach taken

would depend upon the scenario, but it was felt that the FaceGen software could speed up

this part of the process considerably and was suitable for use in this case.

RESULTS & FUTURE WORK

The face of the Egyptian mummy generated using this technique is shown in figure 6. As

can be seen in figure 6, there is a pronounced overbite on the upper lip. Initially, it was

thought that this could be due to the lower mandible having sunk into the skull over the

ages. However, this hypothesis was refuted by a forensic odontologist who confirmed that

the position of the mandible was correct. This overbite on the upper lip particularly excited


Davy et al, 17

the Egyptologists, as the royal family of the pharaoh dynasty from the time the mummy

originated are renowned for their pronounced overbite.

The reconstructions produced during this project were fairly impressive; however, there is

still work that needs to be done to make the process simpler and more automated and the

final images more lifelike. The reconstructed visages look computer generated and are

often evocative of video game characters, but this can be rectified over time with the use of

better skin textures and increased model complexity (an increased number of polygons).

Also, the gradual increase in automation will allow the artist to produce models to a much

higher level of detail than currently possible in a reasonable timeframe, which will only

improve the realism of the reconstructed faces. Additionally, we hope to incorporate

ongoing work in fields such as realistic hair modeling to improve the appearance of the

heads, as well as the appearance of

complicated skin features like the fine

tissues around the eyes.

The processes discussed here were

adapted from the existing physical

methods. There were several

advantages of using a digital medium,

such as the ability to work from x-ray

data, the improved use of musculature

Figure 6: The completed face after mesh

smoothing.


Davy et al, 18

in the reconstruction and the application of different textures. However, the main areas in

which computerization is expected to improve reconstruction techniques (automation, for

example) still require improvement. This is not to say that this study was not valuable;

attempting to implement the traditional methods using modeling software has given the

authors an insight into the unique advantages and disadvantages of working with a virtual

medium. It has also generated ideas of ways to improve and semi-automate the process.

The authors believe that this can be accomplished without falling into the trap of using

‘generic’ faces that fail to convey the individual characteristics of a face.

As for what automation can achieve, it is felt that the process of manually placing the

landmark sites is something that will (for the foreseeable future at least) need to be carried

out by hand. It may be possible to use a script to generate the interpolated points. The

actual building of the face, spline by spline, is something that currently is best

accomplished through human intervention. In the future, a process in which a pre-built

virtual wireframe is snapped onto landmarks, which are then placed by hand may provide a

rapid solution. The authors believe that the manual positioning of the points would ensure

that the nuances of the skull are taken into account, but the automated ‘cage’ would save

time.

3ds max™ has many advanced features may yet be applied to this process. As the process

is developed further, it is our hope to create a range of tissue depth templates that can be

adjusted to fit individual cases. Also, the flexibility of the software allows a continued


Davy et al, 19

update of the data as more research is conducted in areas of craniofacial identification such

as tissue depths and soft tissue/bony feature relationships.

COMPARISON TO OTHER COMPUTERIZATION METHODS

Other researchers within the field of facial reconstruction have been working to create

methods of computerized facial reconstruction for several years. The method discussed in

this chapter, will hopefully be a cost-effective and less time-consuming alternative when

finalized. Rather than to simply recreate all of the steps used in a clay reconstruction using

a computer, it is the intent of the authors to create the most reliable, speedy, and accurate

reconstructions as possible while utilizing technology.

We have opted not to use photographic facial templates/donor faces and morph them

directly onto the skull (Vanesis et al., 2000; Jones, 2001; Tu et al., 2000) in favor of a

developing method that can potentially compensate for a fuller range of individual

features. By using facial templates there is a potential for the production of reconstructions

that look more like the donor face(s) than the deceased’s actual premortem face. The

resulting image is often a composite of the donor faces. The work discussed in this chapter

attempts to avoid this pitfall by using the skull to dictate facial appearance, rather than

existing face templates being anchored to the skull. While some may argue that the use of

computer graphics may sacrifice realism, a counter argument is that often a caricature of a

person is more easily recognizable than a photo-image that may resemble a generic, albeit

realistic, template-based face. Additionally, there is no reason to assume that a computer


Davy et al, 20

reconstruction should be any less accurate or realistic than a clay model when similar

techniques are used. Discrepancies in realism between clay and computerized

reconstructions will continue to be an issue while the fields of forensic facial

reconstruction and computer graphics are regarded as separate. As reconstruction experts

gain experience with graphics packages and graphics experts gain experience with

reconstruction, the quality of the end products can only improve.

THE FUTURE

It is the intention of the authors to continue work using 3ds max™ and other 3D computer

modelling software to incorporate further research on tissue and feature information as it

becomes available. Future plans include the reconstruction of test cases using forensic

cases with accompanying premortem photographs. These cases will be reconstructed

blindly, meaning that the practitioners will not have access to the premortem information.

This will test the reliability of the method as well as to lend information about techniques

that may be improved upon in the future.

References

Albrecht, L., Haber, J., Seidel, H.P. (2003) Construction and Animation of Anatomically

Based Human Hand Models, Proc. ACM SIGGRAPH Symposium on Computer

Animation (SCA) 2003.

Attardi, G., Betrò, M., Forte, M., Gori, R., Guidazzoli, A., Imboden, S., Mallegni, F.

(1999) 3D Facial Reconstruction and Visualization of Ancient Egyptians Mummies Using

Spiral CT Data, ACM SIGGRAPH 99, Sketches and Applications.


Davy et al, 21

Burt, D.M. & Perrett, D.I. (1995) Perception of Age in Adult Caucasian Male Faces:

Computer Graphic Manipulation of Shape and Colour Information. Proceedings of the

Royal Society of London. Vol. 259, pp. 137-143.

Cairns, M., (1999) An Investigation Into The Use of 3D Computer Graphics For Forensic

Facial Reconstruction, First Year Ph.D. Internal Report, Department of Computer Science,

University of Glasgow.

Discreet Software (2002) 3ds max™. CD-ROM [June, 2003].

Evison, M.P. (2001) Modeling Age, Obesity and Ethnicity in a Computerized 3-D Facial

Reconstruction. Forensic Science Communications. 3(2).

George, R.M. (1993) Anatomical and Artistic Guidelines for Forensic Facial

Reconstruction. IN: İşcan, M.Y., Helmer R.P., eds. Forensic Analysis of the Skull. New

York, Wiley-Liss, pp. 215-227.

Gerasimov, M.M. (1971) The Face Finder. Hutchinson & Co, London.

Haglund, W.D. & Reay, D.T. (1991) Use of Facial Approximation Techniques in

Identification of Green River Serial Murder Victims. American Journal of Forensic

Medicine and Pathology, 12(2), pp.132-142.

Macho, G.A. (1986) An Appraisal of Plastic Reconstruction of the Nose. Journal of

Forensic Sciences. 31(4), October, pp. 1391-1403.

Ousley, S.D. and Jantz, R.L. (1996) FORDISC 2.0 Personal Computer Forensic

Discriminant Functions. University of Tennessee, Knoxville.

Prag, J. & Neave, R. (1997) Making Faces Using Forensic and Archaeological Evidence.

Texas A&M University Press, College Station.

Rhine & Campbell (1980) Thicknes of Facial Tissues in American Blacks. Journal of

Forensic Sciences 25(4) pp. 847-858.

Singular Inversions, Inc. (2002) FaceGen Modeller CD-ROM [June 2003].

Stephan, C.N. (2002) Facial Approximation: Globe Projection Guideline Falsified by

Exophthalmometry Literature. Journal of Forensic Sciences. July 47(4) pp. 730-735.


Forensic facial reconstruction using computer modelling software

Documents