Virtual Planning of Facial Reconstructions

Virtual Planning of Facial ReconstructionsA. Sarti1, C. Lamberti1, R. Gori2, G. Erbacci2, L. Bassani3, A. Bianchi3, C. Marchetti41Department of Electronics, Computer Science and Systems, University of Bologna, Bologna, Italy2Cineca, InterUniversity Consortium, High Performance Computing Center, Casalecchio di Reno, Bologna, Italy3Unit of Oral and Maxillofacial Surgery, S. Orsola-Malpighi-Hospital, University of Bologna, Bologna, Italy4Department of Odontostomatological Sciences, University of Bologna, Bologna, Italy

Correspondence to:Alessandro Sarti, DEISUniversity of Bologna, Viale Risorgimento 2, Bologna, ItalyTel: 0039 051 209 3091; Fax: 0039 051 209 3073; E-mail: [email protected]

Key words: Virtual surgery, facial reconstruction, maxillo-facial surgery, computer simulation, CT, MRI.

Summary

In craniofacial surgery it is not easy to predict the shape

of the postoperative face, as muscular changes resulting

from the surgery cannot be found by a simple way.

Three-dimensional (3D) computer simulation of cranio-

facial surgery can be extremely useful to foresee the

surgical outcome. Many authors proposed computer sys-

tems for craniofacial surgical planning based on compu-

ted tomographic (CT) images. A number of methods to

achieve the prediction of soft tissue behaviour have been

proposed from computer-aided surgical planning system

integrating anatomy-based 3D finite element tissue model

to methods for computation of soft-tissue deformation in

craniofacial surgery directly from CT images without any

intermediate geometric model. We present a review of

present techniques on the use of imaging in the presur-

gical planning of facial surgery and reconstruction. The

entire workflow of image acquisition, tissue segmentation,

tissue classification, surgical planning, soft tissue dis-

placement computer simulation and visualization is out-

lined and different cases of real maxillofacial surgery are

illustrated.

Introduction

In craniofacial surgery, it is important (not only for doctors

but maybe for the patients and their families) to know how

patient’s face will be changed by the surgical procedures.

In fact, any surgical procedure has both functional and

aesthetic implications that have important psychological

impact on the patient’s life (1).

However, it is not easy to predict the precise shape of the

postoperative face, as muscular changes resulting from the

surgery cannot be found by a simple way.

Predicting the behaviour of soft tissues is necessary to

address the patient’s expectations in the best way possible.

In 1985, Henderson (2) proposed photocephalometry as a

way to predict the final profile of soft tissues after

orthognathic surgery. However, photocephalometry and

other video-imaging systems are two-dimensional, and the

post-treatment soft tissue outline was added based on

accepted ratios of soft to hard tissue changes (3).

Three-dimensional (3D) computer simulation of cranio-

facial surgery can be extremely useful in clinical practice to

foresee the surgical outcome. Cutting et al. (4) described an

early method for computer-assisted design of craniofacial

surgical procedures taking into account 3D cephalometric

constraints.

Yasuda et al. (5) proposed a Computer system for

craniofacial surgical planning based on computed tomo-

graphic (CT) images to make a rough prediction of the

face shape. This function has been developed only for

relatively simple surgery of brachycephaly, as it strongly

depends on the individual operative strategy. The

operation for correction of brachycephaly includes

moving the anterior part of the skull forward to expand

the volume of the skull. The predicted postoperative face

is constructed by computer from the skull reformed

according to the selected operation plan. Altobelli et al.

(6) applied interactive repositioning using cephalometric

and anthropometric databases showing how 3D compu-

ter simulation of craniofacial surgery can be extremely

useful in clinical practice both for scientific and teaching

reasons. For modelling soft tissue deformations, a

number of models are described in literature. Mass-

spring models represent soft tissues as a collection of

point masses connected by linear or nonlinear springs in

a lattice structure (7). However, a mass tissue-spring

model cannot describe exactly the physical behaviour of

the human tissue. More accurate simulations are based

on continuum model deformation where the behaviour

of soft tissues is described as a solution of the basic

equations of continuum mechanics of matter. The

numerical solution is achieved with the help of comput-

ers by using standard numerical schemes. The most

popular in this field is the finite element technique.

Keeve et al. (8) first presented an anatomy-based 3D

1/2007 n IMAGING DECISIONS

finite element tissue model integrated into a computer-

aided surgical planning system. The model predicts soft

tissue changes resulting from the realignment of the

underlying bone structure. This approach does not

consider the exact anatomical structure of the tissue;

instead the user specifies muscles which define the

connection between the facial skin.

Sarti et al. (9) proposed a method for computation of

soft-tissue deformation in craniofacial surgery from CT

images without any intermediate geometric model (mesh).

The surgical planning system works on the fine anatomical

structure acquired by CT and it captures the elastome-

chanics of the patient soft tissues based on continuum

mechanics equations. The simulation model is biomechan-

ically based and at the same time the computation is

directly performed on the CT grid. The technique has

been fully validated on a statistically meaningful number of

cases (10) and it is now integrated in a framework regularly

used in clinical practice (Simplant CMFª; Materialise,

Leuven, Belgium). Among the several simulation environ-

ments for maxillofacial surgery by using intermediate

geometric models (mesh) and finite element approxima-

tions (11–15), the technique proposed by Zachow et al.

(13, 16) produced also accurate results and it has been

validated on a number of cases. In recent years, many

software applications (Maxilimª, Dolphin’s Treatment

Planningª) have been designed, but they generally lack

scientific precision in handling human imaging data

because they provide 3D editing graphics facilities without

considering the real behaviour of the tissues. The virtual

reality workbench was also used for surgical planning in

which the surgeon is immersed in a virtual reality

environment. Recently many softwares have been pro-

posed to achieve this goal in maxillofacial surgery (17–21).

In the next sections the entire workflow of surgical

planning and simulation is explained on the basis of the

model of physically based direct simulation on the CT

acquisition, as presented in a previous paper (10).

j Fig. 1. CT acquisition. A 3D CT scan is acquired from the patient before the surgery (a).

j Fig. 2. Patient facebeforesurgeryand preoperatory 3D reconstructedCT (a).

3 0 n V I R T U A L P L A N N I N G O F F A C I A L R E C O N S T R U C T I O N S

IMAGING DECISIONS n 1/2007

j Fig. 3. SNAP is a software appli-cation to segment structures in 3Dmedical images (b).

j Fig. 4. Julius is a general software for medical data processing and visualization (c).

j Fig. 5. Analyze is an interactivepackage for multidimensional imagevisualization, processing and analy-sis (d).

j Fig. 6. VolView is a system forvolume visualization of 3D medicalimages (e).

V I R T U A L P L A N N I N G O F F A C I A L R E C O N S T R U C T I O N S n 3 1

1/2007 n IMAGING DECISIONS

CT acquisition

A 3D CT scan is acquired from the patient before the

surgery. A high-speed spiral CT is performed in helical

mode using the following parameters: slice dis-

tance ¼ 3 mm/1.3 s, 120 Kw, 160 mA. The acquisition

protocol is an efficacious compromise between the need

to obtain the best spatial definition with the least radi-

ation, considering the young age of the patients and the

fact that crystalline is included in the study volume.

Normally data consist of 120 images approximately for

patient in the ACR-NEMA DICOM3 format (Figs 1

and 2).

Tissue segmentation, classification and 3D

reconstruction

If suitable values for CT parameters are selected, soft tis-

sues and osseous tissues will be easily separated. A set of

automatic and semi-automatic tools were developed to

process 2D real patients’ CT slices and obtain 3D volumes

necessary for surgery simulation (Figs 3–5).

j Fig. 7. 3D slicer is a software forvisualization, registration, segmenta-tion and quantification of medical data(f).

j Fig. 8. MITK (Medical ImagingInteraction Toolkit) is a medical ima-ging interaction software (g).

j Fig. 9. ASIM PACS is a softwareused to visualize and diagnose med-ical images (h).

j Fig. 10. Force feedback 3D virtual reality bistoury(GHOST�, 2001).

3 2 n V I R T U A L P L A N N I N G O F F A C I A L R E C O N S T R U C T I O N S

IMAGING DECISIONS n 1/2007

Input visualization and surgical planning

The 3D graphical interface allows for direct interaction

with the reconstructed models of hard and soft tissues. A

3D graphical interface was written in Tcl/Tk (22)

and based on the Vtk cross-platform library (Figs 6–9) (23).

Using a force feedback 3D virtual reality bistoury

(GHOST�, 2001, Sens Ablc Technologies, Inc., Woborn,

j Fig. 11. The VOXEL-MAN Tempo-Surg Simulator is a training of surgicalaccess to the middle ear (i).

j Fig. 12. Visuo-haptic environmentfor simulating a variety of surgicalprocedures (j).

j Fig. 13. Hypothesis for a newbone geometry (a).

j Fig. 14. Simplant CMF software(Materialise, Belgium): an exampleof hard tissue surgical planning (k).

V I R T U A L P L A N N I N G O F F A C I A L R E C O N S T R U C T I O N S n 3 3

1/2007 n IMAGING DECISIONS

MA 01801, USA) or simply a mouse, osteotomy lines can

be traced and anatomical regions can be moved and

relocated. Relocations were quantified in terms of trans-

lational and rotational parameters. At the end of the

surgical planning a hypothesis for a new bone geometry

was put forward (Figs 10–14).

Numerical simulation

A physically based simulation kernel computed the soft

tissue deformation caused by the new bone geometry (9,

24). The displacements of hard tissues were established at

the planning stage. The displacements of soft tissues were

modelled as classical continuous mechanic equations by

Fung (25) without distinguishing among them (Fig. 15).

According to the segmentation described above, each

kind of tissue was given a characteristic physical behaviour

represented with equations. In order to come to a

computer-assisted solution of the mathematical problem

the equation system was discretized with centred finite

difference schemes vis-a-vis the natural grid of the CT

image itself, thus avoiding regridding and mesh tuning.

j Fig. 15. CAS (Computer AssistedSurgery), an example of a clinical caseplanning in a virtual environment (l).

j Fig. 16. Preoperatory virtual simulation, postoperatory 3D reconstructed CT and patient face after surgery (a).

3 4 n V I R T U A L P L A N N I N G O F F A C I A L R E C O N S T R U C T I O N S

IMAGING DECISIONS n 1/2007

j Fig. 17. Patient face before sur-gery.

j Fig. 19. Preoperatory 3D texture.

j Fig. 18. Preoperatory 3D recon-structed CT.

V I R T U A L P L A N N I N G O F F A C I A L R E C O N S T R U C T I O N S n 3 5

1/2007 n IMAGING DECISIONS

The result of this simulation was the displacement vector

field associated with every voxel of the CT image.

Discretizing mask 3 centred differences method with a

second-order Taylor approximation for differential oper-

ators on a volumetric regular grid, the equation system

consisted of a group of finite difference equations which

can be numerically solved with the standard methods of

linear algebra.

Output visualization

The original CT image was warped following the physically

based displacement field obtained through numerical

simulation. Finally, the patient’s new facial appearance

could be visualized within the same graphical interface

(Fig. 16).

Case report

A surgical case planned with Simplant CMFª software is

presented (Figs 17–22) (http://www.materialise.com/sim-

plant/cmf_ENG.html).

Conclusions

We presented a review of present techniques on the use of

imaging in the presurgical planning of facial surgery and

reconstruction. In particular, the entire workflow of image

acquisition, tissue segmentation, tissue classification, sur-

gical planning, soft tissue displacement computer simula-

tion and visualization has been outlined and different cases

of real maxillofacial surgery have been illustrated. How-

ever, the field of virtual planning and computer-based

j Fig. 20. Preoperatory virtual simu-lation.

j Fig. 21. Preoperatory virtual simulation with 3D texture.

3 6 n V I R T U A L P L A N N I N G O F F A C I A L R E C O N S T R U C T I O N S

IMAGING DECISIONS n 1/2007

simulations is in continuous development and new inter-

esting research studies are coming out for a faster and more

reliable prediction of the outcome of the maxillofacial

operation, particularly regarding the introduction of non-

linear and viscoelastic models of tissue displacement. The

interdisciplinary cooperation among radiology depart-

ments, maxillofacial surgeons and bioengineers will be at

the base of the success in facing this important challenge in

improving health care.

Acknowledgements

A particular thanks to Dr Stefan Zachow for fruit-

ful discussion. We acknowledge also the following

web resources: http://www-bio.deis.unibo.it/Bioimaging/

VISUproject/Index.htm; http://www.itksnap.org/; http://

www.julius.caesar.de/index.php/Gallery; http://www.

mayo.edu/bir/Software/Analyze/Analyze.html; http://

www.volview.com/; http://www.slicer.org/; http://mbi.

dkfz-heidelberg.de/mitk/screenshots.html; http://www.

3dmedvis.com/3drose.html; http://www.voxel-man.de/

simulator/temposurg/; http://www.techhouse.org/�dmorris/

projects/bonesim/#app2; http://www.materialise.com/

simplant/cmf_ENG.html; http://www.zib.de/visual/projects/

cas/caslong.en.html

References

1. Flanary C. The psychology of appearance and psychological impact

of surgical alteration of the face. In: Bell WH (ed.) Modern Practice in

Orthognathic and Reconstructive Surgery. Saunders, Philadelphia,

1992; 2Y21.

2. Henderson D. A Colour Atlas and Textbook of Orthognathic Sur-

gery. The Surgery of Facial Skeletal Deformity. Wolfe Medical

Publications Ltd, London, 1985.

3. Sinclair PM, Kilpelainen P, Phillips C, et al. The accuracy of video

imaging in orthognathic surgery. Am J Orthod Dentofacial Orthop

1995; 107: 177Y185.

4. Cutting C, Bookstein FL, Grayson B, Fellingham L, McCarthy JG.

Three-dimensional computer-assisted design of craniofacial surgical

procedures: optimization and interaction with cephalometric and CT-

based models. Plast Reconstr Surg 1986; 77: 877–885.

5. Yasuda T, Hashimoto Y, Yokoi S, Toriwaki JI. Computer system for

craniofacial surgical planning based on CT images. IEEE Trans Med

Imaging 1990; 9: 270–280.

6. Altobelli DE, Kikinis R, Mulliken JB, et al. Computed-assisted three-

dimensional planning in craniofacial surgery. Plast Reconstr Surg

1993; 92: 576Y585.

7. Koch RM, Gross MH, Carls FR et al. Simulating facial surgery using

finite element models. ACM Siggraph 1996; 00: 421–428.

8. Keeve E, Girod S, Pfeifle P, Girod B. Anatomy based facial tissue

modelling using the finite element method. In: Proceedings of IEEE

Visualization, San Francisco, CA, USA, 1996; 21–28.

9. Sarti A, Gori R, Lamberti C. A physically based model to simulate

maxillofacial surgery from 3D CT images. Future Generation Com-

put Syst 1999; 15: 217–221.

10. Marchetti C, Bianchi A, Bassi M, Gori R, Lamberti C, Sarti A.

Mathematical modeling and numerical simulation in maxillo-facial

virtual surgery (VISU). J Craniofac Surg 2006; 17: 661–667.

11. Schutyser F, Van Cleynenbreugel J, Schoenaers J, Marchal G,

Suetens P. A simulation environment for maxillofacial surgery inclu-

ding soft tissue implications. In: Taylor C and Colchester A (eds)

Medical Image Computing and Computer-Assisted Intervention, no.

1679. Lecture Notes in Computer Science, Springer-Verlag, New

York, 1999; 1210–1217.

12. Teschner M. Direct computation of soft-tissue deformation in

craniofacial surgery simulation. PhD thesis, Shaker Verlag, Aachen,

Germany, ISBN 3-8265-8317-5, Jan 2001.

13. Zachow S, Gladilin E, Hege HC, Deuflhard P. Finite-element simu-

lation of soft tissue deformation. In: Herausgegeben von Lemke HU,

Vannier MW, Inamura K, Farman AG (eds) Computer Assisted

Radiology and Surgery. Elsevier Science B.V., San Francisco, CA,

2000; 23–28.

14. Barre S, Fernandez C, Paume P, Subrenat G. Simulating facial sur-

gery. In: Visualization, Display, and Image Guided Procedures. Proc.

SPIE 2000; 3960, 334–345.

15. Bettega G, Payan Y, Mollard B et al. A simulator for maxillofacial

surgery integrating 3D cephalometry and orthodontia. Comput Aided

Surg 2000; 5: 156–165.

16. Zachow S, Weiser M, Hege HC, Deuflhard P. Soft tissue prediction in

computer assisted maxillofacial surgery planning: a quantitative

evaluation of histomechanical modeling using pre- and postoperative

j Fig. 22. Patient face after surgery.

V I R T U A L P L A N N I N G O F F A C I A L R E C O N S T R U C T I O N S n 3 7

1/2007 n IMAGING DECISIONS

CT-data. In: Payan Y(ed.) Biomechanics Applied to Computer

Assisted Surgery, Chapter 17. Research Signpost Publisher, Kerala,

India, 2005; 277–298, ISBN: 81-308-0031-4.

17. Chabanas M, Luboz V, Payan Y. Patient specific finite element model

of the face soft tissues for computer-assisted maxillofacial surgery.

Med Image Anal 2003; 7: 131–151.

18. Meehan M, Teschner M, Girod S. Three-dimensional simulation and

prediction of craniofacial surgery. Orthod Craniofac Res 2003;

6(Suppl. 1): 102–107.

19. Westermark A, Zachow S, Eppley BL. Three-dimensional osteotomy

planning in maxillofacial surgery including soft tissue prediction.

J Craniofac Surg 2005; 16: 100–104.

20. Xia J, Ip HH, Samman N et al. Three-dimensional virtual-reality

surgical planning and soft-tissue prediction for orthognathic surgery.

IEEE Trans Inf Technol Biomed 2001; 5: 97–107.

21. Troulis M.J., Everett P., Seldin E.B., Kikinis R., Kaban L.B. Devel-

opment of a three-dimensional treatment planning system based on

computed tomographic data. Int J Oral Maxillofac Surg 2002; 31:

349–357.

22. Welch BB. Practical Programming in Tcl and Tk. Prentice-Hall, New

York, 1999.

23. Schroeder W, Martin K, Lorensen B. The Visualization Toolkit: An

Object-Oriented Approach to 3D Graphics. Kitware, Inc. Publishers,

New York, 2003.

24. Sarti A, Gori R, Bianchi A, Marchetti C, Lamberti C. Maxillo-facial

virtual surgery from 3D CT images. In: Akay M, Marsh A (eds)

Information Technologies in Medicine, Vol. 2, Rehabilitation and

Treatment, pagg, IEEE EMBS Series. Springer-Verlag, New York,

2001.

25. Fung, YC. Biomechanics: Mechanical Properties of Living Tissues.

Springer-Verlag, New York, 1993.

3 8 n V I R T U A L P L A N N I N G O F F A C I A L R E C O N S T R U C T I O N S

IMAGING DECISIONS n 1/2007