Virtual Planning of Facial ReconstructionsA. Sarti1, C. Lamberti1, R. Gori2, G. Erbacci2, L. Bassani3, A. Bianchi3, C. Marchetti41Department of Electronics, Computer Science and Systems, University of Bologna, Bologna, Italy2Cineca, InterUniversity Consortium, High Performance Computing Center, Casalecchio di Reno, Bologna, Italy3Unit of Oral and Maxillofacial Surgery, S. Orsola-Malpighi-Hospital, University of Bologna, Bologna, Italy4Department of Odontostomatological Sciences, University of Bologna, Bologna, Italy
Correspondence to:Alessandro Sarti, DEISUniversity of Bologna, Viale Risorgimento 2, Bologna, ItalyTel: 0039 051 209 3091; Fax: 0039 051 209 3073; E-mail: [email protected]
Key words: Virtual surgery, facial reconstruction, maxillo-facial surgery, computer simulation, CT, MRI.
Summary
In craniofacial surgery it is not easy to predict the shape
of the postoperative face, as muscular changes resulting
from the surgery cannot be found by a simple way.
Three-dimensional (3D) computer simulation of cranio-
facial surgery can be extremely useful to foresee the
surgical outcome. Many authors proposed computer sys-
tems for craniofacial surgical planning based on compu-
ted tomographic (CT) images. A number of methods to
achieve the prediction of soft tissue behaviour have been
proposed from computer-aided surgical planning system
integrating anatomy-based 3D finite element tissue model
to methods for computation of soft-tissue deformation in
craniofacial surgery directly from CT images without any
intermediate geometric model. We present a review of
present techniques on the use of imaging in the presur-
gical planning of facial surgery and reconstruction. The
entire workflow of image acquisition, tissue segmentation,
tissue classification, surgical planning, soft tissue dis-
placement computer simulation and visualization is out-
lined and different cases of real maxillofacial surgery are
illustrated.
Introduction
In craniofacial surgery, it is important (not only for doctors
but maybe for the patients and their families) to know how
patient’s face will be changed by the surgical procedures.
In fact, any surgical procedure has both functional and
aesthetic implications that have important psychological
impact on the patient’s life (1).
However, it is not easy to predict the precise shape of the
postoperative face, as muscular changes resulting from the
surgery cannot be found by a simple way.
Predicting the behaviour of soft tissues is necessary to
address the patient’s expectations in the best way possible.
In 1985, Henderson (2) proposed photocephalometry as a
way to predict the final profile of soft tissues after
orthognathic surgery. However, photocephalometry and
other video-imaging systems are two-dimensional, and the
post-treatment soft tissue outline was added based on
accepted ratios of soft to hard tissue changes (3).
Three-dimensional (3D) computer simulation of cranio-
facial surgery can be extremely useful in clinical practice to
foresee the surgical outcome. Cutting et al. (4) described an
early method for computer-assisted design of craniofacial
surgical procedures taking into account 3D cephalometric
constraints.
Yasuda et al. (5) proposed a Computer system for
craniofacial surgical planning based on computed tomo-
graphic (CT) images to make a rough prediction of the
face shape. This function has been developed only for
relatively simple surgery of brachycephaly, as it strongly
depends on the individual operative strategy. The
operation for correction of brachycephaly includes
moving the anterior part of the skull forward to expand
the volume of the skull. The predicted postoperative face
is constructed by computer from the skull reformed
according to the selected operation plan. Altobelli et al.
(6) applied interactive repositioning using cephalometric
and anthropometric databases showing how 3D compu-
ter simulation of craniofacial surgery can be extremely
useful in clinical practice both for scientific and teaching
reasons. For modelling soft tissue deformations, a
number of models are described in literature. Mass-
spring models represent soft tissues as a collection of
point masses connected by linear or nonlinear springs in
a lattice structure (7). However, a mass tissue-spring
model cannot describe exactly the physical behaviour of
the human tissue. More accurate simulations are based
on continuum model deformation where the behaviour
of soft tissues is described as a solution of the basic
equations of continuum mechanics of matter. The
numerical solution is achieved with the help of comput-
ers by using standard numerical schemes. The most
popular in this field is the finite element technique.
Keeve et al. (8) first presented an anatomy-based 3D
1/2007 n IMAGING DECISIONS
finite element tissue model integrated into a computer-
aided surgical planning system. The model predicts soft
tissue changes resulting from the realignment of the
underlying bone structure. This approach does not
consider the exact anatomical structure of the tissue;
instead the user specifies muscles which define the
connection between the facial skin.
Sarti et al. (9) proposed a method for computation of
soft-tissue deformation in craniofacial surgery from CT
images without any intermediate geometric model (mesh).
The surgical planning system works on the fine anatomical
structure acquired by CT and it captures the elastome-
chanics of the patient soft tissues based on continuum
mechanics equations. The simulation model is biomechan-
ically based and at the same time the computation is
directly performed on the CT grid. The technique has
been fully validated on a statistically meaningful number of
cases (10) and it is now integrated in a framework regularly
used in clinical practice (Simplant CMFª; Materialise,
Leuven, Belgium). Among the several simulation environ-
ments for maxillofacial surgery by using intermediate
geometric models (mesh) and finite element approxima-
tions (11–15), the technique proposed by Zachow et al.
(13, 16) produced also accurate results and it has been
validated on a number of cases. In recent years, many
software applications (Maxilimª, Dolphin’s Treatment
Planningª) have been designed, but they generally lack
scientific precision in handling human imaging data
because they provide 3D editing graphics facilities without
considering the real behaviour of the tissues. The virtual
reality workbench was also used for surgical planning in
which the surgeon is immersed in a virtual reality
environment. Recently many softwares have been pro-
posed to achieve this goal in maxillofacial surgery (17–21).
In the next sections the entire workflow of surgical
planning and simulation is explained on the basis of the
model of physically based direct simulation on the CT
acquisition, as presented in a previous paper (10).
j Fig. 1. CT acquisition. A 3D CT scan is acquired from the patient before the surgery (a).
j Fig. 2. Patient facebeforesurgeryand preoperatory 3D reconstructedCT (a).
3 0 n V I R T U A L P L A N N I N G O F F A C I A L R E C O N S T R U C T I O N S
IMAGING DECISIONS n 1/2007
j Fig. 3. SNAP is a software appli-cation to segment structures in 3Dmedical images (b).
j Fig. 4. Julius is a general software for medical data processing and visualization (c).
j Fig. 5. Analyze is an interactivepackage for multidimensional imagevisualization, processing and analy-sis (d).
j Fig. 6. VolView is a system forvolume visualization of 3D medicalimages (e).
V I R T U A L P L A N N I N G O F F A C I A L R E C O N S T R U C T I O N S n 3 1
1/2007 n IMAGING DECISIONS
CT acquisition
A 3D CT scan is acquired from the patient before the
surgery. A high-speed spiral CT is performed in helical
mode using the following parameters: slice dis-
tance ¼ 3 mm/1.3 s, 120 Kw, 160 mA. The acquisition
protocol is an efficacious compromise between the need
to obtain the best spatial definition with the least radi-
ation, considering the young age of the patients and the
fact that crystalline is included in the study volume.
Normally data consist of 120 images approximately for
patient in the ACR-NEMA DICOM3 format (Figs 1
and 2).
Tissue segmentation, classification and 3D
reconstruction
If suitable values for CT parameters are selected, soft tis-
sues and osseous tissues will be easily separated. A set of
automatic and semi-automatic tools were developed to
process 2D real patients’ CT slices and obtain 3D volumes
necessary for surgery simulation (Figs 3–5).
j Fig. 7. 3D slicer is a software forvisualization, registration, segmenta-tion and quantification of medical data(f).
j Fig. 8. MITK (Medical ImagingInteraction Toolkit) is a medical ima-ging interaction software (g).
j Fig. 9. ASIM PACS is a softwareused to visualize and diagnose med-ical images (h).
j Fig. 10. Force feedback 3D virtual reality bistoury(GHOST�, 2001).
3 2 n V I R T U A L P L A N N I N G O F F A C I A L R E C O N S T R U C T I O N S
IMAGING DECISIONS n 1/2007
Input visualization and surgical planning
The 3D graphical interface allows for direct interaction
with the reconstructed models of hard and soft tissues. A
3D graphical interface was written in Tcl/Tk (22)
and based on the Vtk cross-platform library (Figs 6–9) (23).
Using a force feedback 3D virtual reality bistoury
(GHOST�, 2001, Sens Ablc Technologies, Inc., Woborn,
j Fig. 11. The VOXEL-MAN Tempo-Surg Simulator is a training of surgicalaccess to the middle ear (i).
j Fig. 12. Visuo-haptic environmentfor simulating a variety of surgicalprocedures (j).
j Fig. 13. Hypothesis for a newbone geometry (a).
j Fig. 14. Simplant CMF software(Materialise, Belgium): an exampleof hard tissue surgical planning (k).
V I R T U A L P L A N N I N G O F F A C I A L R E C O N S T R U C T I O N S n 3 3
1/2007 n IMAGING DECISIONS
MA 01801, USA) or simply a mouse, osteotomy lines can
be traced and anatomical regions can be moved and
relocated. Relocations were quantified in terms of trans-
lational and rotational parameters. At the end of the
surgical planning a hypothesis for a new bone geometry
was put forward (Figs 10–14).
Numerical simulation
A physically based simulation kernel computed the soft
tissue deformation caused by the new bone geometry (9,
24). The displacements of hard tissues were established at
the planning stage. The displacements of soft tissues were
modelled as classical continuous mechanic equations by
Fung (25) without distinguishing among them (Fig. 15).
According to the segmentation described above, each
kind of tissue was given a characteristic physical behaviour
represented with equations. In order to come to a
computer-assisted solution of the mathematical problem
the equation system was discretized with centred finite
difference schemes vis-a-vis the natural grid of the CT
image itself, thus avoiding regridding and mesh tuning.
j Fig. 15. CAS (Computer AssistedSurgery), an example of a clinical caseplanning in a virtual environment (l).
j Fig. 16. Preoperatory virtual simulation, postoperatory 3D reconstructed CT and patient face after surgery (a).
3 4 n V I R T U A L P L A N N I N G O F F A C I A L R E C O N S T R U C T I O N S
IMAGING DECISIONS n 1/2007
j Fig. 17. Patient face before sur-gery.
j Fig. 19. Preoperatory 3D texture.
j Fig. 18. Preoperatory 3D recon-structed CT.
V I R T U A L P L A N N I N G O F F A C I A L R E C O N S T R U C T I O N S n 3 5
1/2007 n IMAGING DECISIONS
The result of this simulation was the displacement vector
field associated with every voxel of the CT image.
Discretizing mask 3 centred differences method with a
second-order Taylor approximation for differential oper-
ators on a volumetric regular grid, the equation system
consisted of a group of finite difference equations which
can be numerically solved with the standard methods of
linear algebra.
Output visualization
The original CT image was warped following the physically
based displacement field obtained through numerical
simulation. Finally, the patient’s new facial appearance
could be visualized within the same graphical interface
(Fig. 16).
Case report
A surgical case planned with Simplant CMFª software is
presented (Figs 17–22) (http://www.materialise.com/sim-
plant/cmf_ENG.html).
Conclusions
We presented a review of present techniques on the use of
imaging in the presurgical planning of facial surgery and
reconstruction. In particular, the entire workflow of image
acquisition, tissue segmentation, tissue classification, sur-
gical planning, soft tissue displacement computer simula-
tion and visualization has been outlined and different cases
of real maxillofacial surgery have been illustrated. How-
ever, the field of virtual planning and computer-based
j Fig. 20. Preoperatory virtual simu-lation.
j Fig. 21. Preoperatory virtual simulation with 3D texture.
3 6 n V I R T U A L P L A N N I N G O F F A C I A L R E C O N S T R U C T I O N S
IMAGING DECISIONS n 1/2007
simulations is in continuous development and new inter-
esting research studies are coming out for a faster and more
reliable prediction of the outcome of the maxillofacial
operation, particularly regarding the introduction of non-
linear and viscoelastic models of tissue displacement. The
interdisciplinary cooperation among radiology depart-
ments, maxillofacial surgeons and bioengineers will be at
the base of the success in facing this important challenge in
improving health care.
Acknowledgements
A particular thanks to Dr Stefan Zachow for fruit-
ful discussion. We acknowledge also the following
web resources: http://www-bio.deis.unibo.it/Bioimaging/
VISUproject/Index.htm; http://www.itksnap.org/; http://
www.julius.caesar.de/index.php/Gallery; http://www.
mayo.edu/bir/Software/Analyze/Analyze.html; http://
www.volview.com/; http://www.slicer.org/; http://mbi.
dkfz-heidelberg.de/mitk/screenshots.html; http://www.
3dmedvis.com/3drose.html; http://www.voxel-man.de/
simulator/temposurg/; http://www.techhouse.org/�dmorris/
projects/bonesim/#app2; http://www.materialise.com/
simplant/cmf_ENG.html; http://www.zib.de/visual/projects/
cas/caslong.en.html
References
1. Flanary C. The psychology of appearance and psychological impact
of surgical alteration of the face. In: Bell WH (ed.) Modern Practice in
Orthognathic and Reconstructive Surgery. Saunders, Philadelphia,
1992; 2Y21.
2. Henderson D. A Colour Atlas and Textbook of Orthognathic Sur-
gery. The Surgery of Facial Skeletal Deformity. Wolfe Medical
Publications Ltd, London, 1985.
3. Sinclair PM, Kilpelainen P, Phillips C, et al. The accuracy of video
imaging in orthognathic surgery. Am J Orthod Dentofacial Orthop
1995; 107: 177Y185.
4. Cutting C, Bookstein FL, Grayson B, Fellingham L, McCarthy JG.
Three-dimensional computer-assisted design of craniofacial surgical
procedures: optimization and interaction with cephalometric and CT-
based models. Plast Reconstr Surg 1986; 77: 877–885.
5. Yasuda T, Hashimoto Y, Yokoi S, Toriwaki JI. Computer system for
craniofacial surgical planning based on CT images. IEEE Trans Med
Imaging 1990; 9: 270–280.
6. Altobelli DE, Kikinis R, Mulliken JB, et al. Computed-assisted three-
dimensional planning in craniofacial surgery. Plast Reconstr Surg
1993; 92: 576Y585.
7. Koch RM, Gross MH, Carls FR et al. Simulating facial surgery using
finite element models. ACM Siggraph 1996; 00: 421–428.
8. Keeve E, Girod S, Pfeifle P, Girod B. Anatomy based facial tissue
modelling using the finite element method. In: Proceedings of IEEE
Visualization, San Francisco, CA, USA, 1996; 21–28.
9. Sarti A, Gori R, Lamberti C. A physically based model to simulate
maxillofacial surgery from 3D CT images. Future Generation Com-
put Syst 1999; 15: 217–221.
10. Marchetti C, Bianchi A, Bassi M, Gori R, Lamberti C, Sarti A.
Mathematical modeling and numerical simulation in maxillo-facial
virtual surgery (VISU). J Craniofac Surg 2006; 17: 661–667.
11. Schutyser F, Van Cleynenbreugel J, Schoenaers J, Marchal G,
Suetens P. A simulation environment for maxillofacial surgery inclu-
ding soft tissue implications. In: Taylor C and Colchester A (eds)
Medical Image Computing and Computer-Assisted Intervention, no.
1679. Lecture Notes in Computer Science, Springer-Verlag, New
York, 1999; 1210–1217.
12. Teschner M. Direct computation of soft-tissue deformation in
craniofacial surgery simulation. PhD thesis, Shaker Verlag, Aachen,
Germany, ISBN 3-8265-8317-5, Jan 2001.
13. Zachow S, Gladilin E, Hege HC, Deuflhard P. Finite-element simu-
lation of soft tissue deformation. In: Herausgegeben von Lemke HU,
Vannier MW, Inamura K, Farman AG (eds) Computer Assisted
Radiology and Surgery. Elsevier Science B.V., San Francisco, CA,
2000; 23–28.
14. Barre S, Fernandez C, Paume P, Subrenat G. Simulating facial sur-
gery. In: Visualization, Display, and Image Guided Procedures. Proc.
SPIE 2000; 3960, 334–345.
15. Bettega G, Payan Y, Mollard B et al. A simulator for maxillofacial
surgery integrating 3D cephalometry and orthodontia. Comput Aided
Surg 2000; 5: 156–165.
16. Zachow S, Weiser M, Hege HC, Deuflhard P. Soft tissue prediction in
computer assisted maxillofacial surgery planning: a quantitative
evaluation of histomechanical modeling using pre- and postoperative
j Fig. 22. Patient face after surgery.
V I R T U A L P L A N N I N G O F F A C I A L R E C O N S T R U C T I O N S n 3 7
1/2007 n IMAGING DECISIONS
CT-data. In: Payan Y(ed.) Biomechanics Applied to Computer
Assisted Surgery, Chapter 17. Research Signpost Publisher, Kerala,
India, 2005; 277–298, ISBN: 81-308-0031-4.
17. Chabanas M, Luboz V, Payan Y. Patient specific finite element model
of the face soft tissues for computer-assisted maxillofacial surgery.
Med Image Anal 2003; 7: 131–151.
18. Meehan M, Teschner M, Girod S. Three-dimensional simulation and
prediction of craniofacial surgery. Orthod Craniofac Res 2003;
6(Suppl. 1): 102–107.
19. Westermark A, Zachow S, Eppley BL. Three-dimensional osteotomy
planning in maxillofacial surgery including soft tissue prediction.
J Craniofac Surg 2005; 16: 100–104.
20. Xia J, Ip HH, Samman N et al. Three-dimensional virtual-reality
surgical planning and soft-tissue prediction for orthognathic surgery.
IEEE Trans Inf Technol Biomed 2001; 5: 97–107.
21. Troulis M.J., Everett P., Seldin E.B., Kikinis R., Kaban L.B. Devel-
opment of a three-dimensional treatment planning system based on
computed tomographic data. Int J Oral Maxillofac Surg 2002; 31:
349–357.
22. Welch BB. Practical Programming in Tcl and Tk. Prentice-Hall, New
York, 1999.
23. Schroeder W, Martin K, Lorensen B. The Visualization Toolkit: An
Object-Oriented Approach to 3D Graphics. Kitware, Inc. Publishers,
New York, 2003.
24. Sarti A, Gori R, Bianchi A, Marchetti C, Lamberti C. Maxillo-facial
virtual surgery from 3D CT images. In: Akay M, Marsh A (eds)
Information Technologies in Medicine, Vol. 2, Rehabilitation and
Treatment, pagg, IEEE EMBS Series. Springer-Verlag, New York,
2001.
25. Fung, YC. Biomechanics: Mechanical Properties of Living Tissues.
Springer-Verlag, New York, 1993.
3 8 n V I R T U A L P L A N N I N G O F F A C I A L R E C O N S T R U C T I O N S
IMAGING DECISIONS n 1/2007