Virtual surgery simulatorsSimulatori per chirurgia virtualeElena De Momi, PhD, Politecnico di Milano, Department of Electronics, Information and Bioengineering
Alessandro Perin, PhD, Neurosurgery Department, Fondazione IRCCS Istituto Neurologico Nazionale "C. Besta", Milan
XXXIV Scuola Nazionale di Bioingegneria, 21- 24 Settembre 2015, Bressanone
Elena De Momi
From the cockpit to the ORThe performance combined
2
FatigueWorkload
Time pressure
TechnologyLives at risk
Responsibilities
Team management
Stress
Risk management
Elena De Momi
Situation awareness modelEndsley model
3
Endsley, Toward a Theory of Situation Awareness in Dynamic Systems, 1995
Elena De Momi
How can they improve their performances? 4
Plan
PreparePractice
Elena De Momi
The need of simulators 5
Simulation of the vehicle behavior
Simulation of environment
Emergency situations handling
Simulation of emergency situations
Team decision making training
Pilot performances (to maintain flight privileges) checking
Elena De Momi
How to make surgery more effective? 6
Training is the perfect answer to this need
Patients’ outcome Pathology social costs
Elena De Momi
Surgical training 7
Tests on human cadavers
• PROS• Equal morphology
• CONS• Costs• Ethical issues• Changes in tissues
behavior• Different
physiology
Tests on live animals
• PROS• Similarities in
tissue behavior• CONS
• Different morphology
• Ethical issues• Costs
Tests on phantoms
• PROS• Reproducibility
(objective evaluation)
• Similar morphology• CONS
• If the models are detailed they can be expensive
• Wrong tissue behavior
Elena De Momi
Surgical simulators ADVANTAGES 8
- Simulate the entire intervention workflow or particular step/actions
- Take team decision – discuss
- Simulate emergency situations/ risks management
- Check procedure
- Assess skills
Preparation of launch
Orbit injection
Orbital flight
Elena De Momi
How can we build simulators? 9
"There is no particular mystery in animation...it's really very simple, and like anything that is simple, it is about the hardest thing in the world to do. "Bill Tytla at the Walt Disney Studio, June 28, 1937
2D pictures
Computer animation: deformable models in a simulated physical world
first feature-length computer animated film1995
Adding rules and laws:
Terzopulos, Artificial fishes, 1994
Elena De Momi
The simulator components 10
From medical
images to models
• Segmentation• Surface meshes
Volumetric models
•Constitutive laws•Volumetric meshes
Interaction
•Haptic interfaces•Interaction visualization•Interaction force rendering
Elena De Momi
From medical images to surface models 11
Image dataset
Elena De Momi
Marching squares 12
Wikipedia
Lorensen WE, Cline HE, Marching Cubes: A high resolution 3D surface construction algorithm. In: Computer Graphics, 1987
Elena De Momi
From medical images to surface models 13
Elena De Momi
From images to volumetric models 14
Elena De Momi
The simulator components 15
From medical
images to models
• Segmentation• Surface meshes
Volumetric models
•Constitutive laws•Volumetric meshes
Interaction
•Haptic interfaces•Interaction visualization•Interaction force rendering
Elena De Momi
Tissue models 16
3
3
2
3
1
3
3
2
2
2
1
2
3
1
2
1
1
1
Xx
Xx
Xx
Xx
Xx
Xx
Xx
Xx
Xx
XxF
22
2
1
1212
J
WJWT
Soft tissues have non-linear behavior: they are non-homogeneous and anisotropic
F deformation gradientx current position of the pointX point in the reference configuration
T uniaxial stress tensorW deformation energyλ eigenvalue of FJ deformation invariants
Holzapfel, Nonlinear Solid Mechanics: A Continuum Approach for Engineering, Wiley, NY. 2004
Elena De Momi
Soft tissue models 17
32exp12 22212
BBBWT
13exp 121 JBBW dτ3dτd2
0 3212
ttW
WT
Liver model
W deformation energyT uniaxial stress tensor
Brain model
B1, B2 material parameter µ shear modulusα experimentally determined
Roan E, The Nonlinear Material Properties of Liver Tissue Determined From No-Slip Uniaxial Compression Experiments. ASME. J Biomech Eng. 2006
Miller, Mechanical properties of brain tissue in tension. 2002
Elena De Momi
Interaction models: Finite Element Models 18
i-th element
Discretized geometry
Nodal degrees of freedom in x, y, z directions
Starting from nodal displacement, stress and strain are estimated in every point (Shape function)
j-th node
constraint
SOLUTIONDisplacements of the nodes estimation
DEFORMATIONINITIAL CONFIGURATION
FORCES applied on nodes
Elena De Momi
Interaction models: Finite Element Models 19
eee uKf
fne nodal forcesune elements node displacementK element stiffness matrix
x cartesian coordinates of a point on an elementN shape function matrixu cartesian coordinates of the element nodes
x = Nu
Elena De Momi
Interaction models: Mass-spring models 20
ij
ijijijsi lkxx
xf
fi acts on spring i and it is generated by spring that connects i and j
xij is the distance vector of masses in xiand xj
lij is the initial length
Provot, Deformation Constraints in a Mass-Spring Model to Describe Rigid Cloth Behavior, 1995
row = 4;col = 7;stiffness = -60; % N/mdamping = -0.5; % Ns/mmass = 0.1; % Kg
Elena De Momi
The simulator components 21
From medical
images to models
• Segmentation• Surface meshes
Volumetric models
•Constitutive laws•Volumetric meshes
Interaction
•Haptic interfaces•Interaction visualization•Interaction force rendering
Elena De Momi
The interaction model 22
Displacement estimation
Force estimation
Force feedback to the user
GRAPHICAL DISPLAY
Elena De Momi
Haptic rendering 23
Elena De Momi
Force feedback enhancement
Tactile sensitivity (cutaneous sensation) depends on size, density, frequency range, nerve fiber branching and type of stimulation (skin motion or sustained pressure).
They are useful to recognize the local properties of objects such as shape, edges, embossing and recessed features, thanks to a direct measure of the intensity and direction of the contact forces.
Kinesthesia: perception of limb movement and position and perception of force.
This sensory perception originates primarily from mechanoreceptors in muscles.
Elena De Momi
Haptics 25
Elena De Momi
26Balancing between accuracy and realism
Elena De Momi
Surgical applications: laparoscopy 27
The surgery is performed through a small hole (keyhole 5-15 mm) – trochar.
Laparoscope: a long fiber optic cable system which allows viewing of the affected area
Advantages:
- Reduction of pain and hemorrhage- Reduction of recovery time
Limits:
- Limited dexterity - Limited visibility - Fulcrum effect
Elena De Momi
Surgical applications: laparoscopy training 28
There are 24 SAGES Fundamentals of Endoscopic SurgeryTM (FES) test centers across North America. This number is expected to grow annually now that the FES test, as a component of the larger Flexible Endoscopy Curriculum, is becoming a requirement for certification by the American Board of Surgery (ABS) for residents in surgery graduating in the 2017-2018 academic year.
•New LAP Mentor™ Lobectomy module trains on complicated procedure of Video-Assisted Thoracoscopic Surgery (VATS)
•GI Endoscopy – Fundamental Skills training module helps learners prepare for flexible endoscopic surgery assessment
•RobotiX Mentor™ is the only VR simulator to offer training for robotic clinical procedures
Elena De Momi
Surgical applications: robotic surgery 29
Advantages:
- Improved dexterity
- Improved intervention intuitiveness
Limits:
- Lack of force feedback
- High costs
- Long learning curve Lack of training increases risks for patient (1700 legal actions against Intuitive mainly for issues related to lack of training)
da Vinci, Intuitive Surgical, CA, USA
Elena De Momi
Surgical applications: robotic surgery training
30
da Vinci Surgical simulator, USA
Specific task training (suturing, gastric bypass, palpation)
Team work decision making
Robotic surgery specific training
Elena De Momi
Surgical applications 31
Basic skills- Visuo-motor coordination- Bimanual cooperation
Cognitive skills- Cognitive load- Context related, feed forward control
Xron, UNIVR
Elena De Momi
Simulation centers 32
ORSIRobotic surgery training(Melle, Ghent University, Belgium)
Goodman Surgical Education CenterStanford Hospital, CA, USA
Elena De Momi
Surgical applications: neurosurgery 33
Necessity of accurate planning:- surgical target - surgical access
avoiding vessels and critical structures (e.g. eloquent areas, ventricles)
Need for training on:
- Surgical workflow, surgical steps sequence
- Team coordination, scrub nurse instrument handling
- Emergency handling (e.g. seizure, vessel rupture)
- Accuracy in targeting
- Maximum forces/ tissue stiffness
Elena De Momi
Surgical applications: neurosurgery training 34
• Various scenarios• Case specific rehearsal• Haptic feedback
• Highly accurate and realistic• Excellent haptic feedback• Multiple metrics parameter registration
Neurotouch, Canada
Imm
ersi
veTo
uch,
Bos
ton,
MA,
USA
Elena De Momi
Neurosurgery simulation center @Besta 35
Neurosurgeons training
Objective resident evaluation
Elena De Momi
Surgical applications: interventional cardiology
36
Training is needed to:
- Avoid exerting excessive force on vessel (risk of rupture)
- Limit contrast medium injection
- Avoid unnecessary fluoroscopic images
- Learn procedure workflow
Surgical procedures performed on the heart by catheterization, inserting a sheath into the femoral artery and cannulating the heart under X-ray visualization
- Avoidance of the scars and pain- Avoidance of long post-operative recovery.
Elena De Momi
Surgical applications: interventional cardiology training
37
(a) VIST-Lab (Mentice AB, Sweden) provides a body mannequin, tactile and force feedback with simulated instruments, different screens for displaying simulated fluoro, or vital images, simulation modules for different arterial diseases within coronary, carotid, renal, and iliac arteries, as well as performance metrics such as fluoroscopy time and contrast volume use
(d) ANGIOMentorTM (Simbionix, OH) provides simulated training of different interventional radiology, vascular and cardiac surgery procedures using preoperative data and simulated fluoroscopywhile enabling simultaneous access through two access sites with tactile feedback
(a)
Rafii Tari et al., Current and Emerging Robot-Assisted Endovascular Catheterization Technologies: A Review 2014
Elena De Momi
Is the simulator good? 38
- Does the system present an environment resembling that, which is encountered during a medical procedure?
- Is a skill measured by the system the specific skill desired, and not a different one?
- Can the system capture the difference between expert and novices?
- To what extent does testing performance with the simulator yield the same results as other measures?
- Does performance/ training with the simulator transfer to improvements in clinical practice?
Elena De Momi
39Robotic surgery
Micron, CMU, PA
Hearthlander, CMU, PA
DaVinci, Intuitive
Rav
en, U
W, W
A
Neuromate, Renishaw mayfield
Alf-x
, Sof
ar
Mako
Rosa, Medtech
Master-slave
Autonomous/ hands on
Hands-on
Catheters
Magellan, Hansen Medical
Neu
roAr
m, I
MR
IS
Elena De Momi
The future of simulators 40
• Excellent 3D visual rendering
• Case specific pre-operative analysis
• Integrated with intra-operative navigation systems
Elena De Momi
41
Thanks for your attention www.nearlab.polimi.it