Haptic Virtual Reality Training Simulator for Sacral ...nearlab.polimi.it/wp-content/uploads/2019/01/Presentazione_Leonar… · Virtual reality and Haptic simulator: Virtual reality:

Titolo presentazione

sottotitolo

Milano, XX mese 20XX

Haptic Virtual Reality Training Simulator for Sacral Neuromodulation Surgery:

a Feasibility Study

Advisor: Prof.ssa Elena De Momi

Co-Advisor: Prof. Cristian Luciano,

University of Illinois

Candidate:

Leonardo Crespi

Leonardo Crespi

Index

2

Background

State of the Art

Approach

Implementation

Test

ResultsAnd

discussion

Conclusions

SacralNeuromodulation

Surgical training

Proposedsolution

Hardware And

Software

Leonardo Crespi

• Urological condition defined by urinary

urgency with or without incontinence, urinary

frequency and/or nocturia

• >200M people affected worldwide, mainly

older people

• Not life threatening, mainly affects quality of

life

• Three levels of therapy:

1. Behavioral therapy

2. Drug therapy

3. Sacral Neuromodulation

Overactive Bladder

3

N. Eapen at al, 2016, “Review of the epidemiology of overactive bladder” https://www.sciencepicture.com/

Leonardo Crespi

• Surgical procedure, ≈3%

people affected by OAB are

treated with SNM

• Electrical pulses on the sacral

nerve through an implanted

device

• High costs (≈20000€ per

patient after 5 years)

• Better overall results than other

therapies

Sacral Neuromodulation (1/2)

https://www.xvivo.net/

4

Arlandis et al, 2011,“Cost-Effectiveness of Sacral Neuromodulation Compared to Botulinum Neurotoxin A or Continued Medical Management in Refractory Overactive Bladder”

Implanted device

Leonardo Crespi 5

Sacral Neuromodulation (2/2)

https://www.bidneedham.org/departments/urology/urology-our-specialities/sacral-neurostimulation

KE Matzel et al, 2017, ‘’Standardized electrode placement technique’’Surgical Procedure

Fluoroscopic guidance

• 2D imaging, A-P and Sagittal

• S3 foramen and landmarks identification

Percutaneous access

• S3 foramen

• Skin and fusion plane are the references

• Checked placement

• Guide with radio

marker

• Curved stylet with 4

electrodes

Leonardo Crespi

Current training systems

6

http://syndaver.com/gallery/

https://simbionix.com/simulators/uro-mentor/

Biological bench models• Real tissues and textures• Useful for simple

procedures• Non reusable• Different anatomy• Ethical concerns

Cadavers• Realistic• Difficult to be found

Virtual Reality• Adaptable• Reusable• Measurable

performance• Very expensive• No haptic feedback

Non biological bench models• High/ low fidelity• Very expensive in the first case,

very inaccurate in the second• Non reusable• Not adaptable

Leonardo Crespi

Virtual reality and Haptic simulator:

Virtual reality: anatomical accuracy,

adaptability, safety

Haptics: realism through tactile

feedback

Proposed Solution (1/2)

Three steps simulation:1. Percutaneus access2. Guide insertion3. Electrode lead insertion

7

Requirements of an ideal training

system:

• Anatomical accuracy

• Realism

• Adaptability

• User friendliness

• Measurability of the performance

• Safety

• Commercial viability and relevancy

Leonardo Crespi

Proposed Solution (2/2)

8

Leonardo Crespi

Implementation

CT scan

9

3D Models VR Environment

Haptic device

Leonardo Crespi

Hardware

Haptic device

Processor Graphic Card RAM

PC used at UIC Intel i5-2500K, 3.3GHz NVIDIA Quadro K600 DDR3, 8 GB

PC used at PoliMi Intel i7-6800K, 3.4GHz NVIDIA Titan Xp DDR3, 16 GB

10

Model Omni Touch 3D Stylus™

Positional feedback 6 DOF

Force feedback 3 DOF (Position only)

Workspace dimensions(width x heigth x depth [cm])

26.5x23.6x8.9

Nominal Max force [N] 3.3

Nominal position resolution [mm]

0.084https://it.3dsystems.com/haptics-devices/touch

Leonardo Crespi

Anatomical Models

• Contrast enhancement• Segmentation of the CT scan• 3D modeling to refine the meshes

11

Leonardo Crespi

Software Development Kit

LACE

QuickHaptics VisualizationLibrary

Wykobi

Efficientmathematicalcomputation. v0.0.5

OpenGL basedmiddleware to create 2D and 3D graphics; v2.0.0

Haptic device management; v3.0

12

http://visualizationlibrary.org/docs/2.0/html/index.html

http://www.wykobi.com/https://www.3dsystems.com/haptics-devices/openhaptics

Leonardo Crespi

Application features (1/3)

C-Arm simulation

Antero-posterior view

13

Sagittal view

Raycasting , α-compositing

𝐶𝑓𝑏 =

𝑖=𝑓

𝑏

𝐶𝑖𝑎𝑖ෑ

𝑗=𝑓

𝑖−1

𝑇𝑗

𝑪𝒇𝒃 : composite color ([r g b]) of samples from pixel b (back) to

pixel f (front) taken along a ray to project the volum on a plane;𝒂: opacity; T = 𝟏 − 𝒂 , transparency;

Leonardo Crespi

Application features (2/3)

Animations to simulate the patient’s response to electrical stimulation

▪ Vector of N images used as a flip book to display the patient response, streamed back and forth from 0 to nat 60Hz

▪ n computed through a specifically designed function of distance (d) and current intensity (i)

▪ Current intensity (i) set by the user through the Graphic User Interface

𝑛 =

𝑁, 𝑑 = 0;

𝑎𝑖

𝑑, 0 < 𝑑 ≤ 𝐷;

0, 𝑑 > 𝐷;

N

n

Vector of images

14

Needleclose to the target

Needleless closeto the target

Leonardo Crespi

Application features (3/3)

Bendable electrode▪ LACE_Extrusion implementation▪ Catmull-Rom spline (Continuous piecewise

defined polynomial function 𝑃 𝑡 ) to create the shape, updated each graphic frame

▪ 4 control points set according to the proxy position

▪ Tension parameter (f) set to .5

Equation between two point 𝑃𝑖−1 and 𝑃𝑖

𝑃 𝑡 = 1 𝑡 𝑡2 𝑡3

0 1 0 0−𝑓 0 𝑓 02𝑓 𝑓 − 3 3 − 2𝑓 −𝑓−𝑓 2 − 𝑓 𝑓 − 2 𝑓

𝑃𝑖−2𝑃𝑖−1𝑃𝑖𝑃𝑖+1

𝑃𝑖−2

𝑃𝑖−1

𝑃𝑖

𝑃𝑖+1

𝑃(𝑡)

15

Leonardo Crespi

Haptic effects (1/2)

Shape related

▪ Stiffness, viscosityand popthrough

▪ Range 0 – 1 in QuickHaptics API

▪ Tuned with the help of urologists

Stiffness Viscosity Popthrough

Spine 1 0 Not defined

Skin 0.4 0.5 0.3

Liver 0.2 0.3 Not defined

Lungs 0.15 0.1 Not defined

Kidneys 0.35 0.2 Not defined

Values assigned to the anatomical models

16

Force (F) computed when an object is touched

𝐹 = 𝑠𝑡𝑖𝑓𝑓𝑛𝑒𝑠𝑠 ∗ Δ𝐷 + 𝑣𝑖𝑠𝑐𝑜𝑠𝑖𝑡𝑦 ∗ 𝑣

Δ𝐷: depth of penetration inside the object, computed as the difference between the proxy position and its projection on the surface of the object;v: proxy velocity

Leonardo Crespi

Haptic effects (2/2)

17

Event Related Fulcrum effectTriggered when the skin is entered with the needle to simulate a puncture

Line effectTriggered when the electrodeis inserted in the guide, to simulate the constraint and drive it to the correct spot

p = p’ p

p

p’

p’

r rr

r’r’ r’

F

F

𝒓: 𝑓𝑢𝑙𝑐𝑟𝑢𝑚 𝑝𝑜𝑖𝑛𝑡𝒑: 𝑝𝑟𝑜𝑥𝑦 𝑝𝑜𝑠𝑖𝑡𝑖𝑜𝑛;𝑸 = 𝒒𝒘,𝒒𝒙, 𝒒𝒚, 𝒒𝒛𝑝𝑟𝑜𝑥𝑦 𝑜𝑟𝑖𝑒𝑛𝑡𝑎𝑡𝑖𝑜𝑛;

𝒓′ = 𝒓 + 𝒒𝒙, 𝒒𝒚, 𝒒𝒛 ;𝒍: 𝑙𝑖𝑛𝑒 𝑝𝑎𝑠𝑠𝑖𝑛𝑔 𝑓𝑜𝑟 𝒓 𝑎𝑛𝑑 𝒓′;𝒑′: 𝑝𝑟𝑜𝑗𝑒𝑐𝑡𝑖𝑜𝑛 𝑜𝑓 𝒑 𝑜𝑛 𝒍;

𝑭 = 𝒌 𝒑′ − 𝒑 𝑓𝑜𝑟𝑐𝑒

l l l

Proxy Proxy

Proxy

Leonardo Crespi

Preliminary test to understand how a novice might get along with the application

• 15 Biomedical Engineering Master students

• 2 repetitions for each subject

• Quantitative data: measured by the application in terms of total time, fluoroscopy time, distance from the target and interactions with the periosteum; Wilcoxon signed rank test (α=0.05) for paired data to evaluate statistical significance of differences between the repetitions

• Qualitative data through the System Usability Scale, a ten item Likertattitude test that gives a score in range 0-100, to assess how welldesigned the system is from the user’s point of view

Test

18

Leonardo Crespi

Quantitative data

Results and Discussion (1/2)

First observation Second observation

P-value 0.01

19

P-value 0.02

P-value 0.99 P-value 0.91

Leonardo Crespi

Qualitative data

Results and Discussion (2/2)

𝑆𝑈𝑆𝑠𝑐𝑜𝑟𝑒 = 2.5

𝑖=1

5

[ 𝑄2𝑖−1 − 1 + 5 − 𝑄2𝑖 ]

• Each item has a score which statesthe level of agreement with the sentence, ranging from 1 (minimum) to 5 (maximum)

• 𝑄𝑖 is the score for the 𝑖𝑡ℎ item• The final SUS score is the average

of the subjects𝐹𝑖𝑛𝑎𝑙 𝑆𝑈𝑆𝑠𝑐𝑜𝑟𝑒 = 𝟔𝟎. 𝟔𝟕 ± 𝟏𝟓. 𝟑𝟓

Not at all33%

Somewhat40%

Fairly27%

Very0%

I thought there was too muchinconsistency in this system

Not at all0%

Somewhat27%

Fairly7%

Very33%

Extremely33%

I think i would need the support of a technical personto be able to use this system

20

Not at all0%

Somewhat13%

Fairly34%Very

40%

Extremely13%

I found the various functions in this system were well

intergrated

Not at all

Somewhat

Fairly

Very

Extremely

Leonardo Crespi

Solution features with the

respect to the requirements

Conclusions

• Anatomical accuracy• Realism• Safety • Measurability of the

performance• Commercial viability and

relevance

• User friendliness

• Adaptability

21

▪ Create a library of virtual patients▪ More extensive tests, involving

medical experts and directlycomparing this system with others available

▪ Information about the angle that the needle should be keept at during the percutaneus access given the anatomy

▪ Replacing the 3D monitor with an head mounted display

Future developments

Leonardo Crespi

Aknowledgements

22