Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid

Taking endoscopy to a higher dimensionComputer Aided 3-D NOTES

July 8, 2008

Dipl.-Ing. Kurt Höller1,M. Petrunina2, Dipl. Med.-Inf. J. Penne1, Prof. Dr.-Ing. J. Hornegger1,Dipl.-Ing. A. Schneider3, Dr. med. D. Wilhelm3, Prof. Dr. med. H. Feußner3

1Chair of Pattern Recognition (LME),Friedrich-Alexander-University Erlangen-Nuremberg

2Department of Medicine 1 (MED1),Friedrich-Alexander-University Erlangen-Nuremberg

3Workgroup for Minimal Invasive Surgery (MITI),Klinikum r. d. Isar, Technical University Munich

2 / 36

Content

1 Introduction/Motivation2 NOTES

Idea of NOTESChallenges with NOTES

3 3-D endoscopy contributionsOrientationAugmented RealityEnhanced field of viewCollision Prevention

4 Summarize5 Outlook

K. Höller

3 / 36

Overview





K. Höller

4 / 36

Endoscopic 3-D approachesState of the Art

3-D information can be achieved with

endoscopic ultrasound (EUS)magnetically anchored instrumentspassive optical approaches

stereo visionstructure from motionshape from shading

active optical approachespattern projectiontime-of-flight hybrid system

inertial sensors for gravity related orientation

K. Höller

5 / 36

First prototype of a 3-D endoscopeBased on time-of-flight technology

first presented at

K. Höller

6 / 36

Preliminary resultsLiver phantom with gall bladder

⇒ More details in the talk of Jochen Penne et al.

K. Höller

6 / 36

Preliminary resultsLiver phantom with gall bladder

⇒ More details in the talk of Jochen Penne et al.K. Höller

7 / 36

’Towards NOTES3D’Joint funding application at Deutsche Forschungsgemeinschaft (DFG)

Participating institutes:LME, Erlangen (Prof. J. Hornegger)MITI group, Munich (Prof. H. Feussner)CAMP, Munich (Prof. N. Navab)LGDV, Erlangen (Prof. G. Greiner)MED1, Erlangen (Prof. E.G. Hahn)

Submitted during

K. Höller

8 / 36

Overview





K. Höller

9 / 36

Overview





K. Höller

10 / 36

Time LineFrom open surgery to NOTES

Surgery can be done as:

open surgery→ for hundreds of years

minimally invasive / laparoscopicsurgery→ since the late 80s

and through natural orifices→ "no longer if but when" (W. O. Richards, D. W. Rattner 2005)

⇒ July 22/23, 2005 white paper and foundation of Consortiumfor Assessment and Research (NOSCAR) on NOTES:Natural Orifice Translumenal Endoscopic Surgery

K. Höller

11 / 36

Participating groups with NOTESGreat chance for technical innovations

Figure: Interdisciplinarity of NOTES

K. Höller

12 / 36

Peroral transgastric routeNatural Orifice Translumenal Endoscopic Surgery

K. Höller

13 / 36

Peranal transcolonic routeNatural Orifice Translumenal Endoscopic Surgery

K. Höller

14 / 36

Transvaginal routeNatural Orifice Translumenal Endoscopic Surgery

K. Höller

15 / 36

Peroral transesophageal routeNatural Orifice Translumenal Endoscopic Surgery

K. Höller

16 / 36

NOTES Publications 2004-2007Fast growing community

Figure: NOTES Publications in SE (SAGES), GIE (ASGE), Endoscopy (ESGE), DDW

⇒ In 2008 more than twice the number of publications of 2007K. Höller

17 / 36

Overview





K. Höller

18 / 36

Potential barriers to clinical practiceAccording to the NOTES white paper, New York 2005

Fundamental challenges to the safe introduction of NOTES

Access to peritoneal cavity

⇒ item we can support

Gastric or intestinal closurePrevention of infectionDevelopment of suturing and anastomotic (nonsuturing) devicesMaintaining spatial orientation


Development of a multitasking platform


Management of intraperitoneal complications and hemorrhagePhysiologic untoward eventsTraining other providers

K. Höller

18 / 36

Potential barriers to clinical practiceAccording to the NOTES white paper, New York 2005

Fundamental challenges to the safe introduction of NOTES

Access to peritoneal cavity⇒ item we can supportGastric or intestinal closurePrevention of infectionDevelopment of suturing and anastomotic (nonsuturing) devicesMaintaining spatial orientation⇒ item we can supportDevelopment of a multitasking platform⇒ item we can supportManagement of intraperitoneal complications and hemorrhagePhysiologic untoward eventsTraining other providers

K. Höller

19 / 36

Overview





K. Höller

20 / 36

Overview





K. Höller

21 / 36

Navigation support - OrientationFinding the entry point to the peritonial cavity

Challenge:

More information on position and orientation of the robotic deviceor the endoscope

Solution:

Nonrigid registration of intraoperative 3-D data with preoperativeCT or MR data is possibleCalculated transformation parameters can be used to represent,correct und visualize actual position and orientation

K. Höller

22 / 36

Navigation support - OrientationFinding the entry point to the peritonial cavity

K. Höller

23 / 36

Overview





K. Höller

24 / 36

Navigation support - Augmented RealityFinding the entry point to the peritonial cavity

Challenge:

Avoid injuries of hidden organs and vessels, e.g. while finding theentry point to the peritoneal cavityKnowledge of structures behind the visible wall is needed for asafe incision

Solution:

Registration with preoperative volumesSegmentation of objects of interest in the preoperative volumesAdaption of those objects by iteratively computed transformationparametersVisualization of hidden organs or vessels in intraoperativeendoscopic images by augmented reality

K. Höller

25 / 36


K. Höller

25 / 36


K. Höller

26 / 36

Overview





K. Höller

27 / 36

Navigation support - Off-axis viewFinding the entry point to the peritonial cavity

Challenge:

Overcome boundaries of limited field of view like axis in-line viewand loss of spatial orientation

Solution:

3-D surface knowledge can be used to extend and virtually rotatethe field of viewWith a 3-D mosaicking technique, the field of view can beextended by reconstruction of the operation area.

K. Höller

28 / 36

Navigation support - Off-axis viewFinding the entry point to the peritonial cavity

K. Höller

29 / 36

Overview





K. Höller

30 / 36

Navigation support - Collision preventionFinding the entry point to the peritonial cavity

Challenge:

Provide a higher grade of safety for automatic tools and roboticdevicesEspecially important with multiple instruments through only oneflexible endoscope

Solution:

With real-time distance information efficient collision preventionwith tissue or other instruments can be enabledAuto-positioning depending on respiration or other patientmovements will be very helpful.

K. Höller

31 / 36

Navigation support - Collision preventionFinding the entry point to the peritonial cavity

K. Höller

32 / 36

Overview





K. Höller

33 / 36

Conclusion

Endoscopic 3-D information are precondition to

calculate intra-operative orientation→ registrating with pre-operative MR/CT volumes

avoid injuries of hidden organs and vessels→ making them visible by augmented reality

provide an enhanced field of view→ computing off-axis view or reconstructed area by stitching

to enable collision prevention, motion compensation andautomatic positioning of surgery tools→ using a real-time distance measurement

K. Höller

34 / 36

Overview





K. Höller

35 / 36

OutlookSolution for loss of spatial orientation

real-time information of spatialorientation by measuring gravity

using MEMS-based inertialdevices:3-D accelerometers

K. Höller

36 / 36

The End

Thank you for your attention!

Any further questions?

K. Höller

Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid

Documents