Advances in Telesurgery and Surgical Robotics Dr S. Sanyal.

Advances in Telesurgery Advances in Telesurgery and Surgical Roboticsand Surgical Robotics

Dr S. SanyalDr S. Sanyal

World’s first telesurgeryWorld’s first telesurgery September 2001September 2001: Tele- chole: Tele- chole

Prof Jacques MarescauxProf Jacques Marescaux, , New York & European Institute New York & European Institute of Telesurgery, Strasbourgof Telesurgery, Strasbourg

Round distanceRound distance =14,000 km=14,000 km

RTTRTT = 200 msec ; video and = 200 msec ; video and hi-speed fibre-optic linkhi-speed fibre-optic link

June 2001June 2001: Johns Hopkins : Johns Hopkins University, Baltimore & Rome University, Baltimore & Rome Policlinico Casilino UniversityPoliclinico Casilino University

http://news.bbc.co.uk/2/hi/science/natuhttp://news.bbc.co.uk/2/hi/science/nature/1552211.stmre/1552211.stm

The research pioneers The research pioneers SRI @ Stanford U School of MedicineSRI @ Stanford U School of Medicine HMSL @ MITHMSL @ MIT IMRL @ UC Berkeley and UCSFIMRL @ UC Berkeley and UCSF

BackgroundBackground

Traditional surgeryTraditional surgery Tri-dimensionalTri-dimensional Cognitive inputCognitive input Tactile feedbackTactile feedback Stereoscopic vision Stereoscopic vision

with depth perceptionwith depth perception Time lag -veTime lag -ve

TelesurgeryTelesurgery Two-dimensionalTwo-dimensional Cognitive feedback limitedCognitive feedback limited Tactile feedback –veTactile feedback –ve Binocular vision without Binocular vision without

depth perceptiondepth perception Time lag +veTime lag +ve

DefinitionsDefinitions Telepresence surgeryTelepresence surgery: : Computerized Computerized

interface @ surgical workstation interface @ surgical workstation ↔↔ remote remote operative site; force feedback (haptic)operative site; force feedback (haptic)

Cooperative telesurgeryCooperative telesurgery: tele-surgeon / : tele-surgeon / local (remote) assistant cooperation local (remote) assistant cooperation

http://www2.telemedtoday.com/articles/telesurgery.shtmlhttp://www2.telemedtoday.com/articles/telesurgery.shtml http://http://web.mit.edu/hmsl/www/Telesurgeryweb.mit.edu/hmsl/www/Telesurgery//

Definitions – cont’dDefinitions – cont’d

TeleroboticsTelerobotics: Remote control with a : Remote control with a robotic arm, in conjunction with a robotic arm, in conjunction with a laparoscopelaparoscope

http://www2.telemedtoday.com/articles/telesurgery.shtmlhttp://www2.telemedtoday.com/articles/telesurgery.shtml

Definitions – cont’dDefinitions – cont’d

TelementoringTelementoring: Experienced surgeon : Experienced surgeon acts as tutor / instructor (acts as tutor / instructor (preceptorpreceptor) for ) for remote surgeon via interactive videoremote surgeon via interactive video

TeleproctoringTeleproctoring ( (proctor=supervisor of proctor=supervisor of examsexams): Documentation of performance ): Documentation of performance for privileging purposesfor privileging purposes

http://www2.telemedtoday.com/articles/telesurgery.shtmlhttp://www2.telemedtoday.com/articles/telesurgery.shtml

Technical aspectsTechnical aspects Image transmissionImage transmission: T1 transmission (H-320 : T1 transmission (H-320

compression standard)compression standard) Fibre-optic cableFibre-optic cable MicrowaveMicrowave SatelliteSatellite

Lag timeLag time: should be : should be << 330 ms 330 ms VOR disruption (3-D vertigo; Simulator sickness within VOR disruption (3-D vertigo; Simulator sickness within

20 minutes)20 minutes) Movement scalingMovement scaling: 1cm : 1cm → 1mm→ 1mm HapticHaptic: Force feedback: Force feedback

HapticHaptic Force reflection / feedback; Graduated Force reflection / feedback; Graduated

tactile inputtactile input– – resistance at remote site is transmitted to resistance at remote site is transmitted to

near site by servo motors @ both sites near site by servo motors @ both sites

Robotic vs. human armRobotic vs. human arm

DOFDOF: Number of ways an arm can move: Number of ways an arm can move Human armHuman arm: 7-DOF: 7-DOF Human handHuman hand: >20-DOF: >20-DOF Robotic armRobotic arm: Like human hand, arm and : Like human hand, arm and

moveable elbow - but moveable elbow - but with a fused wristwith a fused wrist Robotic armRobotic arm: 4-6 DOF: 4-6 DOF

Telesurg dynamics @ MITTelesurg dynamics @ MIT Surgeon’s fingers placed in rings of instruments Surgeon’s fingers placed in rings of instruments Rings are connected to Rings are connected to motorsmotors, , gearsgears and and beltsbelts Precisely translate surgeon’s hand / finger motions Precisely translate surgeon’s hand / finger motions

into digital signalsinto digital signals Transmitted through computer- telecomm linkTransmitted through computer- telecomm link To robotic arms @ remote surgical stationTo robotic arms @ remote surgical station Visual inputVisual input: : 2 remote CCD cameras (15 fps each 2 remote CCD cameras (15 fps each

→ 3-D effect→ 3-D effect) ) → Surgeon’s monitor → Mirror → → Surgeon’s monitor → Mirror → Optical Optical 3-D glasses (stereoscopic vision) 3-D glasses (stereoscopic vision)

http://web.mit.edu/hmsl/www/Telesurgeryhttp://web.mit.edu/hmsl/www/Telesurgery

Telesurg components @ Telesurg components @ MIT MIT

Surgeon’s master tool Surgeon’s master tool handle @ MIThandle @ MIT

Surgeon’s master tool Surgeon’s master tool handle @ MIT handle @ MIT

Master phantom haptic Master phantom haptic interface arm interface arm

Slave phantom haptic Slave phantom haptic interface arm interface arm

Tele-operation slave tool Tele-operation slave tool

Tele-operation slave tool Tele-operation slave tool

Tele-operational details – Tele-operational details – Tool Tool

Tele-operational details – Tele-operational details – Interchangeable tool tips Interchangeable tool tips

Experimental task - grasp / Experimental task - grasp / transfer transfer

Experimental task - Grasp Experimental task - Grasp and transfer with and transfer with orientation orientation

Experimental task - Clip Experimental task - Clip application application

Experimental task – Experimental task – grasper / gripper and grasper / gripper and shear / scissors shear / scissors

Lap experiment box @ Lap experiment box @ MITMIT

Lap simulator-1 @ MITLap simulator-1 @ MIT

Lap simulator-2 @ MITLap simulator-2 @ MIT

Dynamics of robotics @ UCDynamics of robotics @ UC SurgeonSurgeon – remote location – TV console – set of – remote location – TV console – set of

handheld controls ~ videogame joystickshandheld controls ~ videogame joysticks JoystickJoystick:: Pencil-sized; 1 for each hand Pencil-sized; 1 for each hand ComputerComputer: Program translates surgeon’s : Program translates surgeon’s

movements movements End-effectorsEnd-effectors: Robotic instruments enter body to : Robotic instruments enter body to

perform actual operationperform actual operation Early modelsEarly models: 3-fingered hand: 3-fingered hand PresentPresent: Hydraulic-powered, single-digit, 3-4” x ½”, 4-: Hydraulic-powered, single-digit, 3-4” x ½”, 4-

jointed (rotate, swivel, to-fro), 2-pronged end grasperjointed (rotate, swivel, to-fro), 2-pronged end grasper

AnthropomorphicAnthropomorphic movements movementshttp://robotics.eecs.berkeley.edu/medical/

Details of robotsDetails of robots

‘‘Robo-docRobo-doc’: 2 robots working in concert’: 2 robots working in concert Holding robotsHolding robots Companion robots / milli-robots / robotic Companion robots / milli-robots / robotic

manipulatorsmanipulators

Holding robotsHolding robots

Pair of large robotic arms Pair of large robotic arms Hydraulic-actuatedHydraulic-actuated Sits on moveable platformSits on moveable platform Driven remotely by surgeon’s joysticksDriven remotely by surgeon’s joysticks Performs like a surgeon’s shoulder, Performs like a surgeon’s shoulder,

allowing positioning of its hydraulic armsallowing positioning of its hydraulic arms

Holding robots – cont’dHolding robots – cont’d

Holds 2Holds 2ndnd robot, wheels instruments robot, wheels instruments into position by patient’s sideinto position by patient’s side

Guides them through dexterity-Guides them through dexterity-requiring surgical procedures requiring surgical procedures (suturing, dissection)(suturing, dissection)

Holds instruments steady while Holds instruments steady while surgeon sutures and ties knotssurgeon sutures and ties knots

Companion / Milli-robots / Companion / Milli-robots / Robotic manipulatorsRobotic manipulators

Sterile, disposable, steel, mm-scale, Sterile, disposable, steel, mm-scale, fingertip-sized fingertip-sized

Slender, jointed, finger-likeSlender, jointed, finger-like toolstools Connected by wires and tubes to larger Connected by wires and tubes to larger

robotrobot Pair of gripping forceps at one end to Pair of gripping forceps at one end to

carry surgical tools carry surgical tools Contains miniscule video-cameraContains miniscule video-camera

Companion robot – cont’dCompanion robot – cont’d

Inserted into body for actual surgical Inserted into body for actual surgical tasks (cutting, suturing) – 10-20 mm tasks (cutting, suturing) – 10-20 mm incisionsincisions

Inserts camerasInserts cameras Provide tactile feedback though force-Provide tactile feedback though force-

deflecting joysticksdeflecting joysticks Provides 7 DOFProvides 7 DOF

Setup @ UC BerkeleySetup @ UC Berkeley

Equipment @ UCBEquipment @ UCB

Robotic manipulator @ UCBRobotic manipulator @ UCB

Mini robot controls @ Mini robot controls @ UCBUCB

Roll-pitch-roll ‘wrist’, gripper and multi-fingered manipulators

Robotic endo-manipulatorRobotic endo-manipulator

Endo-platform with biopsy forceps

Minute threadingMinute threading

Threaded robotic Threaded robotic instruments – knot tyinginstruments – knot tying

2-G RTWL @ UCSF2-G RTWL @ UCSF

Lap interface @ UCSFLap interface @ UCSF

4-DOF lap haptic 4-DOF lap haptic interfaceinterface

Robotic Cardiac Surgery Robotic Cardiac Surgery @ EHIRC @ EHIRC

da Vinci Tele-manipulationda Vinci Tele-manipulation system system Intuitive Surgical Inc., Mountain View, Intuitive Surgical Inc., Mountain View,

CA, USA CA, USA Computer enhanced system Computer enhanced system

Surgeon’s consoleSurgeon’s console Cart-mounted robotic manipulatorsCart-mounted robotic manipulators

http://www.ehirc.com/individuals/services/treatment/robotic_surgery.html#http://www.ehirc.com/individuals/services/treatment/robotic_surgery.html#

Surgeon’s console @ EHIRCSurgeon’s console @ EHIRC

Display systemDisplay system: 3-D : 3-D pictures of chest cavitypictures of chest cavity

SurgeonSurgeon sits at sits at console and gets 3-D console and gets 3-D view of chest interior view of chest interior

Hand motions are Hand motions are captured, transformed captured, transformed and transmitted to tiny and transmitted to tiny robotic manipulators robotic manipulators

Robotic manipulators @ Robotic manipulators @ EHIRCEHIRC

Robot is not autonomous; surgeon-Robot is not autonomous; surgeon-controlled controlled

Hold tiny instruments, which go inside the Hold tiny instruments, which go inside the patient's chest.patient's chest.

Surgeon's hand movements transmitted Surgeon's hand movements transmitted to these instrumentsto these instruments

CABG, mitral valve repair, ASD closure CABG, mitral valve repair, ASD closure

IMA – LAD CABGIMA – LAD CABG

Totally endoscopic CABG Totally endoscopic CABG

AdvantagesAdvantages Only 3-incisions, each 1 cm on the side and Only 3-incisions, each 1 cm on the side and

lower chestlower chest Less painLess pain Faster healing and recoveryFaster healing and recovery Short hospital stay Short hospital stay

Technical innovations Technical innovations

Teletactation (Tactile feedback)Teletactation (Tactile feedback) CyberGloveCyberGlove®® with CyberTouch with CyberTouch Dextrous master gloveDextrous master glove Spatial cognition – Hand assistSpatial cognition – Hand assist Surgical simulationsSurgical simulations Dextrous mini-robotsDextrous mini-robots

Teletactation –Teletactation –Tactile Tactile feedbackfeedback

Sensing tactile information through Sensing tactile information through tactile tactile sensorssensors that transmit feel of tissue to that transmit feel of tissue to surgeon’s fingersurgeon’s finger

CyberTouch – CyberTouch – CyberGloveCyberGlove®® Vibro-tactile, thermal simulators on each Vibro-tactile, thermal simulators on each

fingerfinger and and palmpalm Tactile feedback option Tactile feedback option enables feelenables feel of of

virtual object virtual object

CyberGloveCyberGlove®®

Flexible sensors Flexible sensors measure positionmeasure position / / movementmovement of of fingersfingers and and wristwrist

Dextrous master gloveDextrous master glove ThumbThumb,, index index, , wrist wrist flexion sensors and flexion sensors and

wrist rotation sensorwrist rotation sensor Senses positionsSenses positions of surgeon's fingers/wrist of surgeon's fingers/wrist Used as master to drive slave robotic hand Used as master to drive slave robotic hand

Spatial cognition – Hand Spatial cognition – Hand assist in telesurgeryassist in telesurgery

Non-dominant hand in-vivo possibly enhances spatial skills through tactile cues, which generate a more accurate 3-D representation of anatomy

Lap chole simulationLap chole simulation

Simulated fat and fascia Dissected away; cystic duct clipped

Lap chole simulation – Lap chole simulation – cont’dcont’d

Cystic artery and duct divided successfully in simulated conditions

Karlsruhe Gynec endo Karlsruhe Gynec endo surgery simulations surgery simulations

Gynec surgery Gynec surgery simulations – cont’dsimulations – cont’d

http://www-kismet.iai.fzk.de/VRTRAIN/phD_main.html

http://www-kismet.iai.fzk.de/VRTRAIN/GIF/PHD/surgSim.jpg

Dextrous mini robotsDextrous mini robots 1 – Camera attachment1 – Camera attachment 2 – Equipped with a needle for biopsy2 – Equipped with a needle for biopsy 3 – Moves around abdominal cavity – spiral 3 – Moves around abdominal cavity – spiral

pattern – moves without slippingpattern – moves without slipping http://news.bbc.co.uk/1/hi/health/4647258.stmhttp://news.bbc.co.uk/1/hi/health/4647258.stm

SummarySummary

Technically demanding, labour intensive, Technically demanding, labour intensive, time consuming, expensive researchtime consuming, expensive research

Learning curve with similar Learning curve with similar characteristicscharacteristics

Expensive installation, maintenance and Expensive installation, maintenance and infrastructureinfrastructure

Future applicationsFuture applications

Emergency trauma care – 1Emergency trauma care – 1stst ‘Golden ‘Golden Hour’Hour’

Battlefield surgeryBattlefield surgery Remote area assistanceRemote area assistance One-to-many telementoringOne-to-many telementoring Space station surgerySpace station surgery

Pre-conclusionPre-conclusion

““Science knows no country, because Science knows no country, because knowledge belongs to humanity, and knowledge belongs to humanity, and is the torch which illuminates the is the torch which illuminates the worldworld.”.” Louis Louis PasteurPasteur

““Don't be afraid to take a big step. You Don't be afraid to take a big step. You can't cross a chasm in two small can't cross a chasm in two small jumpsjumps.”.”

David David Lloyd GeorgeLloyd George

ConclusionConclusion