Future and Emerging Technologies Future and Emerging Technologies Biorobotics Science and Engineering: from Bio- Inspiration to Bio- Paolo Dario Scuola Superiore Sant’Anna, Pisa, Italy Inspiration to Bio- Application
Future and Emerging TechnologiesFuture and Emerging
Technologies
Biorobotics Science and Engineering: from Bio-Inspiration to Bio-
Paolo Dario
Scuola Superiore Sant’Anna, Pisa, Italy
Inspiration to Bio-Application
Outline of the talk
� Introduction to Biorobotics and Biomechatronics
� Biorobotics Science
� Building robots to investigate humans and animaland animal
� Biorobotics Engineering
� Robotics for surgery and endoscopy
� Robotics in rehabilitation and assistance
� Conclusions
Outline of the talk
� Introduction to Biorobotics and Biomechatronics
� Biorobotics Science
� Building robots to investigate humans and animaland animal
� Biorobotics Engineering
� Robotics for surgery and endoscopy
� Robotics in rehabilitation and assistance
� Conclusions
Biorobotics and biomechatronic design
Engineering analysis Engineering analysis and modelingand modeling
Biological systemBiological system
Validation
Development of a physical model
Bio-mimetic Bio-mimetic robot
Bio-inspired Bio-inspired robot
ApplicationsApplications
Development of a biomedical robot
Biorobotics Science and Engineering
Biorobotics Science: using robotics to discover…
Biorobotics Engineering: using Biorobotics Engineering: using robotics to invent…
Outline of the talk
� Introduction to Biorobotics and biomechatronics
� Biorobotics Science
� Building robots to investigate humans and animaland animal
� Biorobotics Engineering
� Robotics for surgery and endoscopy
� Robotics in rehabilitation and assistance
� Conclusions
I
O
I
O
I
O
I
O
HYPOTHESIS AND MODEL
Biorobotics Science
PHENOMENON TO BE
Gripforce
Loadforce
Movement
Gripforce
Loadforce
Movement
TO BE EXPLAINED
Biorobotics vs. simulation and animal models
Human modelHuman model
InteractionModel of Model of
interactioninteraction
World modelWorld modelWorldWorld
I
O
I
O
I
O
I
O
HYPOTHESIS AND MODEL
Biorobotics Science
PHENOMENON TO BE
IMPLEMENTATION IN A ROBOT
Gripforce
Loadforce
Movement
Gripforce
Loadforce
Movement
Validation of Validation of the modelthe model
EXPERIMENTComparison between robot and biological system performance
TO BE EXPLAINED
A robotic platform for validating a model of development of sensory-motor grasp control
Objectives:
� To increase knowledge of brain connectivity (architecture) and brain activity (functioning) concerning sensory motor coordination for object manipulation in children
To integrate an � To integrate an anthropomorphic robotic platform for graspingand manipulation to validate a neurophysiological model of the five learning phases of visuo-tactile-motor coordination in infants
PALOMA EU IST-FET Project IST-2001-33073
P. Dario, M.C. Carrozza, E. Guglielmelli, C. Laschi, A. Menciassi, S. Micera, F. Vecchi, “Robotics as a “Future and Emerging Technology: biomimetics, cybernetics and neuro-robotics in European projects”, IEEE RAM, Vol.12, No.2, June 2005, pp.29-43.
HYPOTHESIS AND
Biorobotics Science
IMPLEMENTATION
PHENOMENON TO BE
EXPLAINED(combination of HYPOTHESIS AND
MODEL (comparison between numerical results and
interpolated experimental data of living oligochaeta)
Validation of Validation of the modelthe model
EXPERIMENTComparison between robot and biological system performance
IMPLEMENTATION IN A ROBOT
A. Menciassi and P. Dario, Philos. Transact. Roy. Soc. A Math. Phys. Eng. , 2003
D. Accoto, P. Castrataro, P. Dario, J. Theor. Biology, 2004
(combination of friction and segment number for effective
locomotion)
The Scuola Superiore Sant’Anna “Zoo”
Oligochaeta Role of friction in locomotion Endoscopy of GI tract
Legged insects Modeling compliant substrates Endoscopy of GI tract
PolychaetaNew computational models of locomotion kinematics
Rescue, field robotics
Biological model Scientific problem Engineering application
locomotion kinematics
Swimming cells Swimming at low Re numbers Neuroendoscopy
Cricket Scale effects on locomotion Mobile sensor networks
LampreyNeuroscientific models of goal-driven locomotion
River exploration, new robots (soft bodied)
Octopus Motor performance of hydrostatic muscular limbs
Infinite DOF robots, rescue, marine
Plant roots Soil penetration mechanisms Environmental robotics
Mouse Animal-robot interaction Entertainment, …
The Scuola Superiore Sant’Anna “Zoo”
Oligochaeta Role of friction in locomotion Endoscopy of GI tract
Legged insects Modeling compliant substrates Endoscopy of GI tract
PolychaetaNew computational models of locomotion kinematics
Rescue, field robotics
Biological model Scientific problem Engineering application
locomotion kinematics
Swimming cells Swimming at low Re numbers Neuroendoscopy
Cricket Scale effects on locomotion Mobile sensor networks
LampreyNeuroscientific models of goal-driven locomotion
River exploration, new robots (soft bodied)
Octopus Motor performance of hydrostatic muscular limbs
Infinite DOF robots, rescue, marine
Plant roots Soil penetration mechanisms Environmental robotics
Mouse Animal-robot interaction Entertainment, …
Lamprey & Salamander-like robots
Scientific Problem addressed according to the EMBODIED INTELLIGENCE paradigm:
to investigate neuroscientific open issues on locomotion by means of physical on locomotion by means of physical platforms on which to implement
theoretical models of neural mechanisms
Phylogenetic tree
From aquatic to terrestrial locomotion
Salamander
INT
RO
DU
CT
ION
Lamprey
530
360Ichtyostega
INT
RO
DU
CT
ION
INTRODUCTION
Axial CPG
Forelimb CPG
Hindlimb CPG
Working hypothesis
Hindlimb CPG
Salamander CPG = Lamprey-like axial CPGaxial CPG
extended with 2 limb CPGslimb CPGs
Evolution of spinal locomotor CPG for locomotion
Lamprey Salamander Cat Human
FORWARD SWIMMING(« lamprey like »)
Salamander in anguilliform (lamprey-like) swimming
Ijspeert et coll., 2007
• Traveling waves of lateral
displacement passing downthe body.
FORWARD STEPPING(« crocodile like »)
Salamander in stepping locomotion.
Ijspeert et coll., 2007
• Standing waves of lateral
displacement with fixed nodesat pectoral and pelvic girdles.
LAMPETRA Expected Results
Advances in neuroscienceBetter models of goal directed locomotion, and in particular of:
• mechanisms addressing striatum/basal ganglia in the selection betweendifferent patterns of behaviors based on visual input, other senses andprevious experience;
• motivational control as in the case of hunger, aggression, sexual partner
Advances in ICT technology
• motivational control as in the case of hunger, aggression, sexual partnerselection, day/night cycle.
• Control: rethinking traditional control by exploiting interacting layers ofdifferent behaviours instead of adopting a more traditional approach ofmodelling and planning, allowing to control complex systems (thousandsof receptors, hundreds of actuators, multimodal sensory inputs).
• Hardware: New technologies for actuators, sensors and materialsenabling soft-bodied robotics.
forward backward
lag
Neural activity
Accurate
simulation but affected by
intrinsic simplifying
… need for an
embodied study for more accurate
investigation
Virtual models of the Lamprey
simplifying
hypotheses…
O. Ekeberg and S. Grillner.
Salamander-like robots
To verify models of the transition from aquatic to terrestrial locomotion in vertebrate evolution
An amphibious salamander robot
demonstrates how a primitive neural circuit for swimming can be extended by
phylogenetically more recent limb
oscillatory centers to explain the ability
of salamanders to switch between swimming and walking
Ijspeert, Cabelguen et al. “From swimming to walking with a salamander robot driven by
a spinal cord model” Science, 9 March 2007.
The Lamprey robot(skeletal system)
(Full, Cutkosky, et al., 2002)
The Lamprey robot
(first experiments)
Not stabilized head
Not progressive wave
The Model of Collaboration between Neuroscience and Robotics in Lampetra
NEURO-
SCIENCE
MODELS
NEW SCIENCE
ARTEFACTS
(existing early
prototypes)
(Newly designed)
HYBRID BIONIC
SYSTEMS
NEW TECHNOLOGY
Joint SSSA-KI investigation
Schematic: obstacle avoidance by steering and recovery of the original direction Involved forebrain structures
PALLIUMPALLIUMSTRIATUMSTRIATUM
PALLIDUMPALLIDUM
THALAMUSTHALAMUS TECTUM
Lamprey - Task no. 1: Obstacle Avoidance
Activation sequence: retina; thalamus; pallium and striatum; pallidum;
tectum-eye mvt.; tectum-body orienting mvt.; MLR-DLR.
RETINARETINA MLRMLR
DLRDLR
Locomotion
Eye mvt.
Body orienting
mvt.
Experimental artefact
Early (neuro-robotics) co-design
25 segments, wireless,
on board processing, 1 hour autonomy
Stretch receptors + vision + vestibular
sensors
Joint SSSA-KI investigation
Schematic: pursuit of a movable target
Lamprey - Task no. 2: Target Pursuing
Involved forebrain structures
PALLIUMPALLIUMSTRIATUMSTRIATUM
PALLIDUMPALLIDUM
THALAMUSTHALAMUS TECTUM
Activation sequence: retina; thalamus; pallium and striatum; pallidum;
tectum-eye mvt.; tectum-body orienting mvt.; MLR-DLR.
RETINARETINA MLRMLR
DLRDLR
Locomotion
Eye mvt.
Body orienting
mvt.
Experimental artefact
Early (neuro-robotics) co-design
25 segments, wireless,
on board processing, 1 hour autonomy
Stretch receptors + vision + vestibular
sensors
The Scuola Superiore Sant’Anna “Zoo”
Oligochaeta Role of friction in locomotion Endoscopy of GI tract
Legged insects Modeling compliant substrates Endoscopy of GI tract
PolychaetaNew computational models of locomotion kinematics
Rescue, field robotics
Biological model Scientific problem Engineering application
locomotion kinematics
Swimming cells Swimming at low Re numbers Neuroendoscopy
Cricket Scale effects on locomotion Mobile sensor networks
LampreyNeuroscientific models of goal-driven locomotion
River exploration, new robots (soft bodied)
Octopus Motor performance of hydrostatic muscular limbs
Infinite DOF robots, rescue, marine
Plant roots Soil penetration mechanisms Environmental robotics
Mouse Animal-robot interaction Entertainment, …
Novel Design Principles and Technologies for a New Generation of High Dexterity Soft-bodied Robots Inspired by the Morphology andBehaviour of the Octopus
OCTOPUS (2008-2012)
The new OCTOPUS Project has the objective of designing and developingan 8-arm robot inspired to the muscular structure, neurophysiology andmotor capabilities of the octopus (Octopus vulgaris).
BIOMIMETIC ROBOTICS
The ANGELS Project
ANGuilliform robot with ELectric Sense
Outline of the talk
� Introduction to Biorobotics and biomechatronics
� Biorobotics Science
� Building robots to investigate humans and animaland animal
� Biorobotics Engineering
� Robotics for surgery and endoscopy
� Robotics in rehabilitation and assistance
� Conclusions
Outline of the talk
� Introduction to Biorobotics and biomechatronics
� Biorobotics Science
� Building robots to investigate humans and animaland animal
� Biorobotics Engineering
� Robotics for surgery and endoscopy
� Robotics in rehabilitation and assistance
� Conclusions
The Evolution of SurgeryTRADITIONAL SURGERY
MINIMALLY INVASIVE SURGERY
ENDOLUMINAL SURGERY
Micro-endoscope for spinal cord
FETAL Da Vinci CAS system
Endoscopic capsuleReconfigurable surgical system
FETAL SURGERY
Force-feedback scissor for fetal surgery
CELL SURGERY
Artificial virus for cell therapy
Early Detection of Colon Cancer Saves Life. Colonoscopy is the Gold Standard. But…
• Pain and disconfort for the patient
• Complex and demanding procedure for the doctor
• The active part of the colonoscope is the head, that incorporates the visualization system (optical fibers or camera, optics, illumination)camera, optics, illumination)
• The head must be inserted along the colon by maneuvering and pushing, from outside the body, a relatively stiff shaft
• These actions stretch the colon and originate pain
The case of endoscopic tools for gastrointestinal analysis
Problem: pain,
difficult
maneuverability…
…imitating the worm?
Painless Colonoscopy
Semi-autonomous
inchworm-like
locomotion
SOFT TAIL
CLAMPER
FLEXIBLE
BODY
BENDING
SECTION
CMOS
CAMERA
AND
LIGHT
SOURCE
The E-WORM
Painless
Colonoscopy
System
From “wired”painless colonoscopy to “wireless” GI endoscopy
The case of endoscopic tools for gastrointestinal analysis
Problem: pain,
difficult
maneuverability…
Solution:
inchworm
locomotion,
Problem: slow,
not adequate
for different
gut
diameters…
Solution: legged
locomotion, insect-
like capsular
endoscopy
…like a worm in the gut…
locomotion,
self-
adaptability
endoscopy
Wireless endoscopic capsules with activelocomotion system for the entire GI tract
Single capsule approach: swimming locomotion
Ingestion of liquid in context with the examination allows to obtain organ distension,
thus making possible a low power 3D locomotion in the stomach
Fine control of steering and speedFine control of position
Single capsule approach: swimming locomotion
Ex vivo test in stomach of pig filled with water
Fine control of steering and speedFine control of position as regards the water level
Single capsule approach: legged capsule for tubular organs
Obtaining an active locomotion in
tubular organs of the GI tract, that
cannot be inflated or filled with water,
means having propulsion mechanisms
able to open and distend the tissue
around the capsule.
1. Diameter: 11.1 mm;
2. Length: 28 mm (+camera);
3. 12 legs;
4. 2 DC brushless motors (NAMIKI);
5. Force at the leg’s tip of about 1N;
6. No frontal latex balloon required;
7. On board electronics drivers;
M. Quirini et al., ICRA 2007
7. On board electronics drivers;
8. Power consumption: 0.66 W. Stefanini et al. Int. J. Of Rob. Res.,
2006.
Patent filed
Single capsule approach: legged capsule for tubular organs
Free motion
Control panel
M. Quirini, S. Scapellato, A. Menciassi, P. Dario, F. Rieber, C.-N. Ho, S. Schostek, M.O. Schurr, “Feasibility proof of a legged locomotion capsule for the GI tract”, GASTROINTESTINAL ENDOSCOPY Vol. 67, No. 7, 2008
Simulator test
Colon test (5 cm/min)
AAssembling ssembling RReconfigurable econfigurable EEndoluminal ndoluminal
SSurgical system (ARESurgical system (ARES--FP6FP6--NEST)NEST)
Target
area
Operation
THE VISION:
Performing
complex
internal surgery by tiny
robots that
assemble within the body,
perform tasks, and are then removed
ARAKNES - Array of Robots Augmenting
the KiNematics of Endoluminal Surgery
The ultimate goal: to integrate the advantages of traditional open surgery, laparoscopic surgery (MIS), and robotics surgery into a deeply innovative
system for bi-manual, ambulatory, tethered, visible scarless surgery, based on an array of smart microrobotic instrumentation
Main intended interventions:
Endoluminal and transluminal surgery (bariatric surgery, local excision, others)Single-port laparoscopy
ARAKNES - Array of Robots Augmenting
the KiNematics of Endoluminal SurgeryOPERATIVE TOOLASSISTIVE TOOL
Control board Li-Po battery Module
Prototyping of the Basic ModulesPrototyping of the Basic Modules
Control board Li-Po battery Module
Motor (4 x 17.4 mm) 1.03g, 10.6 mNm
camera
Biopsy
forceps
Tissue
Storage
12 Modules
-Camera X1
-Forceps X1
-Storage X1
-Central X1
-Structural X8
Example of a multi-module robot
integrating a grasping tool
The ARAKNES Project has received funding from the European Community's Seventh Framework Programme (FP7/2007-2013) under grant agreement num. 224565.
Storage -Structural X8
Once the CNT (naturally magnetic thanks to residuals) are attached or internalized, cells can be concentrated in a desired compartment for subsequent localized therapy.
Magnet
Metastatic
TumorCNT
Cell
CNT can be functionalized to bind target
Cell manipulation with magnetic carbon nanotubes
Metastatic cells bound with CNTs
CNT can be functionalized to bind target cells (such as metastatic cells) or to be internalized by the cells; in this sense cells become magnetotactic and can be drag and collected by a permanent magnet.
Human Neuroblastoma cells (SH-SY5Y)displacement after 3 days in culture withMWNTs-modified medium. Control samplenot showed (with Nikon TE2000U invertedoptical microscope, magnifications 20x).
V. Pensabene, O. Vittorio, S. Raffa, A. Menciassi, P. Dario, “Neuroblastoma cells displacement by magnetic carbon nanotubes”, IEEE Trans. On NanoBioScience, 2008.
Outline of the talk
� Introduction to Biorobotics and biomechatronics
� Biorobotics Science
� Building robots to investigate humans and animaland animal
� Biorobotics Engineering
� Robotics for surgery and endoscopy
� Robotics in rehabilitation and assistance
� Conclusions
Stimulation of
the sensory nerves to
provides
sensory
A “cybernetic”
prosthesis
controlled by
the brain
Extraction of
brain commands
from the motor nerves
sensory
feedback
neuralProsthetic hands and neural
The case of the cybernetic hand prosthesis
Neuroscience
Scientific and
technological results
FP5 & FP6
Prosthetic hands and neural
interfaces
CyberHand:
a robotic hand and neural
peripheral interfaces
implanted in a first human
patient (2008)
CYBERHAND
(FP5, 2002-2005)
Cybernetic
prosthetic hand
Life-like
perception
systems
NEUROBOTICS IP
(FP6, 2004-2008)
Hybrid Bionic
Systems
Beyond
Robotics
Mechanisms Sensors
External World Bio
mech
atro
nic
Mechatronic System
Tele
co
mm
un
icatio
n
“User/Patient”
“Mechatronic” (integrated)
Our philosophy: Human-Centred Design
for devices, systems and services
Power Supply
Actuators Embedded Control
Human-Machine Interface
Human User
Bio
mech
atro
nic
Syste
m In
teg
ratio
n
Tele
co
mm
un
icatio
n
(integrated) platform
“User/Therapist”
CYBERHAND Project: Development of a CYBERnetic HAND prosthesis
Electrodes for Recording and Stimulation in the PNS
Sieve ElectrodeIntegrated Electronics for Active Sieve Electrode
Sieve silicon electrode
Sieve Head with Counter Electrodes
Shaft Electrode
Platinum Electrodes on the Shaft Tripolar Cuff Electrodes
LIFE Electrodes
Information Society Technologies
SSSA prosthetic hands
MECHATRONIC HANDS FOR
CYBERHAND robotic hand• A robust stand-alone robotic hand provided with artificial sensors
• It will be implantanted in Rome• 5 fingers, 6 motors, 16 dof, 11 sensors (encoders and cable tension), extrinsic electronics and motors.
Neurobotics - The fusion of Neuroscience and Robotics, FP6-IST-001917 (www.neurobotics.info). A project funded by the Future and Emerging Technologies arm of the IST programme
Future and Emerging Technologies
HANDS FOR PROSTHETICS @ SSSA
SMARTHAND robotic hand• Research prosthetic hand• 5 fingers, 4 motors, 16 dof, 40 sensors, and electronic units integrated in the hand
Information Society Technologies
Clinical experimental set-up
Development of a robust prosthetic hand provided with an advanced sensory system
– The development of new PNS neural interfaces
– The short-term implant in humans of the tfLIFEs for the control of the hand prosthesis
Collaboration
between SSSA,
UCBM, IBMT, UAB
Neurobotics - The fusion of Neuroscience and Robotics, FP6-IST-001917 (www.neurobotics.info). A project funded by the Future and Emerging Technologies arm of the IST programme
Future and Emerging Technologies
CYBERHAND robotic hand
Commercial stimulation and recording system
Efferent processing, control, afferent stimulation (on a PC platform)
LIFE electrodeswith a transcutaneoustranscutaneousconnection
Cybernetic TransradialHand Design
CyberHand and NEUROBOTICS : a robotic hand and neural peripheral interfaces for human prosthetics
Scheme of the shortScheme of the short--term implant term implant of the CYBERHAND systemof the CYBERHAND system
tfLIFE electrodeswith a
transcutaneoustranscutaneousconnection
CYBERHAND prosthesis
Recording and Stimulating Circuitry
(outside the outside the body of the body of the subjectsubject)
Efferent processing, control, afferent
stimulation (on a PC platform)
How can How can robotics robotics technology technology contribute?contribute?
Which Which advantages?advantages?
The Rehabilitation ProcessThe Rehabilitation ProcessFunctionalAssessment
Functional Recovery
FunctionalSubstitution
Functional Surgery
Motor Therapy
advantages?advantages?
Which are the Which are the current current challenges?challenges?Assistive
devicesProfessional Training
Assessment of Residual Abilities
Reintegration into social life and working activity
Two classes of
rehabilitation machines for
robot-mediated therapy
� Systems for physical therapy therapy
� Systems for neuro-rehabilitation
Neurophysiological basis for
neurorehabilitation after stroke
� Brain motor areas which are “not used” but can generate upper limb movements if properly stimulated(Kwan, 1978)
� The same motor function can be activated by multiple (different, non contiguous) brain motor areas (Humprey, 1986, Sato e Tanji, 1989, Huntley & Jones, 1991) Tanji, 1989, Huntley & Jones, 1991)
� Multiple representations of the cortico-spinal output have been shown in the motor cortex. These representations are related to different motor functions and can present several spatial and temporal overlaps (Sanes, Donoghue et al., 1995) Rijntjes et al., J Neurosci, 1999
This situation proves the flexibility of the motor output organization and is a key issue to promote functional recovery after strokeMotor learning and plasticity can be exploited for neurorehabilitation therapy, i. e. to promote functional recovery
MIT-MANUS system: clinical trials at
ASL12
Prof. A. Battaglia, Dr. F. PosteraroReparto di Medicina Riabilitativa - Centro di Alta Specialità per la Riabilitazione dei Traumi Cranici e delle Gravi Cerebrolesioni Acquisite
Clinical Validation of the MEMOS systemClinical Validation of the MEMOS system
Starting position
Clinical trials (2003 – 2004) at Fondazione Maugeri, Veruno (Italy) –
Prof. Fabrizio Pisano, Division of NeurologyP1(X1,Y1)
P2(X2,Y2)
Final position
Colombo et al, 2004Micera et al., 2004
Outline of the talk
� Introduction to Biorobotics and biomechatronics
� Biorobotics Science
� Building robots to investigate humans and animaland animal
� Biorobotics Engineering
� Robotics for surgery and endoscopy
� Robotics in rehabiliation and assistance
� Conclusions
Conclusions
� The biological domain is particularly suitable (although not the only one) for exploring the potential of systematic collaboration between robotics and science
� Robotics can provide tools useful for scientific investigation and discovery, but it can be as well a very attractive research area for discovering basic principles underlying the functioning of living beingsprinciples underlying the functioning of living beings
� Medical applications of robots are increasingly important (for research and education) and successful (for industry and in clinical practice)
An industrial product deriving (2005) from FET research (2001-4): 100.000+ robots sold!
PALOMA EU IST-FET Project IST-2001-33073
Distributed by De Agostini
SpA in 8 countries
Conclusions
� The biological domain is particularly suitable (although not the only one) for exploring the potential of systematic collaboration between robotics and science
� Robotics can provide tools useful for scientific investigation and discovery, but it can be as well a very attractive research area for discovering basic principles underlying the functioning of living beingsprinciples underlying the functioning of living beings
� Medical applications of robots are increasingly important (for research and education) and successful (for industry and in clinical practice)
� Robotics and the “grand challenges” it poses are an extraordinary means to attract, motivate, educate and train many talented and enthusiastic young students
Thank you for your attention