What Robo)cists Need to Know About NeuroMusculoSkeletal Systems Gerald E. Loeb, M.D. Professor of Biomedical Engineering Director of the Medical Device Development Facility University of Southern California
What Robo)cists Need to Know About NeuroMusculoSkeletal Systems
Gerald E. Loeb, M.D. Professor of Biomedical Engineering
Director of the Medical Device Development Facility University of Southern California
Gerald E. Loeb • Background
– 1965-‐1973 B.A., M.D. Johns Hopkins University, internship in surgery – 1973-‐1988 Lab. Of Neural Control, NaQonal InsQtutes of Health – 1988-‐1999 Professor of Physiology, Queen’s Univ., Canada – 1994-‐1999 Chief ScienQst, Advanced Bionics Corp., Los Angeles – 1999-‐present Professor of Biomedical Engineering, Director of the Medical
Device Development Facility, Univ. Southern California – 2008-‐present CEO, SynTouch LLC, Los Angeles
• Research Interests – Sensorimotor neurophysiology and control – BiomimeQc prostheQc and roboQc systems
• Issues in Impedance Control – Biological components (actuators, sensors, feedback loops) are much more
nonlinear, noisy and slow than their mechatronic counterparts, yet they perform much be]er on unpredictable tasks and a baby can learn to use them.
– We need to understand why if we are going to repair or replace them.
InvesQgaQng the role of muscle physiology and spinal circuitry in movement
Current and recent collaborators: Research Asst.Professor: Dr. Rahman Davoodi Post-‐doctoral fellows: Giby Raphael, Yao Li Graduate students: George Tsianos, Cedric RusQn, Jared Goodner, Katherine Quigley Undergraduates: Norman Li, Travis Marziani, Adam Baybu], Enrique Morales, Michelle Fung
CorQcospinal pathway
Other descending pathways
Motor neurons
Muscle spindle
Flexor muscle
Extensor muscle
Kandel et al., 2000
Muscle ≠ Torque Motor
Main Components Involved in Human Arm Movement
Muscle Excitation
Muscle Force
Neural Controller
Sensory Feedback
Movement
Muscle Actuator
Skeletal System
Feedback Sensors
Actuator ProperQes: Underlying Mechanisms
• Force – Length: Myofilament overlap • Force – Velocity: Cross-‐bridge dynamics
• Force – Frequency: Calcium kineQcs
• Moment – Angle: Tendon path
• ElasQc Storage: Collagen ultrastructure • Energy Transfer: MulQarQcular muscles
• Impedance Control: CocontracQon
Muscle accounts for the majority of animal mass and energy consump)on. The evolu)onary pressures to find and exploit any advantage are huge.
Complex Moment Arms at the Human Elbow
Biological ElasQcity: Nonlinear Adap)ve
Muscle ≠ Torque Motor
Virtual Muscle, based on ~20 experimental and modeling papers with Steve Sco], Ian Brown, Milana Mileusnic, George Tsianos, et al.
Available from h]p://bme.usc.edu/gloeb
Actuator ProperQes: Underlying Mechanisms
• Force – Length: Myofilament overlap • Force – Velocity: Cross-‐bridge dynamics
• Force – Frequency: Calcium kineQcs
• Moment – Angle: Tendon path
• ElasQc Storage: Collagen ultrastructure • Energy Transfer: MulQarQcular muscles
• Impedance Control: CocontracQon
Muscle accounts for the majority of animal mass and energy consump)on. The evolu)onary pressures to find and exploit any advantage are huge.
Source: Kandel ER, Schwartz JH, Jessel TM. Principles of Neural Science. 4th ed. New York (NY): McGraw-‐Hill; c2000. Figures 34-‐2a,3a; p.680
In order to produce acQve force: • Myofilaments overlap • Myosin heads cocked • Binding sites exposed
0 50 100 1500.0
0.5
1.0
dispot.
pot.
T
F
0.8 1.0 1.2 1.40.0
0.2
0.4
L
F
dispot.
pot.
0.0
0.2
0.4
0.6
0.8
1.0 SOLmodelCFmodeldata
Force(F0)
Whole-muscle length (cm)
35 pps10 pps5 pps
3 pps
Frequency
…and other nonlinear properties of muscle actuators
Myosin Head
Potentiation Myosin light-chain phosphorylation
shifts resting position of myosin heads
Thin Filaments
Myosin Heads
Ls = 3.65µm Ls = 2.10µm
Light Chain
Cocked myosin head Exposed ac2n site
Source: Kandel ER, Schwartz JH, Jessel TM. Principles of Neural Science. 4th ed. New York (NY): McGraw-‐Hill; c2000. Figures 34-‐2a,3a; p.680
ATP required to release myosin head and move it to the cocked state
Exb depends on the number of cross-‐bridges and shortening velocity
AcQvaQon requires acQon potenQals and calcium release; ATP used to pump ions back.
Ea depends on calcium flux in the sarcoplasm
In order to produce acQve force: • Myofilaments overlap • Myosin heads cocked • Binding sites exposed
Erec ATP and PCr replenishment results in delayed recovery heat
Virtual MuscleTM overview at the motor unit level
Note: the continuous recruitment algorithm lumps all motor units within a fiber type into one.
Erec = R*Einitial + =
Recovery energy rate
Total energy rate
Virtual MuscleTM
Cheng et. al. 2000 Song et. al. 2008
Tsianos et al. in press IEEE-TNSRE
-6 -5 -4 -3 -2 -1 0 1 2 30
0.5
1
1.5
Vce (L0/s)
Forc
e (F
0)
20%
40%
60%
80%
100%
20% 0
50
100
150
Ener
gy R
ate
(W)
40%
0
50
100
150
Ener
gy R
ate
(W)
60%
0
50
100
150
Ener
gy R
ate
(W) 80%
0
50
100
150
Ener
gy R
ate
(W)
100%
0
50
100
150
Ener
gy R
ate
(W)
Exb Eca
0
50
100
150
Ener
gy R
ate
(W)
Virtual Muscle (analogous to Biceps Long)
40% Slow twitch 60% Fast twitch Mass = 300g F0 = 600N L0 = 16cm
ValidaQon of EnergeQcs Model
(Andersen et al., 1985; Gonzalez-‐Alonso et al., 2000)
0 50 100 1500
50
100
150
200
250
300
time (s)
Ener
gy ra
te (W
)
ExperimentModelEinitialErecovery
Total energy (J)
Exp: 31,650-‐44,769 Model: 39,081
Dynamic knee extension
Task
Model
Energe2cs
Model System to Study Preflexes
Examples of:
Mono-articular muscles
Bi-articular muscles
Initial armposition
Arm positionafter 50 ms.
Trajectory ofhand trackedevery 10 ms.
Gun reaction forceapplied to hand.
level of activationindicated bymuscle width
Passivemuscle
Sample Simulation (50 ms):
Activemuscle
AMI-USC
The Effectiveness of Preflexes INTRINSICPROPERTIES
OFMUSCLE
MUSCLE ACTIVATION PATTERNS
ConstantTorque
Force-Length
Force-Length&
Force-Velocity
Sensing & Control: Underlying Mechanisms
• ResoluQon: PopulaQons of receptors • Dynamic Range: PredicQve gain control • Signal Processing: IntegraQve • Feedback: MulQmodal convergent/divergent • Control: Programmable regulator (MIMO) Fast & accurate movement is what dis)nguishes even primi)ve animals from plants. Servocontrol is too limited for mul)ar)cular organisms and inverse models and op)mal control are not feasible analy)cally.
Animals use hierarchical and “good enough” controls.
AMI-USC
Muscles have hundreds of proprioceptive sense organs
Spindles are very complex and provide most
sense of posture and movement
Fusimotor System as Optimal Sensor Control
WB Marks, Appendix: Spindle Transduction Properties in GE Loeb (1984) The Control and Responses of Mammalian Muscle Spindles During Normally Executed Motor Tasks, Exer. & Sport Sci. Revs. 12:157-204.
Textbook Robotics & Biology
Goal State
Controller
Regulator
Actuators
MotorCortex
Premotor Extrapyramidal
Movement Stability
Actuatorsfeedback
feedforwardcorrections
spinalinterneurons
α α
sensors
Biomimetic Hierarchical Control
MotorCortex
spinalinterneurons
Task Planning
Plant
ProgrammableRegulator
AdaptiveController
Goal
α α
preflexes
“teacher”
reflexes
Biomechanical Model Simplified 2-‐axis, 4-‐muscle, Wrist Joint Controller
Raphael, G., Tsianos, G.A. and Loeb, G.E. Spinal-‐like regulator facilitates control of a two degree-‐of-‐freedom wrist. J. Neuroscience 30:9431-‐9444, July, 2010.
Classical spinal circuits Stretch reflex and Ia inhibitory
Propriospinal
Renshaw
Ib
Par2al view of the Integrated Spinal Cord Model “True-‐Antagonists”
Extensor Carpi Ulnaris Muscle
Flexor Carpi Radialis Muscle
IaIa
RRIbIb
PNPN
αα
GO
Gam
ma
Sta
tic
G
amm
a D
ynam
ic
s
s
ss
s
s
s
s
s s s
s
s
s s
s
ss
ss
s
ss
ss s
SETSET SET SET SETSETSETSET
SET
SET
SET
SET
GOGO GO GO GO GO GO
SET
SET
ss
Excitatory Synapse
Inhibitory Synapse
Selective Synapse
s
s
Modeled Pathways 1. Propriospinal 2. MonosynapQc Ia 3. Reciprocal Ia 4. Renshaw 5. Ib inhibitory
Par2al view of the Integrated Spinal Cord Model “Par2al-‐Synergists”
Extensor Carpi Ulnaris Muscle
Extensor Carpi Radialis Muscle
IaIa
RRIbIb
PNPN
αα
GO
Gam
ma
Sta
tic
G
amm
a D
ynam
ic
s
s
ss
s
s
s
s
s s s
s
s
s s
s
ss
ss
ss
s
s
s
ssss
ss
ss s
SETSET SET SET SETSETSETSET
SET
SET
SET
SET
GOGO GO GO GO GO GO
SET
SET
ss
Excitatory Synapse
Inhibitory Synapse
Selective Synapse
Modeled Pathways 1. Propriospinal 2. MonosynapQc Ia 3. Reciprocal Ia 4. Renshaw 5. Ib inhibitory
Computa2onal Model of the Interneuron
Input
Output
Command Space = 184 dimensions
Constant Inputs SET Inputs GO Inputs
• PresynapQc InhibiQon
• 140 Neural pathway gains • 8 Fusimotor inputs • Bias to 16 Interneurons &
4 Motoneurons
• Step input to 16 Interneurons
GO
SET
0 1 2 3s
Random Gradient Descent Op2miza2on
Cost Func2on :
Yes = 1 iteraQon
No
Task 1: Stabilizing response to external force perturba2on
Angle
500
500
0
Ext
FlexExt 0.5
0.5
0
Flex
αOutput
Ext 30N
30N
0
Flex
Muscle Force
Angle
500
500
0
Ext
Flex
Time (sec)
Perturba2on response with and without SLR
Extensor [RED], Flexor [BLUE] muscle force
Perturba2on response a^er Gradient Descent
Extensor [RED], Flexor [BLUE] motoneuron o/p
Liles S L 1985 J Neurophysiology
AcQvity of neurons in putamen during acQve and passive movements of wrist
Time (sec)
Angle
500
500
0
Ext
FlexExt 50N
50N
0
Flex
Muscle Force
Angle
500
500
0
Ext
FlexExt 0.8
0.8
0
Flex
αOutput
Task 2: Rapid voluntary movement to a posi2on target
Extensor [RED], Flexor [BLUE] motoneuron o/p
Extension a^er Gradient Descent op2miza2on
Extensor [RED], Flexor [BLUE] muscle force
Extension to 35° and response to perturba2on
Liles S L 1985 J Neurophysiology
AcQvity of neurons in putamen during acQve and passive movements of wrist
Forc
e (N
)
100
0
Forc
e (N
)
100
0Ext +0.8
+0.8
0
0
Time (s)
Flex
α
α
Ext +0.8
+0.8Flex
Step force produced a^er Gradient Descent
Pulse force produced a^er Gradient Descent
(Brief force rise 2me)
(Long force rise 2me)
Ghez C and Gordon J 1987 Trajectory control in targeted force impulses I. Role of opposing muscles Exp. Brain Res. 67 225-‐240
Task 3: Voluntary isometric force to a target level
Task 4: Adapta2on to viscous curl force fields Experiments: Kluzik 2008, Scheidt 2001; 2000; Diedrichsen 2005; Flanagan
1999; Hwang 2005; Karniel 2002, etc!
Time (sec)
Angle
500
500
0
Ext
Flex
Angle
500
500
0
Ext
Flex
Angle
500
500
0
Rad
Uln
Angle
500
500
0
Rad
Uln
Op2mized: Extension-‐Flexion axis
Op2mized : Radio-‐Ulnar devia2on axis
A^er-‐effects: Extension-‐Flexion axis
A^er-‐effects: Radio-‐Ulnar devia2on axis
Scheidt et.al, 2001
Learning Curves Typical learning curves for all tasks
Random starQng condiQons for rapid movement
Analysis of gain values Conclusion: many local
“good enough” minima
Comparison with Servo-‐Control
TRUE-‐ ANTAGONIST
Ib
Ia
GOGolgi Tendon Organ
Muscle Spindle
gamma-‐ dynamic
gamma-‐ static
12 Local projections4 Descending commands8 Gamma commands
24 Gains
1.00E-‐13 1.00E-‐11 1.00E-‐09 1.00E-‐07 1.00E-‐05 1.00E-‐03 1.00E-‐01
Perturbation
Extension
Force Step
Force Pulse
Viscous Curl
Cocontraction (log scale)
Spinal cord model Servo-control modelc∫ (αEUαFR + αERαFU) dt
Cost (log scale)
Spinal cord model
Servo-control model
Variability in experimental
data
Singleexperimental
datum
Perturbation
Extension
Force Step
Force Pulse
Viscous Curl
∫ (state* -‐ state)2 dt
SLR Controller for Planar Elbow-‐Shoulder Musculoskeletal System
Learning to Resist Sudden
Perturbing Force
100N x 10ms
BioSearch™ CorQcospinal Learning Algorithm
delta gain1
delta gain2
delta gain3
…
delta gain400
gain1
gain2
gain3
…
gain4
Simula2on
Hypothesis: Landscape has so many “good enough” local minima that a Random Walk is a viable learning process
Prob.
ΔC* ΔC
.
?ΔC
ΔC*
. ?
?
?
Steps vs. Learning Curves OpQmal step size Delta gain distribuQons
Cost
Cost
sigm
a
delta
Task:
SET the gains of the SLR to resist an impulsive perturbation at the endpoint.
Simple Dyn. Prog.
Cost with no muscular action
Good-enough performance
Learning curves
Jared Goodner, George Tsianos, Yao Li
Simple Dyn. Prog.
Cost with no muscular action
Good-enough performance
Learning curves Convergence rate
Jared Goodner, George Tsianos, Yao Li
Hierarchical Control Updated
world
limb
muscles
spinal cord
“motor” Cx
“premotor” Cx
mechanics
preflexes
segmental reflexes
transcorQcal reflexes
??
regulatory func)on
control func)on
feedback func)on
consolidaQon ?
Each control stage operates on a lower stage whose local feedback and intrinsic properties constitute a regulator that is programmed by its controller.
Anthropomorphic Design o Copy as many details of biological constructs as possible.
o Hope that the machine does something useful.
Biomime2c Design ü Identify utility of each biological design feature.
ü Understand principle of operation of the design feature.
ü Build a machine based on that principle of operation.
ü Demonstrate human-like capabilities enabled by that design feature.
BRAIN
SPINALCORD
MUSCLES
SKELETON
Cortico-motoneuronalprojections
Valid Analogy?
Theory of Computation
for the Spinal Cord
TLaTLo
BrECRL
Incision 1
Incision 2
Lateral view
EMG recordings & muscle models
BLBS
De
TMe
Incision 3
Task: align to targets
Medial view
70
110
α
Manipulandum
Target lights
Torque PulsePerturbations
Using a Musculoskeletal Model
To Interpret Motor Control Strategies
In a Forearm Pointing Task
Cheng & Loeb (in prep.); Cheng, Brown & Loeb (2000)
Virtual Muscle, J. Neurosci. Meth. 101:117-130
http://ami.usc.edu (Projects, BION)
∫ Integralkinematics
Unper turbedEMG activity
Gain
VirtualMuscle
Angular posit ion
?
With and withoutTorque pulse
? Torque
Quantifying intrinsicmuscle resistanceto perturbations
∫ Integralkinematics
Gain
VirtualMuscle
Angular position
?
UnperturbedEMG activity
Recordedkinematics
Torque
Calibration of model
Unper turbed versusreflex EMG activity
Gain
Vir tualMuscle
Torque
?
Recordedkinematics
Quantifying reflexeffects of individualmuscles
∫ Integralkinematics
Unperturbed versusreflex EMG activity
Gain
VirtualMuscle
Angular posit ion
?
Torque pulse
Torque
Effectiveness ofreflexes in controllingperturbations
Not all synergists are created equal.
BL
BrBSBL
BSBr
Flexion Extension
Torque(Nm)
Time (ms)
AccelAccel
Agonistepoch
Antagonistepoch
Agonist:Antagonist
EMGRatio
Session number
BS
Br
BL
Relative kinematic
independence: Lf(cm)/MA(cm)
6.6/2.5 = 2.6
11.1/3.5 = 3.2
5.4/3.2 = 1.7
Muscles are smart.
-200 0 300
0
0.8
1.6
Preflex torque
Unperturbed
Perturbed
Time (ms)
Torque(Nm)
Even inertia can be smart
Flexion
Extension
Session number
Assisting
Resisting Unperturbed
Trial number
Peakagonistepochacceleration(deg/s2 )
A
B
C
B C
Impulse ΔΓ· t Momentum J·ω .
But momentum
and preflexes have fixed
costs Session 1
Sess ion 16
Sess ion 34
Error: 3.99o
Error: 3.43o
Error: 4.95o
Sess ion 34
Sess ion 16
Session 1
As siting
Resist ing
Unperturbed
Time (ms)
L ow e ne rgy uti li za ti o n
Me d e ne rg y utili zat i on
Hi g h e ne rg y utili za ti o n
Preflex to rque: 0.055
Preflex to rque: 0.087
Preflex to rque: 0.121
Acceleration(deg/s2 )
RateofEnergy
Consumption(W)
Position(deg)
Use Model to Understand
Reflexes
Resisting
-200 0 3000
15
30ReflexEpoch
-200 0 3000
15
30
Time (ms)
-200 0 3000
15
30
TMeLoEMG(V)
µUnperturbed
Non-reflexEpoch
Non-reflexEpoch
Composite
∫ Integralkinematics
Unper turbedEMG activity
Gain
VirtualMuscle
Angular posit ion
?
With and withoutTorque pulse
? Torque
Quantifying intrinsicmuscle resistanceto perturbations
∫ Integralkinematics
Gain
VirtualMuscle
Angular position
?
UnperturbedEMG activity
Recordedkinematics
Torque
Calibration of model
Unper turbed versusreflex EMG activity
Gain
Vir tualMuscle
Torque
?
Recordedkinematics
Quantifying reflexeffects of individualmuscles
∫ Integralkinematics
Unperturbed versusreflex EMG activity
Gain
VirtualMuscle
Angular posit ion
?
Torque pulse
Torque
Effectiveness ofreflexes in controllingperturbations
∫ Integralkinematics
Unper turbedEMG activity
Gain
VirtualMuscle
Angular posit ion
?
With and withoutTorque pulse
? Torque
Quantifying intrinsicmuscle resistanceto perturbations
∫ Integralkinematics
Gain
VirtualMuscle
Angular position
?
UnperturbedEMG activity
Recordedkinematics
Torque
Calibration of model
Unper turbed versusreflex EMG activity
Gain
Vir tualMuscle
Torque
?
Recordedkinematics
Quantifying reflexeffects of individualmuscles
∫ Integralkinematics
Unperturbed versusreflex EMG activity
Gain
VirtualMuscle
Angular posit ion
?
Torque pulse
Torque
Effectiveness ofreflexes in controllingperturbations
They look like reflexes.
Are they important?
-200 0 300
-0.3
0
0.06
-200 0 300
-0 .3
0
0.06
-200 0 300
-0.08
0
0.4
-200 0 300
-0.08
0
0.4
-200 0 300
-200 0 300
-200 0 300
-200 0 300
Time (ms) Time (ms)
BLBSB rECRB
TM e LoTL aTL oDe Pe rturb ed Un pe r tu rb e d
A. Flexion resisting
BS
TMeL o
B. Flexion assisting
C. Extension resisting
D. Extension assisting
Preflexes alone don’t cut it.
Error w/o reflex: -3.67Error with reflex: -2.24
Error w/o reflex: 3.44Error with reflex: 1.75
Perturbed with reflexPerturbed w/o reflex
Unpe rturbed
Perturbed with refle xPerturbed w/o reflex
UnperturbedA. Flexion resisting
B. Flexion assisting
Torque(Nm)
Position(deg)
Time (ms)
Error w/o reflex: -3.87Error with reflex: -3.80
Error with re flex: 1.34Error w/o reflex: 3.61
C. Extension resisting
D. Extension assisting
Are reflexes selectively controlled?
Assisting
Resisting Unperturbed
0 500 10000
1
2
Reflex
Background
0 500 10000
1
2AverageTMeLoEMGforextensionresistingtrials
duringbackgroundandreflexepoch
Trial number
A
B
MSMS: Main Goals and Features
q Provides interactive tools for modeling musculoskeletal and prosthetic limbs and the task environments
q Simulates the models to predict limb’s movement caused by neural controllers and external forces
q Simulates the models in real-time VR environments with the human or non-human primate subject in the loop
Development since 1999 led by Dr. Rahman Davoodi Medical Device Development Facility, University of Southern California
Challenge: Can we endow mechatronic prosthetic & robotic hands with haptic abilities?
Answer: BiomimeQc
tacQle sensors • Contact with object
deforms skin and fluid, changing electrode
impedances • Heat flux into object
idenQfies material properQes
• Skin sliding over textures generates vibraQon spectra
recorded by pressure sensor