Optimality in Motor Control By : Shahab Vahdat Seminar of Human Motor Control Spring 2007
Jan 19, 2016
Optimality in
Motor Control
By : Shahab VahdatSeminar of Human Motor
ControlSpring 2007
Agenda Optimal Estimation
Optimal Control
Proposed Model
Optimality Wolpert, D. M., Ghahramani, Z. & Jordan, M. I. An
internal model for sensorimotor integration. Science (1995).
Van Beers RJ, Baraduc P & Wolpert DM. Role of uncertainty in sensorimotor control. Transactions of the Royal Society (2002)
Emanuel Todorov. Optimality principles in sensorimotor control. NatureNeuroscience (2004)
Emanuel Todorov Optimal Control Theory. Bayesian Brain, Doya, K. (ed), MIT Press (2006)
Kalman Filter
kkkkkk wuBcFc 1
kkkk vcHy
State-space model is described with these equations:
The prediction step consists of two calculations:
State estimate propagation:
Kalman Filter
1)1|( ˆ.ˆ kkkk cFc
Error covariance propagation:
kTkkk QFPF
1kP
The updating step consists of three calculations
Kalman Filter
Kalman gain matrix:
State estimate update:
Error covariance update:
1kk )P(P k
Tkk
Tkk RHHHG
kkkkk IGcc )1|(ˆˆ)1|(ˆ. kkkkk cHyI
kkkkk PHGPP
For controlling a goal-directed arm movement, there are three sources of noise :
(i) noise in the sensory signals that limits perception, (ii) noise in the motor commands, leading to inaccurate
movements (iii) sensorimotor noise, which origins from inaccuracies in the
forward model and causes noisy predicted location of the body during movement.
Therefore, the time-varying Kalman gain is used for minimizing the effect of these noises and uncertainty in the overall estimate.
Sources of Noise
Kalman Filter: Sensorimotor Integration
Sensorimotor Integration
When we move our arm in darkness, we may estimate the position of our hand based on three sources of information:
• proprioceptive feedback.
• a forward model of how the motor commands have moved our arm.
• by combining our prediction from the forward model with actual proprioceptive feedback.
Experimental procedures:
Subject holds a robotic arm in total darkness. The hand is briefly illuminated. An arrow is displayed to left or right, showing which way to move the hand. In some cases, the robot produces a constant force that assists or resists the movement. The subject slowly moves the hand until a tone is sounded. They use the other hand to move a mouse cursor to show where they think their hand is located.
Experiment: Sensorimotor Integration
Optimal Control
Bellman equations:
Continuous control:
Hamilton-Jacobi-Bellman equations:
Deterministic control: Pontryagin’s maximum principle
Hamiltonian:
Linear-quadratic-Gaussian control:
Riccati equation:
Optimal control as a theory of biological movement
state equations:
Optimal control as a theory of biological movement
Open-Loop versus Close-Loop Optimal Controller
Feed forward optimality models explain some of the classical motor properties (bell shaped profiles, etc) Harris & Wolpert, 1998- Min. Variance Flash and Hogan, 1985- Min. Jerk
Task constraints and motor noise combine to determine optimal motor plans
Humans use continuous visual feedback Noise in the sensory system very accurately predicts how
people use feedback Task constraints may also impact feedback control
strategies
Schematic illustration of open- and closed-loop optimization. (a) The optimization phase, which corresponds to planning or learning, starts with a specification of the task goal and the initial state. Both approaches yield a feedback control law, but in the case of open-loop optimization, the feedback portion of the control law is predefined and not adapted to the task.
(b) Either feedback controller can be used online to execute movements, although controller 2 will generally yield better performance. The estimator needs an efference copy of recent motor commands in order to compensate for sensory delays. Note that the estimator and controller are in a loop; thus they can continue to generate time-varying commands even if sensory feedback becomes unavailable. Noise is typically modeled as a property of the sensorimotor periphery, although a significant portion of it may originate in the nervous system.
Proposed Model:
Optimal Primitive State Prediction
k
iii tqqct
1
),,()(
),,()( qqqDt
k
iii tqqcqqqD
1
),,(),,(
Force fields as primitives for internal models:
Proposed Model:
Optimal Primitive State Prediction
Estimation and Control Equations:
)...( )1|()1|()|( kkkkkkkkk CHYGCC
..)|()|1( kkkkkk UBCC
Proposed Model:
Optimal Primitive State Prediction
Proposed Model:
Optimal Primitive State Prediction
Primitive modular representation of the cerebellum