1 Evaluation and Modeling of Learning Effects on Control of Skilled Movements through Impedance Regulation and Model Predictive Control By: Mohammad Darainy.

1

Evaluation and Modeling of Learning Effects on Control of Skilled Movements through Impedance

Regulation and Model Predictive Control

By: Mohammad Darainy

Supervisor: Dr. Towhidkhah

Advisors: Dr. Rostami, Dr. Ostry

22

Contents

Introduction

Control of Human Arm Impedance in Statics

Transfer and Durability of Acquired Patterns of Human Arm Stiffness

The Role of Phasic Muscle Activity on Hand Stiffness during Reaching Movements

Modeling of Human Motor Control System through Impedance control and Model Predictive Control

33

What is The Joint Impedance?

The Automatic Capability of the System to Resist an Applied Load before Voluntary Intervention Takes Place (Winters et al. 1988)

Stiffness is defined as the ratio of change in force or torque to change in length or angle

Viscosity is defined as the ratio of change in force or torque to change in velocity or angular velocity

44

Which Parameters Contribute to the Joint Impedance?

Musculoskeletal Geometry has a Great Impact on Joint Impedance

55

Mussa-Ivaldi et al. 1985

Roughly Speaking, The Axis of Maximum stiffness is a Line that Connect The Hand to a Point between Shoulder and Half Way Through the Elbow

66

Which Parameters Contribute to the Joint Impedance?

Musculoskeletal Geometry has a Great Impact on Joint Impedance

Level of Muscle Co-Contraction

77

Adapted From Gomi & Osu 1998

88

Which Parameters Contribute to the Joint Impedance?

Musculoskeletal Geometry has a Great Impact on Joint Impedance

Level of Muscle Co-Contraction

Reciprocal Activation of Agonist – Antagonist Muscles (Hunter & Kearney et al. 1982)

Stretch Reflex Gain (Bennett 1994)

99

What is Impedance Control?

The Idea that the Nervous System Can Modify the Mechanical Behavior of the Neuromuscular Periphery, Known as Impedance Control Theory (Hogan 1985)

Empirical Evidence:

It was Shown That Subject Increase the Level of Co-Contraction in Proportion to The Level of Destabilizing Load (Milner & Cloutier 1998)

During Two-Joints Movement, It was Shown That The Subject Learn To Stabilize Unstable Dynamics Using Selective Control of Impedance Geometry. (Burdet et al. 2001)

1010

Burdet et al. 2001

Co-Contraction Increase The Mechanical Stability of

The Joint But at Greater Metabolic Cost

1111

The Focus of This Thesis

The Effect of Learning on Joint Stiffness.

The Coordinate System under which Impedance Control will be Learned and Generalized.

The Impedance Control Limitation, Both in Statics and Dynamics.

To Develop a Model of Human Motor System Based on the Empirical Studies on Joint Impedance.

1212

1: Learning to Control Arm Stiffness under Static Conditions

Aim

To Examine The Idea That Impedance Control Involves a Process of Learning That is Acquired Over Time and Permits the Voluntary Control of The Pattern of Stiffness at The Hand.

Prediction

We Predicted That The Subjects Can Voluntarily Control Their Hand Impedance Geometry and With Practice They Can Improve Their Ability of Impedance Control

1313

Experimental SetupExperimental Setup

Interactive Motion Technologies, Cambridge MA

Arm Configuration During The Experiment

1414

Experimental StudyExperimental Study

Subjects trained for three successive days to resist one of three loads

single axis loads in lateral direction.

single axis loads in forward/backward direction.

isotropic loads in eight directions about a circle.

Unpredictable servo-position controlled perturbations used to assess stiffness.

1515

Position Servo Controls Were Used to Estimate The Stiffness

1616

Results

1717

Results

1818

Modeling Results

1919

Conclusion 1Conclusion 1 Stiffness Ellipse Modified With Learning in an Adaptive Fashion, Change in Orientation Matches The Direction of Load.

Modeling Studies Suggest Separate Commands for Shoulder and Elbow Co-Contraction.

Even with Practice, Impedance Change in Statics is Relatively Small in Magnitude. Simulations Suggest This is at a Limit Regardless of Level of Co-Contraction.

Clockwise Rotation of Stiffness Ellipse Greater Than Anti-Clockwise Rotation. Same Asymmetry in Simulations. Asymmetry Due to Geometry Since Simulated Commands Equate for Impedance of Shoulder and Elbow Muscles.

2020

2: Transfer and Durability of Acquired Patterns of Human Arm Stiffness

Aims

To Explore Whether Subjects Could Transfer Stiffness Learning to Other Parts of The Workspace And Also The Specific Pattern of Generalization as a Means to Identify The Coordinate System in Which The Learning Occurs.

To Evaluated The Durability of Stiffness Training. Is The Effect of Learning Persist Well beyond The Time of Initial Learning?

2121

Experimental SetupExperimental Setup

Arm Configuration During The Experiment 1 and 2

2222

Experimental StudyExperimental Study The Tests Conducted With The Arm Stationary.

In Experiment 1 Subjects Were Trained at The Left And Transfer Test Was Conducted at The Right. The Experiment Take Place in One Day.

In Experiment 2 Subjects Were Tested for Three Consecutive Days at The Left. The Direction of The Force Application at The Third Day Was Perpendicular to The Last Two days.

After The Training Phase Subjects Were Randomly Assigned to One of Two Groups.

One Group of Subjects Experienced Similar Torque at The Left And Right (Group J) While for The Other Group The Experienced Force at The Hand Was Similar (Group H).

2323

Results Experiment 1

2424

Results Experiment 1

2525

Result Experiment 2

2626

Conclusion 2Conclusion 2 Experiment 1:

Both Groups of Subjects in The Training Phase of The Experiment Learned Successfully to Modify The Orientation of The stiffness Ellipse.

Transfer of Stiffness Control Was Observed only When The Direction of Load Resulted in Torques That Were Similar to Those Experienced During Training..

Experiment 2:

Subjects in The First Two Days of Experiment Learned to Modify The Orientation of The stiffness Ellipse. The Change in Stiffness Ellipse Orientation Matched The Direction of Force Application

Higher Stiffness Ellipse Size, Longer Time Outside The Target as Well as The Orientation of The Stiffness Ellipse on The Third Days of Experiment Shows That The Stiffness Learning Result in Durable Changes in Neural Signals Underpinning Co-Contraction of Muscles.

2727

3: The Role of Phasic Muscle Activity on Hand Stiffness during Reaching Movements

Aims

To Develop a New Method for Stiffness Estimation During Reaching Movements.

To Explore The Effect of Phasic Muscle Activity On Hand Stiffness During Reaching Movements By Comparing The Static Stiffness with The Stiffness During Movements.

2828

Experimental SetupExperimental Setup

Arm Configuration During The Experiment

2929

Experimental StudyExperimental Study

The Tests Were Conducted in Three Consecutive days

The Stiffness Ellipses Were Estimated in Three Different Conditions: Static, Down to Up and Left to Right Reaching Movements.

The Average of Last Eighty Trials During Reaching Movements and AR Model of Trial By Trial Variability Were Used as a Method for Prediction of Ongoing Movements.

3030

Material and Method

The Result of Position Servo Control in Both X and Y Direction For Reaching Movement in Y Direction.

3131

Results

3232

Results

3333

Conclusion 3

The Result Shows Considerable Counter Clockwise Rotation of Stiffness Ellipse in Compare to The Static Stiffness Ellipse During Down to Up Reaching Movements.

The Rotation of Stiffness Ellipse During Lateral Reaching Movement Wasn’t Statistically Reliable.

Impedance Modification During Reaching Movements in Unstable Environments May Actually Have Limitations That Are Comparable to Those Observed Under Static Conditions. The Large Empirical Difference May be Due Simply to Phasic Muscle Activity During Movement.

34

4: A Model of Motor Control System Based on Impedance and Model Predictive Control

Aims

To Develop a New Model for Motor Control System Based on The Result of Empirical Impedance Measurements During Acquisition of a New Skill.

Try to Model The Effect of Learning on The Joints Impedance.

35

Model Structure

A Combination of Model Based Controller (MPC) and Impedance Controller as a Model of Motor

Control System

36

Simulations Conditions

Reaching Movements of 15cm Toward Each of Eight Equidistance Targets Were Simulated

37

Robot Model

The Model Behavior Evaluated in Curl Field

The Model of Robot Defined as:

In First Step 0 Assigned to K and b=[0 8; -8 0]N/m. This is The Model of a Curl Field.

)( *XXkXbFext

38

Simulation Results

To Control our Arm Perfectly We Not Only Need The Model of Environmental Dynamics But Also We Need to Know The Optimal Impedance of The Joints in Each Situation.

39

Simulation Results

40

Simulation Results

41

Simulation Results

42

Simulation Results

43

Simulation Results

The Model of Robot Defined as:

In This Case 0 Assigned to b and k=[150 0; 0 0]N/m. This is The Model of a Divergent Force Field.

)( *XXkXbFext

44

Simulation Results

The Controller Doesn’t Benefit too much from Having The Model of Disturbance.

45

Conclusion 4

To do a Skilled Movement, We Need a Precise Model of Environmental Dynamics Also The Optimal Impedance of Participating Joints are Necessary as well.

The Empirical Evidence Shows that The Subject’s Performance Decay With The Distance from The Location Where The Adaptation Take place. Our Simulation Argue That The Unavailability of Optimal Impedance in Neighboring Location is The Reason for The Above Phenomena.

Adaptation to The Stable And Unstable Dynamics Have Some Fundamental Deference. While The Adaptation to Stable Dynamics Seems to Happen Mostly in Model Based Controller, Adaptation to The Unstable Dynamics Achieved By Impedance Control.

46

General Conclusions

Joint Impedance Can be Modified With Learning in an Adaptive Fashion. Geometrical Changes in Impedance Matches The Direction of Load.

Impedance Control Learned and Generalized in Intrinsic Coordinate System.

A Combination of Model Based Controller With Impedance Controller Can Mimic The Behavior of Human Subjects in Adaptation to Stable and Unstable Dynamics.

Learning Impedance Results in Durable Changes to The Neural Signals That Underlie Impedance Control.

Impedance Control in Dynamics Have The same Limitation That Were Seen in Statics.

47

Contribution to Knowledge

For The First Time, The Effect of Learning on Geometrical Pattern of Hand Stiffness Was evaluated in This Thesis.

The Coordinate System Under Which The Impedance Control Learned and Generalized, Were Explored. These Finding Help us to Understand How Subjects Percept The Environmental Dynamics.

The Empirical Studies Showed That Although The Stiffness Control Under Static Conditions is Limited But Stiffness Learning is Durable and The Effect of Learning Can be Seen Well Beyond The Time of Initial Learning.

48

Contribution to Knowledge

A New Method of Stiffness Estimation During Reaching Movements Developed and Applied.

For The First Time, Both Static and Dynamic Stiffness at The Same Musculoskeletal Geometry were Estimate. The Results Argue That Stiffness Control During Movements May Have The Same Limitation as it Was Shown Under Static Conditions.

A New Model for Motor Control System Were Presented Based on Combination of a Model Based Controller and Impedance Controller.