Vittorio Sanguineti [email protected] Italian Institute of Technology Via Morego 30, 16163 Genova (IT) www.iit.it PRESENTATION: CogSys 2010, Zurich, SUI MOTOR SKILL LEARNING THROUGH PHYSICAL ASSISTANCE CONCEPT The HUMOUR project aims at investigating and developing efficient robot strategies to facilitate the acquisition of motor skills. We address both the (human) trainee and the (robot) trainer sides, by combining behavioural studies on motor learning and its neural correlates with design, implementation, and validation of robot agents that behave as ‘optimal’ trainers, which efficiently exploit structure and plasticity of the human sensorimotor systems. HU man behavioral M odeling for enhancing learning by Optimizing hUman-R obot interaction (HUMOUR): concept and preliminary results 1 Vittorio Sanguineti, 2 Dejan Popovic, 3 Roberto Colombo, 4 Niels Birbaumer, 5 Herbert Heuer, 6 Etienne Burdet 1 Fondazione Istituto Italiano di Tecnologia, Genoa (IT); 2 Aalborg Universitet, Aalborg (DK); 3 ‘Fondazione Salvatore Maugeri Clinica del Lavoro e della Riabilitazione, Pavia (IT); 4 Eberhard-Karls-Universitat,Tubingen (DE); 5 Forschungesellschaft fur Arbeitphysiologie und Arbeitsschutz e.V., Dortmund (DE); 6 Imperial College of Science, Technology and Medicine, London (UK) GENERAL OBJECTIVES Modular robot system: 2-arm planar manipulandum 3D wrist device 1D hand device Common software platform: Device-independent Multiple devices Multi-OS (Windows, Linux) Open Source, GPL (built on top of H3D, www.h3d.org by SenseGraphics) Analytic and modeling tools Performance evaluation Optimal schemes of assistance Model of robot-assisted motor recovery after stroke Robot-assisted motor skill learning: Redundant tasks (putting, handwriting transfer) – Poster #127 Visuomotor rotation – Poster #114 Neural correlates of assistive force We will focus on the cognitive and neural mechanisms underlying the acquisition of a variety of motor skills, by specifically aiming at understanding the way humans physically cooperate in acquiring a motor skill and how physical assistance affects motor learning. Experiments will enable us to identify determinants and dynamics of the learning process in representative motor tasks, and will provide the foundations for designing efficient schemes of assistance. 1. Robot trainers. To develop robot agents based on an advanced understanding of human neuromotor control, its development, and skill acquisition, which will enable them to mimic (and surpass) human trainers in supporting motor skill learning and neuromotor rehabilitation 2. Application to motor skill learning and rehabilitation. To validate robot training agents for a number of different motor skills, modalities of interaction and rehabilitation applications 3. Brain-computer interfaces. To extend the domain of Brain-Computer Interface (BCI) technologies to the fields of motor learning and neuromotor rehabilitation 4. Behavioural studies on physical interaction and motor learning. To understand how physical interactions affect motor learning and – on this basis - to develop a general theoretical and technological framework for more effective motor skill learning Movement ASSISTANCE MODULE ACTION MODULE Assistance Movement Subgoals CONTROLLER BODY INTERNAL MODEL TASK MODEL TRAINEE ROBOT TRAINER Neural correlates PERFORMANCE METRIC Performance Performance CONTROLLER Assistance TRAINEE LEARNING Estimated Volitional Control Desired Volitional Control REGULATION OF ASSISTANCE TRAINEE LEARNING MODEL RESULTS (first year) www.humourproject.eu Movement ASSISTANCE MODULE ACTION MODULE Assistance Movement Subgoals <Scene DEF="world"> <!— THE DEVICE (robot end effector and attached tool--> <Inline load="true" url="default.x3d"/> <Inline DEF="device" load="true" url="deviceinfo.x3d"/> <IMPORT inlineDEF="device" exportedDEF="HDEV" AS="HDEV"/> … <!— THE TARGET--> <Transform DEF='target' rotation='1 0 0 1.507'> <Shape> <Appearance> <Material DEF='tgtColor' ambientIntensity='0.1' diffuseColor='1 1 1' shininess='0'/> </Appearance> <Sphere radius='0.01'/> </Shape> </Transform> <!—A SOUND (used as cue or feedback) --> <Sound> <AudioClip DEF="Ask" url="ding.wav"/> </Sound> … </Scene> <automaton> <!-- STATE LIST--> <state id="0" name="init" type="start"/> <state id="1" name="moving"/> <state id="2" name="stopped"/> <state id="3" name="feedback"/> <state id="4" name="back"/> <!-- STATE TRANSITIONS --> <transition from="init" to="moving"> <gt type="double" field="velocity" value="0.05"/> </transition> <transition from="moving" to="stopped"> <and> <timer delay="1"/> <lt type="double" field="velocity" value="0.05"/> <lt type="double" field="distance" value="0.01"/> </and> </transition> <transition from="stopped" to="feedback"> <timer delay=".1"/> </transition> <transition from="feedback" to="back"> <timer delay="1"/> </transition> <transition from="back" to="init"> <and> <timer delay="2"/> <lt type="double" field="distance_from_center" value="0.01"/> </and> </transition> </automaton> WORLD MODEL ACTION MODEL 500 1000 1500 0 2 4 w i 500 1000 1500 0 0.2 0.4 y i 500 1000 1500 0 0.2 0.4 y i est 500 1000 1500 -0.1 0 0.1 x i Trial Number 200 400 600 800 1000 1200 0 2 4 6 8 w i 200 400 600 800 1000 1200 0 0.2 0.4 y i 200 400 600 800 1000 1200 0 0.2 0.4 y i est 200 400 600 800 1000 1200 -0.2 0 0.2 x i Trial Number <PositionEffect r0="p REFx p REFy p REFz " K="K p11 K p12 K p13 K p21 K p22 K p23 K p31 K p32 K p33 "/> <VelocityEffect v0="v REFx v REFy v REFz ” B="K v11 K v12 K v13 K v21 K v22 K v23 K v31 K v32 K v33 "/> ASSISTANCE 0 20 40 60 80 100 120 0 10 20 30 40 0 5 10 15 20 25 30 35 0 10 20 30 40 0 0.5 1 1.5 2 2.5 0 10 20 30 40 0 20 40 60 80 100 0 10 20 30 40 Efficacy Velocity Accuracy Efficiecy Session # Session # Session # Session # Performance Metrics