Software Framework For Robot Application Developmenta constraint-based task programming approach
Dominick Vanthienen, TinneDe Laet, Joris De Schutter,
Herman Bruyninckx
Presented by Wilm Decre
Dept. of Mechanical Engineering,KU Leuven
06/05/2013
Introduction Modular Framework Experiments Conclusions
Robot application development
How much work to generate a laser tracing application for twoplatforms? Double as much work as for one?
2 / 26
Introduction Modular Framework Experiments Conclusions
Reuse?
Two things are required for reusability:1. a systematic approach for the specification of tasks2. a modular software framework
3 / 26
Introduction Modular Framework Experiments Conclusions
a systematic approach for task specification
� iTaSC 1
� instantaneous Task Specification and estimation using Constraints(iTaSC)
� constraint-based programming: robot nor task frame centric� auxiliary coordinates to
� express task constraints (feature coordinates), and� model geometric uncertainty (uncertainty coordinates)
� robot application = optimization problem consisting of a set ofconstraints, and one or multiple objective functions.
1J. De Schutter et al. Constraint-based task specification and estimation forsensor-based robot systems in the presence of geometric uncertainty, IJRR 2007.4 / 26
Introduction Modular Framework Experiments Conclusions
Concept
How to create a new iTaSCapplication
� empty Scene
� add Robots and Objects� define Object Frames� add SubTasks
� Virtual Kinematic Chains� Constraint/Controller
� add a Solver
5 / 26
Introduction Modular Framework Experiments Conclusions
Concept
How to create a new iTaSCapplication
� empty Scene� add Robots and Objects
� define Object Frames� add SubTasks
� Virtual Kinematic Chains� Constraint/Controller
� add a Solver
5 / 26
Introduction Modular Framework Experiments Conclusions
Concept
How to create a new iTaSCapplication
� empty Scene� add Robots and Objects� define Object Frames
� add SubTasks
� Virtual Kinematic Chains� Constraint/Controller
� add a Solver
o2b
o1a
o1b
5 / 26
Introduction Modular Framework Experiments Conclusions
Concept
How to create a new iTaSCapplication
� empty Scene� add Robots and Objects� define Object Frames� add SubTasks
� Virtual Kinematic Chains
� Constraint/Controller� add a Solver
i
h
f1bo2b
f2b
g
o1a
o1b
5 / 26
Introduction Modular Framework Experiments Conclusions
Concept
How to create a new iTaSCapplication
� empty Scene� add Robots and Objects� define Object Frames� add SubTasks
� Virtual Kinematic Chains� Constraint/Controller
� add a Solver
i
h
f1bo2b
f2b
g
o1a
o1b
5 / 26
Introduction Modular Framework Experiments Conclusions
Concept
How to create a new iTaSCapplication
� empty Scene� add Robots and Objects� define Object Frames� add SubTasks
� Virtual Kinematic Chains� Constraint/Controller
� add a Solver
i
h
f1bo2b
f2b
g
o1a
o1b
5 / 26
Introduction Modular Framework Experiments Conclusions
Concept
How to create a new iTaSCapplication
� empty Scene� add Robots and Objects� define Object Frames� add SubTasks
� Virtual Kinematic Chains� Constraint/Controller
� add a Solver
i
h
f1bo2b
f2b
gw
o1a
o1b
5 / 26
Introduction Modular Framework Experiments Conclusions
Modular software framework
� follow workflow of iTaSC methodology� division along two ’orthogonal directions’
1. separation of concerns2. hierarchy of three levels
6 / 26
Introduction Modular Framework Experiments Conclusions
Modular software framework
� follow workflow of iTaSC methodology� division along two ’orthogonal directions’
1. separation of concerns following the principle of the 5C’s 2
� Communication� Computation� Coordination� Configuration� Composition
2. hierarchy of three levels
2E. Prassler et al. The use of reuse for designing and manufacturing robots,Robot Standards project, white paper 20096 / 26
Introduction Modular Framework Experiments Conclusions
Modular software framework
� follow workflow of iTaSC methodology� division along two ’orthogonal directions’
1. separation of concerns following the principle of the 5C’s 2
2. hierarchy of three levels2.1 application level = iTaSC level and parts outside the iTaSC formalism2.2 iTaSC level = task level and iTaSC specification elements2.3 task level = functionalities that together form a single task
2E. Prassler et al. The use of reuse for designing and manufacturing robots,Robot Standards project, white paper 20096 / 26
Introduction Modular Framework Experiments Conclusions
Computation
application level components:� Drivers: hardware interfaces� Setpoint generators: desired values for the controllers of iTaSC� iTaSC: iTaSC level
iTaSC level components:� Robots and objects: kinematic and dynamic models of the robots
and objects ⇒ robot coordinates q� Tasks: task level� Solver: optimization problem solver resulting in the desired values
for the low-level robot controllers� Scene: core of the iTaSC composite component
7 / 26
Introduction Modular Framework Experiments Conclusions
Computation
task level components:� Virtual Kinematic Chain (VKC): model of the space between pairs
of object frames, defined on robots and objects ⇒ the state of theVKC, the auxiliary (feature) coordinates χf
� Constraint-Controller (CC): output equation y = f (χf , q) and thecontrol law
8 / 26
Introduction Modular Framework Experiments Conclusions
Computation
Scene Description
Constraint - Controller (CC)
Virtual Kinematic Chain (VKC)
Task
Driver
Hardware
iTaSC
Application
SceneRobot
Object
Solver
Setpoint Generator
reference implementation: Orocos components
9 / 26
Introduction Modular Framework Experiments Conclusions
Coordination
� fixed structure, hierarchical finite-state machine (FSM) for eachlevel element
� FSM’s of all levels form hierarchical, distributed statechart� pure event processors, independent of software component
framework� modelled with rFSM: a light-weight Lua-based statechart model 3
� reference implementation: rFSM models loaded in a SupervisorOrocos component
3M. Klotzbuecher and H. BruyninckxCoordinating robotic tasks and systems with rFSM Statecharts, JSER 201210 / 26
Introduction Modular Framework Experiments Conclusions
Coordination
Configure
Stop Start
Run
Application
iTaSC
Task 1 Task 2
11 / 26
Introduction Modular Framework Experiments Conclusions
Configuration, communication, composition
Configuration� configuration of each of the other concerns� loaded at configuration time� reference implementation: XML format filesCommunication� data and event communication� port structure� reference implementation: Orocos-ROS functionallity (port-topic)Composition� scripts details composition per level� reference implementation: Lua scripting language
12 / 26
Introduction Modular Framework Experiments Conclusions
Laser tracing example
How much work to switch between the following examples?
13 / 26
Introduction Modular Framework Experiments Conclusions
Laser tracing with a KUKA LWR
14 / 26
Introduction Modular Framework Experiments Conclusions
Laser tracing with a PR2
15 / 26
Introduction Modular Framework Experiments Conclusions
Laser tracing with a KUKA youBot
16 / 26
Introduction Modular Framework Experiments Conclusions
Laser tracing with a KUKA LWR: computation
Scene Description
CC_sixDof_pff
VKC_sixDof
laser tracing
lwr_driver
hardware
iTaSC
Application
scene
LWR
table
WDLS_prior_vel_solver
lissajous_generator
CC_cartesian
VKC_cartesian
CC_joint_limit_avoidance
move to reference frame
joint limit avoidance
transX, transY, rotX, rotY, rotZ, transZ
reference_frame
17 / 26
Introduction Modular Framework Experiments Conclusions
Laser tracing with a PR2: computation
Scene Description
CC_sixDof_pff
VKC_sixDof
laser tracing
pr2_driver
hardware
iTaSC
Application
scene
pr2
wall
WDLS_prior_vel_solver
lissajous_generator
CC_cartesian
VKC_cartesian
CC_joint_limit_avoidance
move to reference frame
joint limit avoidance
transX, transY, rotX, rotY, rotZ, transZ
reference_frame
18 / 26
Introduction Modular Framework Experiments Conclusions
Laser tracing with a KUKA youBot: computation
Scene Description
CC_sixDof_pff
VKC_sixDof
laser tracing
youbot_driver
hardware
iTaSC
Application
scene
youbot
ceiling
WDLS_prior_vel_solver
lissajous_generator
CC_cartesian
VKC_cartesian
CC_joint_limit_avoidance
move to reference frame
joint limit avoidance
transX, transY, rotX, rotY, rotZ, transZ
reference_frame
19 / 26
Introduction Modular Framework Experiments Conclusions
Laser tracing: coordinationcomposite_task_fsm.luaMoveToInitial MoveToHomeIdleTraceFigurez_off z_onz_constrainte_toggle_z_constrainte_toggle_robot_weightsrot_on rot_offrot_constraint circle/infinityjoint_weightspenalize_base penalize_arme_start_tracinge_stop_tracinge_cartesian_generator_moveFinished e_trace_circle / e_trace_infinitye_cartesian_generator_moveFinishede_move_to_homee_toggle_rot_constraint20 / 26
Introduction Modular Framework Experiments Conclusions
Laser tracing: coordinationReuse of the same coordination for all three cases
composite_task_fsm.lua
entry
raise_event("e_cartesian_tracing_PoseInputMode")
MoveToHome
raise_event("e_reference_frame_gen_start")
raise_event("e_reference_frame_gen_moveToHome")
exitraise_event("e_reference_frame_gen_stop")raise_event("e_cartesian_tracing_DoNothing")
entry
raise_event("e_cartesian_tracing_PoseInputMode")
MoveToInitial
raise_event("e_reference_frame_gen_start")
raise_event("e_reference_frame_gen_moveToInit")
exitraise_event("e_reference_frame_gen_stop")raise_event("e_cartesian_tracing_DoNothing")
entry
raise_event("e_robot_base_avoidance_Avoid")
raise_event("e_joint_limit_avoidance_Avoid")
entry
Idle
exit
TraceFigure
entry
raise_event("e_laser_tracing_Activate")
raise_event("e_lissajous_generator_start")
exitraise_event("e_lissajous_generator_stop")raise_event("e_laser_tracing_DoNothing")
entry
penalize_arm
raise_event("e_pen_arm")
entry
penalize_base
raise_event("e_pen_base")
joint_weights
entry
z_on
raise_event("e_laser_tracing_zOn")
entry
z_off
raise_event("e_laser_tracing_zOff")
z_constraint
entry
rot_on
raise_event("e_laser_tracing_rotOn")
entry
rot_off
raise_event("e_laser_tracing_rotOff")
rot_constraintcircle/infinity
21 / 26
Introduction Modular Framework Experiments Conclusions
Configuration
reconfiguration needed between the three cases only for� tasks
� possibly retuning of control gains� adapt joint limits for each robot
� robots and objects� adapt configuration to used robot
� setpoint generators� make sure reference frame is within robot workspace
22 / 26
Introduction Modular Framework Experiments Conclusions
Composition: Lua scriptsFor example: the iTaSC level compositionyoubotLoc:fromtab( {M={X_x=1,Y_x=0,Z_x=0,X_y=0,Y_y=1,Z_y=0,X_z=0,Y_z=0,Z_z=1},p={X=0.0,Y=0.0,Z=0.0}} )FixedObjectBaseLoc:fromtab( {M={X_x=1,Y_x=0,Z_x=0,X_y=0,Y_y=1,Z_y=0,X_z=0,Y_z=0,Z_z=1},p={X=0.0,Y=0.0,Z=2.0}} )
addRobot("youbot", youbotLoc)addRobot("FixedObject", FixedObjectBaseLoc)
addObjectFrame("youbot_gripper","gripper_palm_link", "youbot")addObjectFrame("youbot_base","plate_link", "youbot")addObjectFrame("ee_FixedObject","ee", "FixedObject")
addVirtualKinematicChain("VKC_cartesian_motion", "youbot_base", "youbot_gripper")addVirtualKinematicChain("VKC_laser_tracing", "ee_FixedObject", "youbot_gripper")
addConstraintController("CC_cartesian_motion", "", "VKC_cartesian_motion", 1)addConstraintController("CC_laser_tracing", "", "VKC_laser_tracing", 1)addConstraintController("CC_joint_limit_avoidance", "youbot", "", 1)
addSolver("Solver")23 / 26
Introduction Modular Framework Experiments Conclusions
Communication
� non-buffered vs. buffered connections� non-event-driven vs. event-driven ports� Orocos ports and ROS topics
24 / 26
Introduction Modular Framework Experiments Conclusions
ConclusionsiTaSC software framework� modular framework for constraint-based programming� modularity and reusability due to
1. separation of concerns2. division of each of the concerns into small entities within a hierarchy
� reference implementation using Orocos components, rFSMstatecharts, Lua/Orocos scripts, and XML configuration
Future work� formal modeling: Domain Specific Language (DSL) for
constraint-based programming� iTaSC DSL = application, soft-and hardware independent
programming language� further development of a software component framework
independent iTaSC functionality library25 / 26
Introduction Modular Framework Experiments Conclusions
Strategic research directions in robotic softwaredevelopmentMany problems are theoretically solved but almost none on thesoftware level, especially those working at the different levels ofabstraction that are needed in typical advanced robotics tasks.Why?� lack of common agreement on concepts and the resulting lack of
standards� poor separation of concerns: most software is
robot/platform/OS... specificSolutions?� The modelling approach as advocated by BRICS, Proteus,
Smartsoft, ROCK...: agree on models and standardize them, thenmake software implementations
26 / 26