SENSITIVITY ANALYSIS FOR BUILDING ADAPTIVE ROBOTIC SOFTWARE Pooyan Jamshidi, Miguel Velez and Christian Kästner INTENT DISCOVERY: SENSITIVITY ANALYSIS FOR CONFIGURATION OPTIMIZATION REDUCING COSTS WITH TRANSFER LEARNING USE CASES Systematic System Evolution To automate or guide intelligent design choices. Runtime Adaptation To enable runtime adaptation of software configurations to maintain quality of performance under dynamic conditions (changing environment, goals, and tasks). Performance Debugging To guide robot software developers to identify potential bugs causing low quality of performance. RESULTS Motivation: Robotic software expose configurable parameters. These tunable parameters affect performance of robots. This can be leveraged to optimize performance. Source Response Target Response Transfer learning combines: Lots of data gathered cheaply from the simulator With much less data gathered expensively from the target robot To make better predictions overall PUBLICATIONS P. Jamshidi, M. Velez, C. Kästner, N. Siegmund, and P. Kawthekar. Transfer learning for improving model predictions in highly configurable software. Int’l Symp. Software Engineering for Adaptive and Self-Managing Systems (SEAMS), 2017. P. Kawthekar and C. Kästner. Sensitivity analysis for building evolving & adaptive robotic software, Workshop on Autonomous Mobile Service Robots (WSF), 2016. Predictive Model Learn Model Measure Measure Data Source Target Simulator (Gazebo) Robot (TurtleBot) Predict Performance Predictions Adaptation Use for analysis 5 10 15 20 25 number of particles 5 10 15 20 25 number of refinements 0 5 10 15 20 25 5 10 15 20 25 number of particles 5 10 15 20 25 number of refinements 0 5 10 15 20 25 5 10 15 20 25 number of particles 5 10 15 20 25 number of refinements 0 5 10 15 20 25 CPU usage [%] CPU usage [%] (a) (b) (c) (d) Prediction without transfer learning 5 10 15 20 25 5 10 15 20 25 10 15 20 25 Prediction with transfer learning Using only a few real data points to predict yields poor results across configuration space Using transfer learning to combine the few real data points with lots of approximate data yields a good model Machine Learning Configuration Parameters Design of Experiment Configuration Space Predictive Model Sensitivity Analysis Data Measurem ents Configuration Space Data Accuracy Energy CPU 0 5 10 15 20 25 30 35 mean CPU utilization 0 500 1000 1500 2000 2500 3000 3500 number of configurations 5 10 15 20 25 number of particles 5 10 15 20 25 number of refinements 6 8 10 12 14 16 18 20 22 24 26 5 10 15 20 25 number of particles 5 10 15 20 25 number of refinements 5 10 15 20 25 30 35 40 45 CPU Localisation Error