Artificial Intelligence & Human-Robot Interaction Luca Iocchi Dept. of Computer Control and Management Eng. Sapienza University of Rome, Italy Robotic Applications • Industrial/logistics/medical robots • Known environment • Minimal interaction with expert users • Rescue robots • Unknown/partially known environment • Minimal interaction with expert users • Home service robots • Known environment • Long-term interaction with a few (trained) users • Service robots in public environments • Known and dynamic environment • Short-term interaction with many naïve users AIRO 2017 - AI & HRI 2
18
Embed
Artificial Intelligence & Human-Robot Interaction · • Multi-robot support PNPGen generates PNP from several planners (MDP solver, ROSPlan, HATP, …) PNP-ROS uses ROS actionlib
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Artificial Intelligence & Human-Robot Interaction
Luca IocchiDept. of Computer Control and Management Eng.
• Known environment• Minimal interaction with expert users
• Rescue robots• Unknown/partially known environment• Minimal interaction with expert users
• Home service robots• Known environment• Long-term interaction with a few (trained) users
• Service robots in public environments• Known and dynamic environment• Short-term interaction with many naïve users
AIRO 2017 - AI & HRI 2
AI&HRI Motivations• Interacting with people requires more "intelligence" than
interacting with the environment• Difficulty in modeling and perception (Unpredictability)
• Difficulty in decision making (Social norms)
• Some examplesIn HRI
• proper decisions about when to start an interaction are required
• failures have more severe consequences
• wrong assumptions or guesses may be socially unacceptable
• …
AIRO 2017 - AI & HRI 3
ICAPS 2017 Tutorial: AI Planning for Robotics and Human-Robot InteractionAAAI 2017 Fall Symposium: AI for Human-Robot InteractionRSS 2017 Workshops
AI & HRI
Robotic basic skills commonly used in HRI
Computer Vision fairly used for simple recognition tasks
Machine Learning used in subtasks (perception, speech recognition) and in Learning by Demonstrations
Natural Language Processing less used in robotics
KR & Decision making (autonomous behaviors) are less investigated
AIRO 2017 - AI & HRI 4
HRIapplication domain
(i.e., problem generator)
AI&RO methodologies and techniques
to solve complex problems(i.e., solution generator)
HRI Experimental Methodology
Wizard-of-Oz: an operator (hidden from the user evaluatingthe HRI system) replaces perception and decision making of the robot
Issues• Over-estimation of actual abilities of the robot
• Partial evaluation of the HRI system (does not measure"intelligence in interaction")
• Expectations not met by current technology, preventing or reducing possibilities of deployment of actual systems
AIRO 2017 - AI & HRI 5
HRI in public environments
Main challenges• Interaction with naïve users• Follow complex social norms• Assumption of having
complete information is not realistic
• Perception involving humans is generally more difficult
• Guessing missing information may bring to socially unacceptable behaviors
• Need of very robust perception and decision-making
AIRO 2017 - AI & HRI 6
Analysis of planning techniques RoboCup@Homelargest worldwide competition on service robots
Planning in the 24 teams in RoboCup@Home 2016
Complex robotic systems integrating:• Robotics: navigation, manipulation, …• Computer Vision: object/face/person detection, recognition and tracking• HRI: Speech recognition and natural language processing
Plan GenerationHigh-level Behavior
N/A Total
2 (ASP, MDP) 9 13 24
AIRO 2017 - AI & HRI 7
RoboCup@Home Teams
WrightEagle: ASP Markvito: MDP/SPUDD
Golem: cognitive architecture. Dialogue models specified in SitLogHomer: state machinesLeon: state machine (BICA)Pumas: HTN / High-level Petri NetsSepanta: SMACHSocRob: POMDP + Discrete Event System / SMACHTech United: SMACHToBI : BonSAI + State Chart XML = SMACHWalking Machine: SMACH
AIRO 2017 - AI & HRI 8
Decision making in HRIDecision making in noisy and partially known real environments is still a challenging open issue in AI
HRI researchers not expert in AI & decision making do not find a practical and easy way to apply these techniques
Manual behavior programmingand Wizard-of-Oz experimentsare commonly used
AIRO 2017 - AI & HRI 9
TODO
• Extend applicability
• Improve usability
• Improve robustness
Automatic planning in HRI
AIRO 2017 - AI & HRI 10
Planning improves scalability wrt complexity and flexibility• Manual plan writing is error prone and not scalable
• Each task must be defined explicitly/no reuse of components
• Verification/validation is not possible
• Explanations are very difficult
Advantages• Compact representations
• Less effort in generating many plans
• High-level domain specification language for non-expert designers
Service robot has to make sure user needs are satisfied. User needs are not known in advance.
AIRO 2017 - AI & HRI 11
Example
Classical planning(complete knowledge about initial state)
• Guess a user need
• Plan with this guess
• Execute the plan
• If guess is wrong, adjust conditions and replan
When guess is wrong, behavior is not socially acceptable.
• The robot does not move, but the user needs something.
• The robot prepares food or drink that is not requested by the user.
AIRO 2017 - AI & HRI 12
Example
Plan with explicit sensing action• Go to person
• Ask if s/he needs something // Sensing action
• if (need_food AND need_drink)• Go to the kitchen
• Prepare food and drink
• Serve food and drink to person
• if (need_food)• …
• else• Do nothing
AIRO 2017 - AI & HRI 13
Example
Pro-active behavior in which the robot does not
just wait for orders.
Conditional Planning for service robots
Advantages• Does not require complete knowledge about the initial state
at planning time (can model situations where some user needs are not known)
• Allows for minimal execution of sensing (reduces wrong behaviors due to incorrect perception)
Issues• Plan generation more complex • Conditional Planners less developed• Writing domains is still difficult• Loop generation (e.g., while conditions) not available
AIRO 2017 - AI & HRI 14
Not-only planning
AIRO 2017 - AI & HRI 15
Robot planning
Planning expert
Robot/plan expert user
Robot planning
Domain expert
Naïve user
Robust plans
• Plans generated by planners are usually no robust to unmodelledevents
• Interaction with naïve users requires increased robustness of plans
AIRO 2017 - AI & HRI 16
Robot planning
Domain expert
Naïve user
Generation of robust plans
DomainPlanner
???
Execution
π
Goal
Planning and Execution Component
π*
UILearning
AIRO 2017 - AI & HRI 17
???
Proposed solution
ROSPlanContingent-FF
Robustification
PN
P-R
OS
π
Planning and Execution Component
Execution
Rules
PNP
AIRO 2017 - AI & HRI 18
Domain
Goal
[Sanelli et al., ICAPS 2017]
ROSPlan and Contingent-FF
ROSPlan,
Contingent-FF
http://kcl-planning.github.io/ROSPlan/
AIRO 2017 - AI & HRI 19
[Cashmore et al., ICAPS 2015]
[Hoffmann and Brafman, ICAPS 2005]
Petri Net PlansFormalism for high-level plan representationbased on Petri Nets
• Ordinary and sensing actions• Conditions and loops• Interrupts• Parallel execution (fork and join operators)• Multi-robot support
PNPGen generates PNP from several planners (MDP solver, ROSPlan, HATP, …)
PNP-ROS uses ROS actionlib to run plans including ROS actions
pnp.dis.uniroma1.it
AIRO 2017 - AI & HRI 20
[Ziparo et al., JAAMAS 2011]
Execution rules
Adding to the conditional plan• interrupt (special conditions that activate recovery paths)• recovery paths (how to recovery from unexpected events)• social norms• parallel execution (multi-modalities)
Main feature• Easy definition• Execution variables are generally different from the ones
in the planning domain (thus not affecting complexity of planning)
AIRO 2017 - AI & HRI 21
Examples
if personhere and closetotarget during goto do skip_actionif personhere and not closetotarget during goto do
say_hello; waitfor_not_personhere; restart_actionif lowbattery during * do recharge; fail_planafter receivedhelp do say_thanksafter endinteraction do say_goodbyewhen say do display
AIRO 2017 - AI & HRI 22
Execution rules
Petri Net Plans generation
PNP Generation 1. Translation of conditional plan to PNP
2. Introduction of execution rules
Robust plan with sensing and loops
Algorithm is linear in the size of the plan and of the execution rules (average case)
[Sebastiani et al., ICAPS 2017][Iocchi et al., ICAPS 2016]
Domain description
• In previous examples planning domains written by planner experts
• HRI domain experts (not expert in planning) need high-level framework for interaction design
AIRO 2017 - AI & HRI 27
Robot planning
Domain expert
Naïve user
MODIM
Multi-modal Interaction Manager• Formalism for high-level description of multiple modalities interactions• Based on interaction templates that can be instantiatedto generate many
actual interactions• Represent robotic and interaction actions in a unique framework• Not yet a framework to specify planning domains• Actions types:
MODIM was used by a school teacher to design and realize a physics lesson and to perform HRI studies
AIRO 2017 - AI & HRI 33
Conclusions• AI Decision making techniques useful in HRI tasks
• Conditional planning + robustification improve robustness of robot plans for HRI applications
• High-level frameworks for interaction design improve usability by non-expert designer and developer
• More work is needed to improve usability and applicability
• More experiments in real environments with real users and real limitations (no Wizard-of-Oz)
Thank you for your attention
AIRO 2017 - AI & HRI 34
[Hoffmann and Brafman 2005] Hoffmann, J., and Brafman, R. Contingent planning via heuristic forward search with implicit belief states. In Proc. of ICAPS, 2005.
[Cashmore et al., 2015] Cashmore, M.; Fox, M.; Long, D.; Magazzeni, D.; Ridder, B.; Carrera, A.; Palomeras, N.; Hurtos, N.; and Carreras, M. Rosplan: Planning in the robot operating system. In Proc. of ICAPS, 2015.
[Iocchi et al., 2016] Luca Iocchi, Laurent Jeanpierre, Maria Teresa Lazaro, Abdel-IllahMouaddib: A Practical Framework for Robust Decision-Theoretic Planning and Execution for Service Robots. In Proc. of ICAPS, 2016.
[Sanelli et al., 2017] V. Sanelli, M. Cashmore, D. Magazzeni, L. Iocchi. Short-Term Human Robot Interaction through Conditional Planning and Execution. In Proc. of ICAPS 2017
[Sebastiani et al. 2017] E. Sebastiani, R. Lallement, R. Alami, L. Iocchi. Dealing with On-line Human-Robot Negotiations in Hierarchical Agent-based Task Planner. In Proc. ICAPS 2017.
[Ziparo et al., 2011] V. A. Ziparo, L. Iocchi, P. U. Lima, D. Nardi, P. F. Palamara. Petri Net Plans - A framework for collaboration and coordination in multi-robot systems. In Autonomous Agents and Multi-Agent Systems, 23 (3), 2011.