Interaction Challenges for Intelligent Assistants Jim Blythe USC Information Sciences Institute.

Interaction Challenges for Intelligent Assistants

Jim Blythe

USC Information Sciences Institute

How to build “truly useful assistants”? Personalized, Learn, Engender trust, Become partners

Organizer: Neil Yorke-Smith

Committee: Pauline Berry, Timothy Bickmore, Mihai Boicu, Justine Cassell, Ed Chi, Mike Cox, John Gersh, Jihie Kim, Jay Modi, Donald Patterson, Debra Schreckenghost, Richard Simpson, Stephen Smith, Sashank Varma

28 accepted papers

Topics

• Trust

• When to assist?

• Learning

• Modeling

• Desktop assistants

• Panel with symp. on multidisciplinary collaboration for socially assistive robots

• Panel with intentions in intelligent systems

How To Make Users Happy

• And avoid annoying users

- Brad Myers’ invited talk

User Happiness?

Hu = f (Performance)

User Happiness?

Hu = f (Performance, Trust)Hu = f (Performance, Trust)

User Happiness!Hu = f (EAssistant ENegative EPositive EValue EUser ECorrected

EBy-hand ECost EAvoided EApparentness ECorrect-difficulty

ESensible WQuality WCommitment TBy-hand TBy-Hand-start-

up TBy-Hand-per-unit TAssistant TTraining-start-up TAssistant-

per-unit TInteraction-per-unit TMonitoring TCorrecting

TResponsiveness TSystem-Training TUser-training

TAverage-for-each-correction AError-rate Nunits PPleasantness

UPerceive UWhy UProvenance UPredictability IAssistant-

interfere IScreen-space ICognitive IAppropriate-Time CAutonomy

CCorrecting SSensible-Actions SUser-models SLearning

RSocial-Presence DHand VImportance)

Hu = f (EAssistant ENegative EPositive EValue EUser ECorrected

EBy-hand ECost EAvoided EApparentness ECorrect-difficulty

ESensible WQuality WCommitment TBy-hand TBy-Hand-start-

up TBy-Hand-per-unit TAssistant TTraining-start-up TAssistant-

per-unit TInteraction-per-unit TMonitoring TCorrecting

TResponsiveness TSystem-Training TUser-training

TAverage-for-each-correction AError-rate Nunits PPleasantness

UPerceive UWhy UProvenance UPredictability IAssistant-

interfere IScreen-space ICognitive IAppropriate-Time CAutonomy

CCorrecting SSensible-Actions SUser-models SLearning

RSocial-Presence DHand VImportance)

A Tale of Two Associates• Pilot’s Associate (1985-

1991)– Single Pilot– Direct pilot interaction with

associate meant added workload

– Design philosophy minimized direct pilot interaction with associate

– Moderate user acceptance

The Pilot is The Pilot is ALWAYS in ALWAYS in

charge.charge.

The Effort The Effort required of the required of the pilot to control pilot to control the associate the associate

must be less than must be less than the effort saved the effort saved by the associateby the associate

• Rotorcraft Pilot’s Associate (1994-1999)– Two Pilots

– 1/3 of human activity is crew coordination

– Design philosophy included some direct pilot interaction with associate

– Improved User Acceptance

Why and how to model multi-modal interaction for a mobile robot companion

Shuyin Li & Britta Wrede Best paper

• Tested policies with users interacting with a robot

• Communicate pre-interaction attention

• Need to make social remarks with non-verbal methods (because people tend to reply in kind)

Biron and Barthoc

Interaction Challenges for Agents with Common Sense

Invited talk from Henry Lieberman

• We now have several sources of common sense knowledge, e.g. Cyc, Open Mind, ThoughtTreasure

• Some strategies and examples of exploiting common sense to build better interfaces

Strategies for using common sense in interfaces

• Find underconstrained situations

• Find situations where every little helps

• Know a little about everything, but not too much about anything

• Make better mistakes! Not just ‘right’ and ‘wrong’, being reasonable is better– Plausible mistakes can increase trust

• Set user expectations

Examples of interfaces using common sense

• ARIA photo agent: more powerful matching of tags using common sense

• Predictive typing:

“I’m having landlord problems because my roommate was late with my r..”

• BEAM

(Gil & Chklovski)

Trust

• Openness and understanding more important as systems become more complex.

• Methods to improve understanding: explanations [McGuinness et al.]

• HTN metamodel [Wallace]• Patterson: would I trust a fork? a bridge? a

space shuttle?– predictability, understandability, similarity, liability,

social/emotional

Learning (and Trust)

• Adaptive (Learning) vs Adaptable (Instructed by user) – important for believability and trust

Supporting interaction in Robocare intelligent assistant agent

Endowed with human like I/O channels by engineering state of the art components

•Face: Lucia (Piero Cosi, ISTC, Pd)

•Voice: Sonic (Univ.Colorado)

•Simple Interaction Manager

The Motion Skills

The Interaction Skills

Robust continuous behavior at home with person

Use of multiagent technologyCesta et al. Best application paper

Multiple Intelligent Systems

Supporting interaction in Robocare intelligent assistant agent

Integrates multiple systems to produce a socially acceptable robotic care assistant

• Interesting DCOP solution to allow multiple systems and guarantee coherent behaviour

• System follows a STN to notice deviations from expected behaviour

• Experiments in face/no-face in RoboCare

• People prefer no-face – “less artificial”, “more integrated in the

domestic environment”

Desktop assistants

• Many papers on desktop assistants– 6 from the Calo project

PeXA architecture

Towel todo manager

• Towel [Conley et al]: taking an IM approach to give access to tasks

Inspired by Diamond Help [Rich et al. 06]

Did Ken sacrifice himself to User Testing?

• Registered to give talk at AAAI Spring symposium

Should Ken have worked on meeting scheduling?

• Registered to give talk at AAAI Spring symposium

• Booked another trip in same week

Should Ken have worked on meeting scheduling?

• Registered to give talk at AAAI Spring symposium

• Booked trip to Hawaii in same week

• Ultimate in user testing? You decide..