Common Sense Computing • MIT Media Lab
Interaction Challenges for Agents with Common Sense
Henry Lieberman
MIT Media Lab
Cambridge, Mass. USA
Http://www.media.mit.edu/~lieber
Common Sense Computing • MIT Media Lab
Agents with Common Sense
Some AI interfaces are now beginning to make use of large knowledge bases of Common Sense
Explicitly collected
Cyc, Open Mind, ThoughtTreasure
Distilled from other sources
Semantic Web, Wikipedia, Web resources
Web mining, other resource mining
Common Sense Computing • MIT Media Lab
Interaction Challenges
Finding opportunities for applying Common Sense in interfaces
Setting users' expectations
Making interfaces fail-soft
Taking advantage of user interaction
Common Sense Computing • MIT Media Lab
Interaction Challenges
Making better mistakes
Get lots of knowledge, but not too much
Common sense inference vs. mathematical inference
Debugging
Evaluating Common Sense interfaces
Common Sense Computing • MIT Media Lab
Interaction challenges for AI/Common Sense
Many interaction challenges for Common Sense interfaces are the same as for AI in general
But some are unique or critical for Common Sense…
Can't be sure what will be known
Reasonable, rather than right
Know a little about everything, not much about anything
Don't miss the obvious
Try not to make stupid mistakes
Common Sense Computing • MIT Media Lab
Open Mind Common Sense
Asks the Web community to contribute English sentences expressing Common Sense knowledge
"The Wikipedia version of Cyc" (#2 after Cyc)
Launched in 2001 by Push Singh
Now contains ~800 Kilofacts
Freely available / Open Source
Some versions in other languages/cultures
Brazilian Portuguese, Korean, Japanese, Chinese
Common Sense Computing • MIT Media Lab
Open Mind Common Sense
English sentences parsed by POS Tagger
Pattern-directed mining of 22 relations
(isA, PartOf, UsedFor…)
Strong focus on easy integration with applications
Semantic Net (ConceptNet: Liu, Eslick)
Natural Language toolkit (MontyLingua: Liu)
Tools for: Context, Analogy, Affect and more
Common Sense Computing • MIT Media Lab
What do we mean by “Common Sense”?
Simple statements about everyday lifeThings fall down, not upA wedding has a bride and a groomYou go to a restaurant to eat
And…the ability to use that knowledge when appropriate
Common Sense Computing • MIT Media Lab
Open Mind Common Sense- Push Singh & colleagues
Common Sense Computing • MIT Media Lab
ConceptNet - Liu, Singh, Eslick
Common Sense Computing • MIT Media Lab
Common Sense projects
Predictive typing
Speech recognition -- disambiguation & error correction
Storytelling with photo libraries
Searching social networks
Macro recording using Common Sense generalization
World construction for video games
Phrasebook for tourist information
Debugging problems in e-commerce interactions
Common Sense Computing • MIT Media Lab
Common Sense projects
Video editing based on story structures
Goal-oriented interfaces for consumer electronics
Mining Common Sense from the Web
Multi-lingual and multi-cultural Common Sense; translation
Games for acquiring, verifying and using Commonsense knowledge
Commonsense "Captchas"
Understanding imprecise qualities such as affect
ShapeNet and Expectation-driven Image Recognition
Understanding sensor data using Commonsense
Common Sense Computing • MIT Media Lab
Opportunities for using Common Sense
Find UI situations that are underconstrained
Ordinary system would either take no action or do something arbitrary
Then, give user some reasonable choices
Provide intelligent defaults
Make the most likely thing easiest to do
Common Sense Computing • MIT Media Lab
Opportunities for using Common Sense
Recognize users' likely goals
Help users map from goals to actions
Sanity checking
In the case of trouble, help users debug
Common Sense Computing • MIT Media Lab
Opportunities for using Common Sense
Find situations where every little bit helps
A little bit of knowledge is better than none
A little bit of knowledge about a lot of things can be more useful than a lot of knowledge about a few things
Common Sense Computing • MIT Media Lab
Setting users' expectations
Avoid direct question-answer interfaces
Right or wrong. Only one shot.
Better to cast system in role of advisor
Making suggestions, help
Adapting interface to most likely uses
Remove unnecessary steps in the interface
The user only expects intelligent behavior only once in a while
Common Sense Computing • MIT Media Lab
Take advantage of user interaction
Repurpose input that the user gives you for other reasons
Every time the user tells the interface something, they're telling you what their interests are -- learn from it
Common Sense Computing • MIT Media Lab
Make Common Sense interfaces "fail-soft"
There should no dire consequences of being wrong or not knowing what to do
Don't interfere if the user wants to use the application without interaction with the agent
If the relevant knowledge is missing, incomplete or wrong, the user is no worse off than without the agent
Common Sense Computing • MIT Media Lab
Make better mistakes
Common Sense approaches have the advantage that when they make mistakes, they tend to make plausible mistakes
Statistical approaches can make arbitrary mistakes
Better mistakes improve user trust in interfaces
Common Sense Computing • MIT Media Lab
Evaluation of Common Sense interfaces
Evaluation is tough because
Depends on what's in the knowledge base
Depends on limited-depth and other kinds of approximate inference
Standardized tasks don't test breadth of coverage
Try to relativize testing to coverage
Start with easier cases, then move to "typical" or "hard" cases
Intelligent User Interfaces 2008www.iuiconf.org
• Location: Canary Islands, Spain
• Dates: 13-16 January 2008
• Deadline: late September 2007