Tasks, Abilities, and Term Projects

COMPUTER SCIENCEUASCIENCE

Paul Cohen

Tasks, Abilities, and Term Projects

Paul Cohen

COMPUTER SCIENCEUASCIENCE

Paul Cohen

But first...Names!

• Emily• Oliver• Curious George• ...• Come on, people!

COMPUTER SCIENCEUASCIENCE

Paul Cohen

What we want

Task 1 Task 2 Task 3 Task N

Ability 1 X X X

Ability 2 X X

Ability 3 X X X

...

Ability N X X X

Define a handful of tasks, each of which requires multiple abilities. Each ability is used for more than one taskEach task requires more that one abilityThe outputs of some abilities are the inputs of othersEach team works on one ability (or a small number) but exercises it in all tasks

COMPUTER SCIENCEUASCIENCE

Paul Cohen

Tasks

• Meet and Greet• Turntaking games• Tidying up• Whazzat (Wubble World)

COMPUTER SCIENCEUASCIENCE

Paul Cohen

Meet and Greet

• Amr Gaber’s idea:– Robot sits somewhere public, greets people, learns to

recognize them and their names, learns to recognize moods and attitudes, learns social conventions re chatting...

• Examples– What to say to someone you do/don’t recognize– What to say when two people are talking to each other– What to say when people are hanging around/walking fast

“Hi Paul, nice to see you again. How’s Carole? I remember you were in the lab last thursday with Cody. He is here a lot. Sometimes I think he sleeps here. ...”

“Hi Ian, I see you are talking with someone and don’t want to interrupt, but could you introduce me to your friend?”

COMPUTER SCIENCEUASCIENCE

Paul Cohen

COMPUTER SCIENCEUASCIENCE

Paul Cohen

Meet and Greet Abilities

Meet&Greet Turntaking Games

Clean Up Wubble World

Face recognition x

See someone’s there x

Recall previous chats x

Generate smalltalk x

Detect affect x

Recognize activities x

Physical control eyes

Learn stuff about people

COMPUTER SCIENCEUASCIENCE

Paul Cohen

Turntaking Games

Peekaboo !

Pattycake !

COMPUTER SCIENCEUASCIENCE

Paul Cohen

Turntaking Games Abilities

Meet&Greet Turntaking Games

Clean Up Wubble World

Face recognition x

See someone’s there x x

Recall previous chatsgames

x x

Generate smalltalk game interaction

x x

Detect affect x x

Recognize activities x x

Physical control eyes arm

Learn stuff about people

repetitive activities

classes of things and

where they go

COMPUTER SCIENCEUASCIENCE

Paul Cohen

Clean Up

COMPUTER SCIENCEUASCIENCE

Paul Cohen

Clean Up Abilities

Meet&Greet Turntaking Games

Clean Up Wubble World

Face recognition x

See someone’s there x x

Recall previous chatsgames plans

x x x

Generate smalltalk game interaction actions

x x x

Detect affect x x

Recognize activities x x x

Physical control eyes arm arm, wheels

Learn stuff about people

repetitive activities classes of things and where they go

COMPUTER SCIENCEUASCIENCE

Paul Cohen

Go to the red cone

COMPUTER SCIENCEUASCIENCE

Paul Cohen

Work with Daniel Hewlett and Wesley KerrWubble and room design by Teresa Dey

Wubble World

COMPUTER SCIENCEUASCIENCE

Paul Cohen

COMPUTER SCIENCEUASCIENCE

Paul Cohen

How Wubbles Learn the Meanings of Words

"Go to the blue cylinder"

goto

the blue cylinder

1. parse

4. act or ask

3. score correspondences between retrieved representation and scene

2. retrieve a semantic representation of a word or phrase

5. given positive feedback, update semantic representation

COMPUTER SCIENCEUASCIENCE

Paul Cohen

Wubble Abilities

Meet&Greet Turntaking Games

Clean Up Wubble World

Face recognition x

See someone’s there x x

Recall previous chatsgames plans word meanings

x x x x

Generate smalltalk game interaction actions

x x x x

Detect affect x x

Recognize activities x x x

Physical control eyes arm arm, wheels path planning

Learn stuff about people

repetitive activities classes of things and where they go

some word meanings

COMPUTER SCIENCEUASCIENCE

Paul Cohen

Other Abilities

Meet&Greet Turntaking Games

Clean Up Wubble World

Face recognition x

See someone’s there x x

Recall previous chatsgames plans word meanings

x x x x

Generate smalltalk game interaction actions

x x x x

Detect affect x x

Recognize activities x x x

Physical control eyes arm arm, wheels path planning

Curiosity / motivation x x

Control x x x x

Conceptual Glue x x x x

Learn stuff about people

repetitive activities classes of things and where they go

some word meanings

COMPUTER SCIENCEUASCIENCE

Paul Cohen

Assignment for Next Week

• Break into groups for the four tasks (we’ll organize around abilities later)

• Storyboard a few minutes in the life of the robot doing the task• Make a refined list of abilities • Search for off-the-shelf software for these abilities in ROS and

online. Example 1, Example 2, Example 3, Example 4,• Pay attention to glue: concepts and control• Write and present a 15-minute dissertation on all of the above