Empathic Computing

EMPATHIC COMPUTING Mark Billinghurst [email protected]

June 10th 2015

Hiroshi Ishii – AWE 2014

•  daf

ARToolKit (1999)

First open source AR SDK http://artoolkit.sourceforge.net/

MagicBook (2001)

• First AR story book • Transitional AR to VR experience

AR Tennis (2005)

• First collaborative AR game on a mobile phone

Mobile AR Advertising (2007)

• First mobile AR ad campaign (Saatchi & Saatchi)

AR Business Today

• Around $600 Million USD in 2014 (>$2B 2015) • > 80% Games and Marketing applications

“Do you want to sell sugar water for the rest of your life or do you want to come with me and change the world?"

Steve Jobs to John Sculley 1983

Mark’s Midlife Crisis • Then

• Recently Married • Finished sabbatical at Google • Looking for new opportunities

• Now • Resigned my job • Moved to a new country • Creating a new research group

Interaction Technology Natural

Time

Punch Card

Keyboard

Mouse

Speech

Gesture

Emotion

1950 1960 1980 1990 2000 2010

Thought

Physiological Sensing

Emotiv Empatica

Interaction Technology Natural

Time

Punch Card

Keyboard

Mouse

Speech

Gesture

Emotion

1950 1960 1980 1990 2000 2010

Thought

Implicit

Explicit

Content Capture Realism

Time

Photo

Film

Live Video

Panorama

360 Video

3D Space

1850 1900 1940 1990 2000 2010

3D Image/Space Capture

Google Project Tango Samsung Project Beyond

Content Capture Realism

Time

Photo

Film

Live Video

Panorama

360 Video

3D Space

1850 1900 1940 1990 2000 2010

2D Static

Immersive

Live

Experience

Networking Speeds Log (b/s)

Time

100 b/s

10 Kb/s

1 Mb/s

1980 1985 1990 1995 2000 2010

100 Mb/s

2005

Network Innovation

Universal Connectivity

Networking Speeds Log (b/s)

Time

100 b/s

10 Kb/s

1 Mb/s

1980 1985 1990 1995 2000 2010

100 Mb/s

2005

Text

Audio

Natural

Video

Natural Collaboration

Implicit Understanding

Experience Capture

Natural Collaboration

Implicit Understanding

Experience Capture

Empathic Computing

EMPATHIC COMPUTING Systems that Create and Share Understanding

Empathy

“Seeing with the Eyes of another,

Listening with the Ears of another,

and Feeling with the Heart of another..”

Alfred Adler

Empathic Computing

1. Understanding: Systems that can understand your feelings and emotions

2. Sharing: Systems that help you better understand the feelings of others

Understanding: Affective Computing

• Ros Picard – MIT Media Lab • Systems that recognize emotion

Appliances That Make You Happy

• Jun Rekimoto – University of Tokyo/Sony CSL • Smile detection + smart appliances

Sharing

Can we develop systems that allow us to share what we are seeing, hearing and feeling with others?

•  sdfs

•  sdfgs

•  axcvxca

Movies are like a machine that generates Empathy

Roger Ebert

Wearable AR for Empathic Interfaces

• Wearable AR can: • Be unobtrusive • Capture emotion • Share sights, and sounds • Support remote collaboration • Enhance interaction in the real world

Example: Google Glass

• Camera + Processing + Display + Connectivity

• Ego-Vision Collaboration (But with Fixed View)

Social Panoramas (ISMAR 2014)

• Capture and share social spaces in real time

• Supports independent views into Panorama

Implementation

• Google Glass • Capture live image panorama (compass + camera)

• Remote device (tablet) •  Immersive viewing, live annotation

User Interfaces

Glass View

Tablet View

Lessons Learned

• Good • Communication easy and natural • Users enjoy have view independence • Very natural capturing panorama on Glass • Sharing panorama enhances the shared experience

• Bad • Difficult to support equal input • Need to provide awareness cues

Example: CoSense (CHI 2015)

• Real time sharing - Emotion, video, and audio • Wearable (sender) – Send emotion and view • Desktop (receiver) - See remote view and emotion

Google Glass e-Health 2.0 board

+

Implementation

Data Capture

Feature Detection

Emotion Recognition

Emotion Representation

Empathic User Interface

Hardware

User Interface

Wearable Interface

• Google Glass + e-Health + Spydroid + SSI • Measure GSR, pulse oxygen, ECG, voice pitch • Share video and audio remotely • Representative emotions sent back to Glass user

!

!

Desktop Interface

Lessons Learned

• Good • System was wearable • Sender and receiver mirrored emotion • Minimal cues provided best experience

• Bad • System delays • Need for good stimulus • Difficult to represent emotion

Scaling Up

• Seeing actions of millions of users in the world • Augmentation on city/country level

AR + Smart Sensors + Social Networks

• Track population at city scale (mobile networks) • Match population data to external sensor data • Mine data for applications

Example: MIT SENSEable City Lab

http://senseable.mit.edu/wikicity/rome/

Example: CSIRO WeFeel Tool

• Emotionally mining global Twitter feeds

• http://wefeel.csiro.au

Research Challenges

• How to capture emotion?

• How to measure empathy?

• Interface/interaction models?

• How to communicate emotion?

• How to create strong empathic bonds?

• How to scaling up to city/country scale?

Potential Applications • Education • Sports training • Rich life logging • Remote meeting support • Psychological treatments • Virtual Travel/Entertainment • Surrogate Adventure Tourism • First responders (stress, team cohesion)

Questions to Think About..

What’s your Vision?

How do you want to be remembered?

“Do you want to sell sugar water for the rest of your life or do you want to come with me and change the world?"

www.empathiccomputing.org

@marknb00

[email protected]