Top Banner
Duet Making Localization Work for Smart Homes Shichao Yue Presenting on behalf of Deepak Vasisht, Anubhav Jain, Chen-Yu Hsu, Zachary Kabelac, Dina Katabi
36

Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

May 26, 2020

Download

Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

DuetMaking Localization Work for Smart Homes

Shichao YuePresenting on behalf of

Deepak Vasisht, Anubhav Jain, Chen-Yu Hsu, Zachary Kabelac, Dina Katabi

Page 2: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

The Smart Home Dream

Page 3: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Problem Statement

Smart homes need continuous tracking of location and identity of occupants

Cannot use camera, privacy-invasive

How about RF?

Page 4: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

RF-Based Localization

Page 5: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Problem 1: People Do Not Always Carry Phones

Page 6: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Problem 1: People Do Not Always Carry Phones

People don’t carry their phone over 50% of the time

Page 7: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Problem 2: Wireless Signals get Blocked

Page 8: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Problem 2: Wireless Signals get Blocked

Bathroom tiles block wireless signals

Page 9: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

RF based location data is:

• Error-prone: Users don’t always have their phone• Intermittent: Homes have several blockages for

RF signals (TV, bathroom tiles, etc)

Page 10: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Problem Statement

Smart homes need continuous tracking of location and identity of occupants

in spite of error-prone and intermittent RF data

Page 11: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Duet•Delivers continuous tracking of occupant location and

identity with error-prone, intermittent RF data• Error-prone data: Combine information from device-free

and device-based systems• Intermittent data: Use probabilistic logic to encode spatio-

temporal constraints

• Evaluated over two weeks in two environments with user devices

Page 12: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Problem 1: People Do Not Always Carry Phones

Idea: Use device-free localization

Page 13: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Device-free Localization

Uses reflections to track peopleDoesn’t need a device

But… No Identity

Page 14: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Device-based Localization

Device-free Localization

Needs people to carry cellphones

Can identify people

Doesn’t need cellphones

Cannot identify people✓⨯

Idea: Track both people and devicesUse interactions to match

Page 15: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Idea: Capture interaction between people & devices

Page 16: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Idea: Capture interaction between people & devices

Page 17: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Idea: Capture interaction between people & devices

Page 18: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Idea: Capture interaction between people & devices

Page 19: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Idea: Capture interaction between people & devices

Page 20: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Idea: Capture interaction between people & devices

Page 21: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Problem 2: Wireless Signals get Blocked

Page 22: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Observation 1: Logical Spaces have Transition Points

Page 23: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Observation 2: Logical Dependencies in Space-Time

Page 24: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Observation 2: Logical Dependencies in Space-Time

Page 25: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Logical Dependencies in Space-Time

•Cannot be present in two places at the same time

•Cannot enter places that they already occupy

•Cannot exit from places that they don’t occupy

Page 26: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Step 1: Track Entries and Exits to Spaces

•Duet uses a Hidden Markov Models to identify entry and exits trajectories

•Does not need training per region

HMMNoisy RF-data Entry/Exit Trajectories

Page 27: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Step 2: First Order Logic Formulation

!" = $% & = 1,2, …+$% = (-, ., /)

P: Possible identities for the individual

I: Impossible identities for the individual

R: The location of the individual

State

Page 28: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Step 2: First Order Logic Formulation

!" = $% & = 1,2, …+$% = (-, ., /)

• Can reason about a rich set of constraints

• Provable satisfiability algorithm to prune out invalid states

Page 29: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Experimental Evaluation

Page 30: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Implementation

• 2-week studies in two setups: home and office space

• Occupants used their own cellphones, did not install an app• One time registration with the system

• Required no changes to user behavior

Page 31: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Implementation: Home

DeepakHomeRevised

13m

9m

LivingRoomCouch

Bed

TV

• 2 occupants, 2 frequent visitors

• Smallest area: couch (1.3 m2)

Page 32: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Implementation: Office

15m

OfficeA OfficeB

OfficeC

10m

LabSetup

• Office A: 3, Office B: 5, Office C: 1 occupants

Page 33: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Implementation: Office

15m

OfficeA OfficeB

OfficeC

10m

LabSetup

8.5m

4m

Page 34: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Evaluation: Accuracy

16.5

41.7

96.4 94.8

0

20

40

60

80

100

Home Office

Accu

racy

(%)

Device Only Duet

Page 35: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Evaluation: Event Accuracy

36.3

44

94.6 93.4

0

20

40

60

80

100

Home Office

Even

t Acc

urac

y (%

)

Device Only Duet

Page 36: Making Localization Work for Smart Homespeople.csail.mit.edu/deepak/assets/slides/duet_UbiComp_2018.pdf · Making Localization Work for Smart Homes ShichaoYue Presenting on behalf

Conclusion

• Duet: Combine information from multiple modes of RF tracking

• Uses First Order Logic based reasoning to overcome intermittent, partially correct information

• User study over two weeks and two different environments