SurroundSense

SurroundSenseMobile Phone Localization via Ambience Fingerprinting

Scott SetoCS 495/595

November 1, 2011http://scott-seto.com/surroundsense

Introduction

• Mobile phones are becoming people-centric

• Location-based advertising is coming soon

• There is an absense of well-established logical localization schemes

• Physical localization does not work well indoors

What is SurroundSense?

• Uses the overall ambience of a place to create a unique fingerprint for localization

• Fingerprint location based on ambient sound, light, color, RF, etc.

• Sensor data is distributed to different modules

Motivation

• Installing localization equipment in every area is unscalable

• A scheme with accuracy of 5 meters may not place a person on the correct side of a wall

Challenges

• Fingerprints from various shops vary over time

• Colors may be different based on daylight or electric light

• A sound fingerprint from a busy hour might not match a low-activity period

SurroundSense Architecture

Detecting Sound

• Ambient sound can be suggestive of the type of place

• Use sound as a filter• Eliminate outliers• Compute the pair-wise Euclidean

distance between candidate and test fingerprints

Detecting Motion

• People are stationary for a long period in restaurants and less in grocery stores

• Place motion fingerprints into buckets

• Differentiate between sitting and moving places

Detecting Color/Light

• Extract dominant colors and light intensity from pictures of floors

• Translate the pixels to the hue-saturation-lightness (HSL) to decouple the actual floor colors from the ambient light intensity

Fingerprinting Wifi

• Adapt existing WiFi based fingerprinting to suit logical localization

• Use the MAC addresses of visible APs as an indication of the phone’s location

• Avoid false negatives

Implementation

• Groups of students visited 51 stores using a Nokia N95 phone running SurroundSense

• Collected fingerprints from each store

• Visited each of them in groups of 2 people (4 people in total).

• Keep the camera out of pocket

Implementation

• While in the store, try to behave like a normal customer

• Went to different stores so that the fingerprints were time-separated

• Mimiced the movement of another customer also present in that store

• No atypical behavior: one may interpret the results to be partly optimistic

Future Work

• Independent research on energy efficient localization and sensing

• Use the compass to correlate geographic orientation to the layout of furniture and shopping aisles in stores

• Group logical locations into a broader category

Conclusion

• SurroundSense fingerprinted a logical location based on ambient sound, light, color, and human movement

• Created a fingerprint database and performed fingerprint matching for test samples

• Localization accuracy of over 85% when all sensors were employed for localization

Questions?