Pervasive Indoor Localization and
Tracking Based on Fingerprinting
Gary Chan
Professor, CSE
HKUST
2
Catchphrase: Location, Location,
Location!
3
Outdoor Location-Based
Services (LBS)
• Based on GPS (Global Positioning System) with satellites– Widely and successfully commercialized
• Navigation, places of interest, hotels, parks, gas stations, etc.
4
Indoor Location-Based
Services (LBS)
Even larger and more lucrative market: more than 70%
of our time is spent indoors Don Dodge, developer advocate at Google
5
Market Opportunities
• A multi-billions market
• Malls, super-marts or exhibition centers– Help customers to find items, shops or booths
– Location-based recommendations, mobile advertisement and marketing
– Enable O2O (online-to-offline) business
– Track people and crowd/flow analysis
– Customer behavior
• Airport– Help tourists to find their gates for flights
– Shop while waiting for flights
• Social applications: finding nearby people sharing the same interest
• Location-based games (e.g., treasure hunt)
• Track assets
• Health– Track people, patients, etc.
Indoor Positioning:
Localization and Tracking
• Localization: Where am I?
• Tracking: Where are you?
User experience depends largely on the
accuracy of the technique!7
Challenges of Indoor GPS
• Global Positioning System (GPS) simply does not work indoor!– Line-of-sight not easily achievable indoor
• Complicated indoor environment leads to complex signal fading, shadowing and interference
• High requirements– Cost-effective to deploy
– High accuracy (~5m)
– Computationally light-weighted
– Power and memory efficient
– Privacy, etc.
8
Application Venues
• Malls are the most
mapped venues around
the world
• Airports come second
• Growing works on
advertising and analytics
to target customers in
these venues
Commerce79%
Airports13%
Other8%
4th Qtr0%
9
Wherami: The HKUST
Indoor Positioning Technology
10
Turning Research to
Commercialization
• Government– ITC LSCM
– Innovation and Technology
Funds (ITF)
• Technology Transfer
Center, Department and
School of Engineering at
HKUST– Advices and support on
contractual agreements
– R&D facilities and human
resources
• Industry sponsors and
collaborators– Funding, technical
support, trial sites,
equipment, etc
Tech.
Transfer
Wherami
11
Use of Wi-Fi for Positioning
• Pervasive deployment– In HKUST atrium, we can detect more than 400 AP hot
spots (HKUST 2013-08-08)
• The dominant localization technology by
2015
• Easy and cost-effective to deploy
(source: ABI research indoor location report)
12
Wi-Fi (Possily combined
with sensor fusion, Cell or
other)75%
Bluetooth 4.015%
Other (LED...)
10%
Wherami: Pervasive Indoor
Positioning
• Supported by ITC LSCM and industries
• Smart guide on smart-phones– Beyond just “digital map”
• Indoor navigation: How can I get there?
• Are any good promotions being offered around me?
• Are my friends around me? Where are my kids?
• Where are my customers? What are their behaviors?
• …
13
Fingerprint Concept
• Each person has a unique fingerprint
– Stored in a database
• Given a fingerprint
– Match it to the database
– Find the person
• Wi-Fi fingerprinting
– Fingerprint = Wi-Fi signals
– Person = Physical location
14
Wi-Fi Signals Collected at a
Location
15
The Wi-Fi Signals Form a Unique
Fingerprint for the Location
Site Survey and Fingerprint
Collection
User Location Calculation
Research Challenge: Signal
Measurement Noise
• Minimize the overall expected signal difference in signal map
• Subject to constraints on possible regions the target may be in
• 𝑚𝑖𝑛𝑖𝑚𝑖𝑧𝑒
– 𝑖𝑁𝑚 Δ 𝑝, 𝑞𝑖 ⋅ 𝜔𝑖
• 𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜– 𝑥 = 𝑖 𝜔𝑖 ⋅ 𝑟𝑖 , 𝑟𝑖 ∈ 𝐶𝑚 ,
– 𝑥 − 𝑥𝑙 ≤ 𝑑𝑙 , 𝑙 ∈ Λ
– 𝑖𝑁𝑚 𝜔𝑖 = 1, 0 ≤ 𝜔𝑖 ≤ 1
Stochastic Optimization
under Noisy Signals
Client-based Localization
• App-based
– Android + iOS
• Non-intrusive and transparent to existing
WiFi infrastructure
• “No string attached” besides your phones
Integration with Other Signals
• Bluetooth (BLE)/iBeacon
• Dead reckoning/INS (inertial
navigation system)
– Step counter, gyroscope,
accelerometer, etc.
• Magnetic field
• GPS/AGPS/NLP (Network
location provider)
• Barometer
• Ultrasound
• RFID
• Etc.22
Beacon 1 Beacon 2
Beacon 4
Beacon 3
• Good improvements on the area where WiFisignals are weak
BLE (Bluetooth Low Energy)
integration
23Case I – do nothing Case II – Restrict the estimation
Automatic Fingerprint Updating
for Survey Reduction
• Using Implicit crowdsourcing and Gaussian process regression (GPR)
• The samples has a joint Gaussian distribution
• Regression function
• Specified by mean and covariance function
• Joint Gaussian
24
𝑣 = 𝑓 𝑙 + 𝜀
𝑓 𝑙 ~ 𝐺𝑃(𝑚 𝑙 , 𝑘(𝑙, 𝑙′))
𝑐𝑜𝑣 𝒗 = 𝑲 + 𝜎𝑛2𝑰𝒗 ∼ 𝑵(𝒎 𝑳 , 𝑐𝑜𝑣 𝒗 )
𝒇∗|𝑳, 𝒗, 𝒍∗ ∼ 𝑵(𝝁∗, 𝜎∗
2)𝒇∗
𝒗∼ 𝑵(
𝒎(𝒍∗)
𝒎 𝑳,𝒌(𝒍∗, 𝒍∗) 𝒌 𝒍∗, 𝑳 𝑻
𝒌(𝒍∗, 𝑳) 𝑲 + 𝜎𝑛2𝑰
)
𝜀 ∼ 𝑵(𝟎, 𝜎𝑛2)
AP-Based Solution for
User Tracking
25
Design Dimensions
Without infrastructure
installation
Infrastructure outlay
(APs and iBeacons)
No client app installation
Client-based apps
26
User tracking and
analytics
User localization
and navigation
Better user positioning
accuracy and analytics
AP-based Tracking vs Client-
based Localization
AP-based Client-based
Device
Capability
Any Wi-Fi devices (including iOS,
Andriod, Wi-Fi tag, Windows, etc)
Require Wi-Fi API
Need ibeacon to localize iphones
Tracking
Transparency
Transparency to client side Client app interacts with the users
Accuracy
Improvement
Installation of more APs
Antenna improvement
Fusion of different signals
Computation Server Client, hence the device can be offline
Positioning
interval
Seconds Real time (less than a second)
Wherami Technology Features
• Driven by system research and advanced development
• Full solution suite for user positioning
• AP-based tracking
• Client-based localization– Joint AP-client localization to further improve accuracy
• Indoor-outdoor and multi-storey roaming (3D routing)
• Low cost, easy and fast to deploy
• Accurate and low computation – Precision 2m ~ 10m
• Low site survey cost– Adaptive to signal changes
• Ready to be integrated into other applications– Location API is provided
28
29
Main Functions
• Real-time positioning
• Real-time navigation– Destination search
– Best path selection
– Moving trajectory
– Panoramic visual clue
• Targeted services– Nearby facility search
– Program search
– Event/Ad prompt
Shopping Mart or Superstore
• Aim for better shopping
experience in large area
with many aisles
– Location-based promotions
– For bargain hunters
– For shoppers with a
shopping list
• Targeted advertisement
(behavior analysis)
30
Shopping Mall: Pushed Ads
31
Hospital, Out-Patients, Kids
and Elderly home
• Smart Wrist Band– Bluetooth or Wi-Fi enabled
– Detect the heart rate, step count, sleep status,
etc
• Tracking by SmartAP– Track and trace the location of patient/elderly
– Real time monitor and report the health
condition
33
Context-aware Computing or Smart
Space: Museum and Theme Parks
Location-based Games and
Treasure Hunting
34
Intelligent Transportation
Deployment Trials: HKIA,
Cyberport, Olympian City, Union
Hospital, HKCEC, HKUST, etc.
36
• Where is the check-in counter for Airline X?
• How to get to Gate 24?
• The closest restaurant?
• Push ads: there is a discount in the coffee shop nearby!
Hong Kong International Airport
37
HKIA trials
38
Checkin asile
K, J, H
Gate 20~26
Meeting Hall
Trials in HKIA
39
How we conduct trials
40
1. Identify reference points
2. Collect WiFi fingerprinting
3. WiFi data analysis and pre-process
Demonstration (youtube
video)
41
Cyberport Trial
42
Trial area:
1. The Arcade – whole
building
2. Cyberport 1 – floor L
3. Cyberport 2 – floor L
4. Cyperport 3 – floor L3
and L4
43
LBS in Cyberport
Location advertisementLocation game: treasure hunt Panoramic Visual Clue
Trial at Olympian City
44
• Where is the Pathology Centre?
• Where is the nearest washroom or food court?
Hospital
45
• Where is the xyz company booth?
• Are there any forum discussions this afternoon and where are they held?
• Where is the “3D Printing Solutions” zone?
• Where is my friend?
Hong Kong Convention and
Exhibition Centre
46
HKUST Campus
47
Trial area:
1. Academic Building
2. Lee Shau Kee
Business Building
3. Institute of Advanced
Study Building
• Where is classroom for Course X or Seminar Y?
• How to get to Room Z?
• My friend shares his position to me – I am going for a discussion with him!
HKUST Deployment
48
49
Wherami main functions
Navigation Content SearchLocalization
50
Wherami Trial Venues
Core Features Value Added Features
Wherami Trial Venue
Offline M
ap
Navig
ation
Targ
et
Searc
h
Augm
ente
d
Panora
mic
Vie
w
Location-b
ased A
ds
Cours
e/F
light
Searc
h
Sem
inar/
Pro
motion
Events
Pro
motion g
am
es
(e.g
. tr
easure
hunt)
Outp
atient
Pro
cedure
Flo
w
Real Tim
e Q
ueue
Sta
tus
Patient
Identity
Verification
Real Tim
e F
low
Contr
ol
University Campus• HKUST
Shopping Mall• Cyberport and OlympianCity
Airport• HKIA
Hospital• Union Hospital
Exhibition• HKCEC
Visit us at:
http://mwnet.cse.ust.hk/wherami
http://www.cse.ust.hk/~gchan
We are Happy to
Demonstrate Our
Technology at Your Sites
51