CrowdMap: Accurate Reconstruction of Indoor Floor Plans from Crowdsourced Sensor- Rich Videos Si Chen, Muyuan Li, Kui Ren, Chunming Qiao Department of Computer Science and Engineering University at Buffalo – State University of New York
CrowdMap: Accurate Reconstruction of Indoor Floor Plan from Crowdsourced Sensor-Rich Videos
Aug 10, 2015
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
CrowdMap: Accurate Reconstruction of Indoor Floor Plans from Crowdsourced
Sensor-Rich Videos
Si Chen, Muyuan Li, Kui Ren, Chunming Qiao Department of Computer Science and Engineering
University at Buffalo – State University of New York
Page 2
Outline
1. Introduction
2. System Architecture and Design Details
3. Implementation and Evaluation
4. Future Work
Page 3
Outdoor Maps
Google Map
OpenStreetMap
Page 4
Indoor Maps (Floor Plans)
Unlike outdoor environment, acquiring digital indoor floor plan information is very challenging.
The state-of-the-art Google Indoor Maps only have 10,000 locations available worldwide, which is not in a position to compete with the total number of indoor environments around the world.
Page 5
Indoor Maps
The complexity of the indoor environment is the major obstacle to achieve ubiquitous coverage.
Existing centralized collection and on-site calibration techniques demand professional devices and multi-party coordination, which are time consuming, inconvenient and costly.
Google Trekker http://www.navvis.lmt.ei.tum.de/about/
Page 6
Crowdsourcing
http://cdn.thephonebulletin.com/wp-content/uploads/2013/01/smartphone-sensors.jpg
Recently, the wide availability of smartphones and wearable devices (e.g. google glasses) equipped with built-in visual, acoustic and inertial sensors makes the mobile user easier than ever to devote themselves to contribute mobile data.
Page 7
Crowdsourcing
http://senda.uab.es/node/15
Page 8
Indoor Floor Plan Reconstruction by Crowdsourcing
There have been several studies trying to explore the possibility of using crowdsourced inertial sensory data to generate an indoor floor plan automatically.
Alzantot, Moustafa, and Moustafa Youssef. "Crowdinside: automatic construction of indoor floorplans." Proceedings of the 20th International Conference on Advances in Geographic Information
Systems. ACM, 2012.
Page 9
Drawback of Inertial Sensor Only Methods
However, current crowdsourcing floor plan reconstruction systems are not able to produce accurate enough results.– Most of existing indoor floor plan reconstruction systems heavily rely on inertial
data.
The premise of their work is that users would be able to move across all edges and corners in an indoor environment.
• the edge of an indoor scene is usually blocked by furniture or other objects,
• some restricted areas in an indoor environment are also inaccessible for most of the users
Visual information preserve more context information for an unknown indoor environment, such as the geometric information, color information, lighting conditions and text information.
[1].http://www.sigmobile.org/mobicom/2014/talks/slides_6_3.pdf
[1]
Page 10
CrowdMap: Key Idea
CrowdMap: An accurate indoor floor plan reconstruction system based on sensor-rich videos.
Key idea: leverage the spatio-temporal relationship between each consecutive frame of the crowdsourced video.
Page 11
CrowdMap: Floor Plan
Basic elements of building floor plan
Page 12
Outline
1. Introduction
2. System Architecture and Design Details
3. Implementation and Evaluation
4. Future Work
Page 13
CrowdMap: System Architecture
mobile front-end
cloud back-end
i) crowdsourced data collection module
ii) indoor path modeling module
iii) room layout modeling module
iv) floor plan modeling module
Page 14
CrowdMap: Crowdsourced Data Collection Module
We assume that users actively get involved in the data collecting tasks. Example: a user opens our mobile application and inputs the floor number, starts capturing the room environment by spinning his/her body (SRS task); then, walks towards the hallway (SWS task).
The walking distance |AB| is calculated by the step counting method. In addition, the direction change of each step is calculated by jointly using compass, gyroscope and accelerometer.
Page 15
CrowdMap: Crowdsourced Data Collection Module
By using the inertial sensor data, we are able to reconstruct the trajectory of the user when they perform the SWS task.
Page 16
CrowdMap: Indoor Path Modeling Module
How to aggregate multiple user trajectories and reconstruct the path of the floor?
Page 17
CrowdMap: Indoor Path Modeling Module
[1].Bay, Herbert, Tinne Tuytelaars, and Luc Van Gool. "Surf: Speeded up robust features." Computer vision–ECCV 2006. Springer Berlin Heidelberg, 2006. 404-417.
We select the state-of-the-art SURF [1] algorithm to precisely match two candidate key-frames
Page 18
CrowdMap: Indoor Path Modeling Module
Correct Matching
Page 19
CrowdMap: Indoor Path Modeling Module
Incorrect Matching
Page 20
CrowdMap: Indoor Path Modeling Module
We use multiple key-frames to determine whether the two user trajectories can be merged.
If there is a match between the two trajectories generated in the same floor, there should be a common path between them in a high probability. Hence, we use the longest common subsequence to capture this notion.
Where Ta and Tb are the two user trajectories with length of i and j, respectively. Parameter represents the maximum length difference between two user trajectories and is the distance threshold.
Page 21
CrowdMap: Indoor Path Modeling Module
Page 22
CrowdMap: Room Layout Modeling Module
We utilize crowdsourced images to create the panorama for each place, and then use computer vision techniques to process the panorama, and thereby, generate the room layout.
Page 23
CrowdMap: Room Layout Modeling Module
Page 24
CrowdMap: Room Layout Modeling Module
Page 25
CrowdMap: Floor Plan Modeling Module
Page 26
CrowdMap: Floor Plan Modeling Module
Page 27
CrowdMap: Floor Plan Modeling Module
Page 28
Outline
1. Introduction
2. System Architecture and Design Details
3. Implementation and Evaluation
4. Future Work
Page 29
CrowdMap: Implementation
Page 30
CrowdMap: Implementation
Screenshot for CrowdMap Mobile Frontend
Page 31
Microsoft Azure Platform
Page 32
CrowdMap: Evaluation
Page 33
CrowdMap: Evaluation
Page 34
Compare with Structure from Motion (SfM)
Page 35
Outline
1. Introduction
2. System Architecture and Design Details
3. Implementation and Evaluation
4. Future Work
Page 36
CrowdMap: Future Work
We will focus on further processing of the room panorama to extract more context information of the room:
• object detection• object recognition.
We also plan to further study several issues:• user incentive mechanism• privacy preservation mechanism.
Page 37
Q & A
Related Documents
