Marcos Vinícius Naves Bedo (speaker) [email protected] Gustavo Blanco, Willian Oliveira, Mirela Cazzolato, Alceu Costa, José F. Rodrigues Jr., Agma Traina, Caetano Traina Jr. Techniques for effective and efficient fire detection from social media images
Techniques for effective and efficient fire detection from social media images
Jul 17, 2015
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Marcos Vinícius Naves Bedo (speaker) [email protected]
Gustavo Blanco, Willian Oliveira, Mirela Cazzolato, Alceu Costa, José F. Rodrigues Jr.,
Agma Traina, Caetano Traina Jr.
Techniques for effective and efficient fire detection from social media images
junio
Typewriter
Full paper at: http://www.icmc.usp.br/pessoas/junio
junio
Typewriter
junio
Typewriter
Outline
Introduction
Background
The Fast-Fire Detection Method
Experiments
Conclusions
Outline
Introduction
Rescuer Project
Fire Detection Module
Background
The Fast-Fire Detection Method
Experiments
Conclusions
Rescuer Project
The RESCUER project is a BR-EU consortium aiming at developing solutions to improve the decision-making process in disaster situations:
Industrial plants;
Densely populated area;
Crowded events;
Project details: http://www.rescuer-project.org/
A smartphone user
sends textual data
about the situation.
The user may also upload
multimedia content such as
photo and video
Multimedia
data are
automatically
analyzed
http://www.rescuer-project.org/
A smartphone user
sends textual data
about the situation.
The user may also upload
multimedia content such as
photo and video
Multimedia
data are
automatically
analyzed
http://www.rescuer-project.org/ http://www.rescuer-project.org/
A smartphone user
sends textual data
about the situation.
The user may also upload
multimedia content such as
photo and video
Multimedia
data are
automatically
analyzed
http://www.rescuer-project.org/ http://www.rescuer-project.org/
A smartphone user
sends textual data
about the situation.
The user may also upload
multimedia content such as
photo and video
Multimedia
data are
automatically
analyzed
http://www.rescuer-project.org/ http://www.rescuer-project.org/
Image Analysis
• Fire detection
– Presence of fire images located near an emergency scenario
Image Analysis
• Fire detection
– Presence of fire images located near an emergency scenario
Image Analysis
• Fire detection
– Presence of fire images located near an emergency scenario
Identify fire presence in images
arriving from the Flickr social network
Problem Definition
• Problem Definition
– Given an image previously updated to a social media service, return 'true' if there is fire or 'false' otherwise.
Outline
Introduction
Background
The Fast-Fire Detection Method
Experiments
Conclusions
Outline
Introduction
Background
Feature Extractor Methods
Evaluation Functions
Instance-Based Learning
The Fast-Fire Detection Method
Experiments
Conclusions
Fire Detection Module
• Feature Extractor Methods
– MPEG-7: designed to represent color, texture and shape
– Standardize representation for color images
MPEG7 - Color Extractor Methods
Spatial Correlation Number of Features Color Space
Color Layout Yes 16 YCbCr
MPEG7 - Color Extractor Methods
Spatial Correlation Number of Features Color Space
Color Layout Yes 16 YCbCr
Color Structure Yes 128 HMMD
MPEG7 - Color Extractor Methods
• Hue
• Saturation
• Value
Haar transform
Spatial Correlation Number of Features Color Space
Color Layout Yes 16 YCbCr
Color Structure Yes 128 HMMD
Scalable Color No 256 HSV
MPEG7 - Color Extractor Methods
Spatial Correlation Number of Features Color Space
Color Layout Yes 16 YCbCr
Color Structure Yes 128 HMMD
Scalable Color No 256 HSV
Color
Temperature
No 1 XYZ
MPEG7 - Texture Methods
- Local Count
- Global Count
Count Approach Number of Features
Edge Histogram Yes 150
MPEG7 - Texture Methods
Count Approach Number of Features
Edge Histogram Yes 150
Texture-Browsing No 12
Evaluation Functions
Evaluation Function Distance Function Acronym
City-Block Yes CB
Euclidean Yes EU
Chebyshev Yes CH
Canberra Yes CA
• We employed six evaluation functions as possibles setting to the classification task
Evaluation Functions
• We employed six evaluation functions as possibles setting to the classification task
Evaluation Function Distance Function Acronym
City-Block Yes CB
Euclidean Yes EU
Chebyshev Yes CH
Canberra Yes CA
Kullback-Leibler No KU
Jeffrey Divergence No JF
Image Descriptors
• Image Descriptor
– An image descriptor is a pair <feature extractor method, evaluation function>
– By using the previous evaluation functions and feature extractor methods, 36 can be arranged.
– Image descriptors define the search space.
24
Instance-Based Learning
• Assumption: Elements of the same class belong to the same neighborhood
Iq
Instance-Based Learning
• Assumption: Elements of the same class belong to the same neighborhood
Iq
Instance-Based Learning
• Assumption: Elements of the same class belong to the same neighborhood
Iq
Instance-Based Learning
The Iq is labeled according to its k
nearest neighbors.
Iq
Outline
Introduction
Background
The Fast-Fire Detection Method
Experiments
Conclusions
Outline
Introduction
Background
The Fast-Fire Detection Method
Architecture
Fire-Flickr Dataset
Experiments
Conclusions
Fire Detection Module
Image
Fire Detection Module
0.7 0.4 0.1 0.9 0.2 ...
Data representation
through a Feature
Extractor Method
Image
Fire Detection Module
0.7 0.4 0.1 0.9 0.2 ...
Data representation
through a Feature
Extractor Method
Image
Classifier
Knowledge
Database
Fire Detection Module
0.7 0.4 0.1 0.9 0.2 ...
Data representation
through a Feature
Extractor Method
Image
Data classification –
may require an
Evaluation Function
Classifier
Knowledge
Database
The FFireDt Method
• FFireDT: Our proposal
– Setting: Image Descriptor
– Set of modules to perform image analysis:
• Feature Extractor Module
• Evaluation Functions Module
• IBL classifier module
The FFireDt Method
The FFireDt Method
The FFireDt Method
Fire-Flickr Dataset
• Downloaded 5,962 images from Flickr API
– Textual descriptors as ‘fire car accident’, ‘criminal fire’, ‘house burning’, etc.
– 7 subjects (non-blinded) evaluated the images as containing or not traces of fire
• Average disagreement 7.2%
– 1,000 images with and without fire
Fire-Flickr Dataset
{fire}
{not fire}
• Dataset avaliable at: www.gbdi.icmc.usp.br
– Including the extractors and functions
Outline
Introduction
Background
The Fast-Fire Detection Method
Experiments
Conclusions
Outline
Introduction
Background
The Fast-Fire Detection Method
Experiments
F-measure
Precision x Recall and ROC curves
Performance Evaluation
Conclusions
Experiments
• Metrics to evaluate FFireDt
– Test to evaluate F-Measure
– Precision vs. Recall curves
– ROC curves
• Processing Performance
– Image Descriptors
• ‘Cost x Benefit’ Analysis
Color
Layout
Scalable
Color
Color
Structure
Color
Temperature
Edge
Histogram
Texture
Browsing
0.9
0.85
0.8
0.75
0.7
0.65
0.6
0.55
0.5
0.45
City-Block
Jeffrey Divergence
Canberra
Kullback Leibler
Euclidean
Chebyshev
Results
• F-Measure using all possible settings for FFireDt
F-Measure -> Higher is better
Results
• F-Measure using all possible settings for FFireDt
0.847
Image Descriptor <Color Layout, Euclidean>
Color
Layout
Scalable
Color
Color
Structure
Color
Temperature
Edge
Histogram
Texture
Browsing
0.9
0.85
0.8
0.75
0.7
0.65
0.6
0.55
0.5
0.45
City-Block
Jeffrey Divergence
Canberra
Kullback Leibler
Euclidean
Chebyshev
Results
• F-Measure using all possible settings for FFireDt
Color
Layout
Scalable
Color
Color
Structure
Color
Temperature
Edge
Histogram
Texture
Browsing
0.9
0.85
0.8
0.75
0.7
0.65
0.6
0.55
0.5
0.45
City-Block
Jeffrey Divergence
Canberra
Kullback Leibler
Euclidean
Chebyshev
0.843
Image Descriptor <Scalable Color, City-Block>
Results
• F-Measure using all possible settings for FFireDt
Color
Layout
Scalable
Color
Color
Structure
Color
Temperature
Edge
Histogram
Texture
Browsing
0.9
0.85
0.8
0.75
0.7
0.65
0.6
0.55
0.5
0.45
City-Block
Jeffrey Divergence
Canberra
Kullback Leibler
Euclidean
Chebyshev
0.866
Image Descriptor <Color Structure, Jeffrey>
Results
• F-Measure using all possible settings for FFireDt
Color
Layout
Scalable
Color
Color
Structure
Color
Temperature
Edge
Histogram
Texture
Browsing
0.9
0.85
0.8
0.75
0.7
0.65
0.6
0.55
0.5
0.45
City-Block
Jeffrey Divergence
Canberra
Kullback Leibler
Euclidean
Chebyshev
0.800
Image Descriptor <Color Temperature, Canberra>
Results
• F-Measure using all possible settings for FFireDt
Color
Layout
Scalable
Color
Color
Structure
Color
Temperature
Edge
Histogram
Texture
Browsing
0.9
0.85
0.8
0.75
0.7
0.65
0.6
0.55
0.5
0.45
City-Block
Jeffrey Divergence
Canberra
Kullback Leibler
Euclidean
Chebyshev
0.815
Image Descriptor <Edge Histogram, Jeffrey>
Results
• F-Measure using all possible settings for FFireDt
Color
Layout
Scalable
Color
Color
Structure
Color
Temperature
Edge
Histogram
Texture
Browsing
0.9
0.85
0.8
0.75
0.7
0.65
0.6
0.55
0.5
0.45
City-Block
Jeffrey Divergence
Canberra
Kullback Leibler
Euclidean
Chebyshev
0.766
Image Descriptor <Texture Browsing, City-Block>
Results
• The top-6 image descriptors grouped by feature extractor methods were:
– ID1: Color Strucuture and Jeffrey Divergence
– ID2: Color Layout and Euclidean
– ID3: Scalable Color and City-Block
– ID4: Edge Histogram and Jeffrey Divergence
– ID5: Color Temperature and Canberra
– ID6: Texture Browsing and City-Block
Results
• The Precision-Recall curves show that ID1, ID2 and ID3 achieved a better behavior than others
Results
• The Precision-Recall curves show that ID1, ID2 and ID3 achieved a better behavior than others
Results
• The Precision-Recall curves show that ID1, ID2 and ID3 achieved a better behavior than others
–We discard the bottom-3 candidates
Results
• We checked the ROC curves for ID1, ID2 and ID3
Results
• We checked the ROC curves for ID1, ID2 and ID3
Results
• We checked the ROC curves for ID1, ID2 and ID3
Results
• We checked the ROC curves for ID1, ID2 and ID3
The choice becomes a
matter of
performance!
Results
• Processing Time
– Feature Extractor Method
Results
• Processing Time
– Feature Extractor Method
Results
• Processing Time
– Evaluation Function costs
Results
• Processing Time
– Evaluation Function costs
Results
• Performance Analysis (cost vs. benefit)
– Feature Extractor Methods
• Color Structure and Scalable Color
– Evaluation Functions
• City Block, Euclidean, and Chebyshev as Evaluation Functions
Results
• Performance Analysis (cost vs. benefit) – Color Structure and Scalable Color
– City Block,Euclidean, and Chebyshev 0.853
Color
Layout
Scalable
Color
Color
Structure
Color
Temperature
Edge
Histogram
Texture
Browsing
0.9
0.85
0.8
0.75
0.7
0.65
0.6
0.55
0.5
0.45
City-Block
Jeffrey Divergence
Canberra
Kullback Leibler
Euclidean
Chebyshev
Outline
Introduction
Background
The Fast-Fire Detection Method
Experiments
Conclusions
Conclusions
• We designed a new approach to fire detection: FFireDT
• FFireDT has achieved a precision closer to human annotation in fire detection
– Experiments show the precision and computational cost
– Determine the most suitable Image Descriptor as FFireDT setting
Thank you for your attention!
Techniques for effective and efficient fire detection from social media images
Marcos Vinícius Naves Bedo (speaker) [email protected]
Gustavo Blanco, Willian Oliveira, Mirela Cazzolato, Alceu Costa, José F. Rodrigues Jr.,
Agma Traina, Caetano Traina Jr.
Results
• FFireDt using Instance Based Learning vs. other classifiers
Color
Layout
Scalable
Color
Color
Structure
Color
Temperature
Edge
Histogram
Texture
Browsing
FFireDT
Naive-Bayes
RandomForest
J48
0.8
0.7
0.6
0.5
0.4
0.3
0.9
Color
Layout
Scalable
Color
Color
Structure
Color
Temperature
Edge
Histogram
Texture
Browsing
FFireDT
Naive-Bayes
RandomForest
J48
0.8
0.7
0.6
0.5
0.4
0.3
0.9
Results
• FFireDt using Instance Based Learning vs. other classifiers
Related Documents
