Using Video Analysis and Tracking and Managing Resources
Post on 30-Apr-2022
1 Views
Preview:
Transcript
Using Video Analysis and Ad-hoc Wireless Connection Technologies to
Detect Specific Objects for Tracking and Managing Resources
Turning Videos into LBS
LSCM LBS Forum October 20, 2015
Dorbin Ng, PhD
Department of Systems Engineering and Engineering Management The Chinese University of Hong Kong
3
LBS for Timely Resource Management
Otherwise… Shortages
Bottlenecks
Cascading Delays
Service Disruptions
Resources E.g., staff & equipment
Supporting consumption for facilitating operations
Timely replenishment for ensuring seamless operations
Knowing the shortages Triggering timely actions
Equipment in Hospital
4
Will wheelchairs or stretchers be available when needed for patient care?
Trolleys in Airport
5
Looking for trolleys?
Replenishing trolleys in timely manner?
Stroller Services in Malls & Airports
6
Are they available when needed?
What are the Possible LBS Solutions for Tracking & Managing Resources?
7
RF Solutions?
Tagging all items?
Reader network infrastructure?
Maintenance cost?
Setup cost?
Visual/image solutions?
Setup cost?
Identification?
Camera network infrastructure?
Maintenance cost?
Robotic solutions? Proximity solutions? … Multi-modal solutions? What would be the right mix of modals? …
Video Analytics for Resource Tracking & Management
Camera/CCTV Infrastructure
Enabling Technologies Real-time
Notifications
Turning Videos into Tracking Data & Actionable Intelligence
Resource Analytics
8
Video Sensing Management & Analytics Architecture Framework
9
Video Sensing Management & Analytics Architecture Framework
Video Stream
Feature Extraction
Object Detection
Object Learning Module
Sensor Devices
Alert Management
Monitoring Visualization
Resource Visualization
Content Analysis
Application & Presentation
Video & Sensor Input Sources
Event Detection for Alert Notification
Object & Event Monitoring
Management Module Analytics Module
Resource Modeling & Management
Monitoring Modeling & Management
Monitoring & Management
Platform
Making use of computation power, machine learning, image processing, & system engineering
Providing monitoring & counting capability for resource management
Image Feature Extraction: Motivation
A subjective decision on defining similar images
Different people interpret similarity differently
Even the same person interprets similarity differently in different situations
Difficulties on searching similar images
Choosing features to represent images
Color, texture, shape, object, semantics, …
Accessing images from the defined, extracted features
High dimensional feature vectors with continuous values
Efficiency in finding similar feature vectors in a high dimensional space
Enabling multi-mode search on multimedia data 10
Image Features
Images Containing Similar Colors
11
Images Containing Similar Shapes
12
Images Containing Similar Content Soccer game
Mountains as prominent scenery
13
Challenges in Image Similarity
14
• Similar color distribution
• Similar texture pattern
• Similar shape/pattern
• Similar real content
Deg
ree
of
dif
ficu
lty
Histogram matching
Texture analysis
Edge, contour analysis
Life-time goal :-)
• Similar object Image Segmentation, Pattern recognition
Image Feature Extraction Techniques
Color histogram
Histo144 image feature extraction
Relying only on color information
Color and texture histograms
MCP image feature extraction
Combining texture and color information
Color coding
Cuebik image feature extraction
Coding a color to each of the pre-defined image region
More features available via OpenCV APIs
http://docs.opencv.org/modules/refman.html
OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library.
15
Color Histogram: Histo144
Dividing an image into 3x3 regions
Possible to have 5x5, 7x7, … regions
For each region, calculating a 16-bin color histogram using
Munsell Color Space
Which uses hue, chroma, and value instead of red, blue, and green
In which close colors are perceptually similar.
Combining all regions’ color histograms to form
A 144-dimensional feature vector
i.e., 144 doubles representing an image
Reducing the dimension to 50 by a Single Value Decomposition (SVD) matrix, trained by corpus-specific images 16
Histo144 Feature Extraction Results
17
Texture & Edge Detections
18
19
Slide from http://images.slideplayer.com/14/4246341/slides/slide_4.jpg
Object Detection: Motivation
Human perceives objects (e.g., faces, trolleys) easily
But mysteriously in terms of how it is being done
Many different object types exist in environment
Groups of people, vehicles, chairs, etc.
Objects in each object type may have variations but have common features.
Automated object detection can be useful in
Supporting image retrieval
Labeling and categorizing images
Enhancing photography for automatically focusing, color balancing, and zooming on a specific object
Building systems for automatic object identification
Counting number of objects for management and decision making purposes
20
Object Detection & Machine Learning
Challenges in Object Detection
Amount of variation in visual appearance Color, texture, contour, shape, etc.
Orientation and rotation with respect to the camera
Distance from the camera
Variation in surrounding environment Light sources
Varying intensity, color, and location with respect to the object
Nearby objects Casting shadows on the object
Reflecting additional light on the object
21
Object Detection Techniques
A two-part strategy to cope with all this variation
View-based approach
Multiple detectors to deal with variation in orientation
Statistical modeling
Within each detector to account for the remaining variation
22
View-based Detectors
Independent development of individual detectors
Each detector specialized to
A specific orientation of the object; e.g.:
Specialized to the different rotating views of objects
A specific size within a rectangular image window
Operation
Re-applying detectors for all possible positions
Re-applying detectors for all resized images
Combining individual detectors’ results
Choosing the strongest detection if multiple detections happening at the same or adjacent locations
23
Statistical Form of Detector
Two statistical distributions are modeled within each view-based detector:
The statistics of the given object is modeled as P(image | object)
The statistics of the rest of the visual world, called “nonobject” class, is modeled as P(image | nonobject)
Detection decision using the likelihood ratio test
If the likelihood ratio (the left side) is greater than the right side, the object is decided to be present
24
P( image | object ) >
P( nonobject )
P( image | nonobject ) P( object )
Learn
Detect
Training Object Detectors via Machine Learning Techniques
25
Training Image Data
Testing Image Data
New Image Data
Machine Learning
Feature Extraction
Object Detector
Result Evaluation
Trained Object
Detector
Detected Objects
Feature Extraction
Feature Extraction
Support Vector Machine (SVM) A supervised learning technique to define a discriminative
classifier by a separate hyperplane
Given a set of labeled data, SVM yielding an optimal hyperplane, which is able to categorize new examples
26
Multiple green lines offer a solution to separate 2 groups of objects. Is any of them better than the others?
SVM gives the largest minimum distance to the training data by having the optimal separating hyperplane maximizing the margin of the training data.
Graphics from http://docs.opencv.org/doc/tutorials/ml/introduction_to_svm/introduction_to_svm.html
Trolley Detection
27
Transform Detect
Filter Transform
Input
Real-time Notification for Trolley Shortage
28
Image processing Feature extraction Feature grouping
Machine learning Feature selection Ensemble methods
Difficulties Different viewing
angles, sizes, orientations, & lighting conditions
Leveraging on Large quantity of
same type of rigid objects
Pilot Project: Trolley Management at HKIA
18 Trolley Stations in HKIA Baggage Reclaim Hall
2 3 4 5 9 8 7 6 10 11 12 13
Custom Hall
Immigration
Trolley pick up point
5.2B
5.2A
5.4A
5.4B 5.5B
5.5A
5.6B
5.6A
5.8B
5.7A 5.7B
5.8A
5.9B
5.9A
5.10B
5.10A 5.12A
5.12B
Trolley Stations in HKIA Baggage Reclaim Hall
29
About another 100 trolley stations in airside Departure & Arrival Halls and at curbside outside Terminals 1 & 2
Which station(s) is out of trolleys? Or, having shortage? What is the efficient way to orchestrate limited staff resources to timely replenish trolleys to the right station(s)?
30
Trolley shortage
notification
Real-time Notification
via Mobile App
(iOS & Android)
Out of trolley notification
Notification history
31
1. Photo viewing for the particular message being selected in the “Message” screen
2. Photo view supporting zoom-in & zoom-out functions
1. Default view with a full photo fitting on the screen
Message
32
Performance
Hour-box view for each location
Chart view showing
details on performance
“Daily Record” & “History Record”
on following pages
33
Performance
1. “Daily Record” view with hourly timeline information for a selected date
2. Current date being the default selected date
34
Performance
1. Selecting date 2. Clicking “Add” to the date list 3. Repeating #1 & #2 for selecting
more dates 4. When done with selecting
date(s), clicking “Generate” to see a chart view
Chart view showing
details on performance
on selected date(s)
Clicking “Save” to capture the chart views to
an image file
35
Map
Select Location Menu Button
36
Oct 2 Fri
Oct 3 Sat
Oct 4 Sun
Oct 5 Mon
Oct 6 Tue
Oct 7 Wed
Oct 8 Thu
Typhoon Mujigae being within 400
km from HK
Normal day pattern
Flight schedule disruption 1
Flight schedule disruption 2
Flight schedule disruption 3
Flight schedule disruption 4
Normal day pattern being
resumed
An
alytics: Re
sou
rce M
on
itorin
g B
usin
ess In
tellige
nce
Acknowledgement
37
• Video Analytics for Resource Management
2015.09.22 – 2017.09.21, ITP/048/15LP, Prof. CH Cheng, Dr. Dorbin Ng, Mr. Tim Chan
Asian Institute of Supply Chains & Logistics, CUHK • Resources Management via Real-time RFID Tracking Data Analytics – A Case in the
Healthcare Industry 2014.01.01 – 2015.12.31, Prof. CH Cheng, Dr. Dorbin Ng
• Image-based Drug Verification for Risk Management in Drug Dispensing 2014.10.01 – 2016.09.30, Prof. CH Cheng, Dr. Dorbin Ng
http://www.se.cuhk.edu.hk/~ssi/
top related