REAL-TIME OBJECT DETECTION AND TRACKING By: Vanya V. Valindria Hammad Naeem Rui Hua
Jan 04, 2016
REAL-TIME OBJECT DETECTION AND TRACKING
By:Vanya V. ValindriaHammad Naeem
Rui Hua
Outline• Introduction•Hardware in RT Object Detetion & Tracking •Methods
•Result and Conclusion
Traditional Methods:
Absolute Differences Census Method Feature Based
Method
Modern Methods:
KLT Meanshift
Introduction
Definition:Object detection detect a particular
object in an image
Object tracking to track an object (or multiple objects) over a sequence of images
Application: Traffic Information
http://www.youtube.com/watch?v=vA35sXbn7zs
Application: Surveillance
http://www.youtube.com/watch?v=o25fClk9cdg
Application: Mobile Robot
http://www.youtube.com/watch?v=Q4zycRGJFFs
Problems??
•Temporal variation/dynamic environment
•Abrupt object or camera motion
•Multi-camera? Multi-objects?
•Computational expensive
Hardware in Real-time Tracking
•MEMORYImportant Tracking system encountering limited memory
problems.
•FRAME RATE
~30 FPS
•PROCESSORS - DSP• Allow saturated arithmetic operation• Powerful operation ability• Can do several memory accesses in a single instruction
METHODOLOGIES
Object Detection and Tracking
• In a video sequence an object is said to be in motion, if it is changing its location with respect to its background
•The motion tracking is actually the process of keeping tracks of that moving object in video sequence i.e. position of moving object at certain time etc.
Flow ChartIdle
Imageacquisition
ObjectDetection
Imageacquisition
Objecttracking
ObjectLost? No
Yes
Method 1: Absolute Differences = Image subtraction D(t)=I(ti) – I(tj)
Gives an image frame with changed and unchanged regions
Ideal Case for no motion: I(ti) = I(tj), D(t)=0
Moving objects are detected
Results:
Frame1 Frame10
Difference of Two Frames
Absolute DifferenceMethods for Motion Detection
Frame Differencing Background Subtraction
Draw Backs:
involves a lot of computations
Not feasible for DSP implementation
Method 2: Census Transforms
Signature Vector
Extraction
1
124 74 32
124 64 18
157 116 84
1 1 0
1 x 0
1 1 1
1 1 0
1 x 0
1 1 1
If (Center pixel < Neighbor pixel)
Neighbor pixel = 1
Signature Vector11011101
Signature Vector Generation
List Generation2
Signature vector matching 3
128
26 125
243
87
96 76 43 236
125
128
129
235
229
209
228
251
229
221
234
227
221
35 58 98
Image
Signatur vector generation for all pixels
Signature Vectors1 0 1 1 0 1 0 10 0 1 0 1 0 1 1 . . .1 0 1 1 1 0 1 0
List population
1 0 1 1 0 1 0 1
0 0 1 0 1 0 1 1...1 0 1 1 1 0 1 0Generated List
Census Transform:Advantages:
Compare only two values 0 or 1. Similar Illumination Variation for pixel and
neighbouring pixels
Draw Backs:
As we only deal with only 0`s and 1`s, this method is sensitive to noise.
Calculate, store and match process computationally Expensive
Background estimation
Frame differencing
Object Registration
Method 3: Morphology Based Object Tracking
Morphology Based Object Tracking
Background Estimation
•Image Differencing
•Thresholding
Object Registration
•Contours are registered
•Width, height and histogram are recorded for each contour
Feature Vector
•Each object represented by a feature vector (the length, width, area and histogram of the object)
Tracked object
Segmented object
Morphology Based TechniquesAdvantages:
Can Track Multiple objects Objects are registered based on their anatomy
Helpful for Object Merging
Draw Backs:
Object registration complex and slow process For multiple object registration per frame more
complex
Method 4: Lucas-Kanade Technique• Visual motion pattern of objects and surface in a
scene by Optical Flow
Frame 1 Frame 2
Method 5: Mean shift
• An algorithm that iteratively shifts a data point to the average of data points in its neighborhood
Choose a search window
size in the initial location
Compute the MEAN location in
thesearch window
Center the search window
at the mean
Repeat untilconvergence
Intuitive Description
Distribution of identical balls
Region ofinterest
Center ofmass
Mean Shiftvector
Objective : Find the densest region
Intuitive Description
Distribution of identical balls
Region ofinterest
Center ofmass
Mean Shiftvector
Objective : Find the densest region
Intuitive Description
Distribution of identical balls
Region ofinterest
Center ofmass
Mean Shiftvector
Objective : Find the densest region
Intuitive Description
Distribution of identical balls
Region ofinterest
Center ofmass
Mean Shiftvector
Objective : Find the densest region
Intuitive Description
Distribution of identical balls
Region ofinterest
Center ofmass
Mean Shiftvector
Objective : Find the densest region
Intuitive Description
Distribution of identical balls
Region ofinterest
Center ofmass
Mean Shiftvector
Objective : Find the densest region
Intuitive Description
Distribution of identical balls
Region ofinterest
Center ofmass
Objective : Find the densest region
Process
CAMSHIFT --Continously Adaptive Meanshift
Modified to adapt dynamically to the colour probability distributions
More real time
For each frame-> MEAN-SHIFT is applied with several iteration
Store the location of the mean and calculate new window
size for next frame
New development
•Combine with different features. SIFT features, colour feature & texture information
•Camshift algorithm combined with the Kalman filter.
Result
Algorithm
Arithmetic and
Logic operations
Time taken by
Algorithm
AbsoluteDifferencing 4230100 16
Census Transform 2416000 5. 4
MorphologicalTracking 352210 14.2
Kanade Lucas 500825 0.486
Comparison
Absolute Differences
Easy to implement
Allows continuous tracking
Computationally expensive
Slow and low accuracy
Census Transform
Immune to noise and
Illumination changes
Complex if Multiple objects
per frame
Computationally expensive
Feature Based Can track multiple objects well
Large Memory consumption
Slow
Comparison
KLT
High accuracy
Less execution time
Large memory
MeanShift & CAMShift
Ineffective ifthere is
heavyocclusion
Robust to noise and dynamic scene
Computationally less expensive
Conclusion
•KLT algorithm has the best performance with higher accuracy and less computation time
• It requires combination of methods to achieve the appropriate object detection and tracking according to the proposed scenario
References• S. Shah, T. Khattak, M. Farooq, Y. Khawaja, A. Bais, A. Anees, and M. Khan, “Real Time
Object Tracking in a Video Sequence Using a Fixed Point DSP,” Advances in Visual Computing, pp. 879–888.
• K. Huang, L. Wang, T. Tan, and S. Maybank, “A real-time object detecting and tracking system for outdoor night surveillance,” Pattern Recognition, vol. 41, no. 1, pp. 432–444, 2008.
• J. Li, F. Li, and M. Zhang, “A Real-time Detecting and Tracking Method for Moving Objects Based on Color Video,” in 2009 Sixth International Conference on Computer Graphics, Imaging and Visualization. IEEE, 2009, pp. 317–322.
• W. Junqiu and Y. Yagi, “Integrating color and shapetexture features for adaptive real-time object tracking,” IEEE Trans on Image Processing, vol. 17, no. 2, pp. 235–240, 2008.
• Q. Wang and Z. Gao, “Study on a Real-Time Image Object Tracking System,” in Computer Science and Computational Technology, 2008. ISCSCT’08. International Symposium on, vol. 2, 2008.
• Y. Meng, “Agent-based reconfigurable architecture for real-time object tracking,” Journal of Real-Time Image Processing, vol. 4, no. 4, pp. 339–351, 2009.
• [Y. Yao, C. Chen, A. Koschan, and M. Abidi, “Adaptive online camera coordination for multi-camera multi-target surveillance,” Computer Vision and Image Understanding, 2010.