BILKENT REAL-TIME DETECTOR FOR REAL-TIME DETECTOR FOR UNUSUAL BEHAVIOR UNUSUAL BEHAVIOR Showcas e
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
BILKENT
REAL-TIME DETECTOR FOR REAL-TIME DETECTOR FOR UNUSUAL BEHAVIORUNUSUAL BEHAVIOR
REAL-TIME DETECTOR FOR REAL-TIME DETECTOR FOR UNUSUAL BEHAVIORUNUSUAL BEHAVIOR
ShowcaseShowcase
HighlightsHighlightsEventsusual non-usual
Motion and shape based
Statistically relevant irrelevant
Alert generation on unusual event
Storing events in database
Eventsusual non-usual
Motion and shape based
Statistically relevant irrelevant
Alert generation on unusual event
Storing events in database
PlatformPlatform
Visualisation: Web browser– SZTAKI will provide a communication
module that will call the module functions provided by the partners.
Software platform: – C++, OpenCv, IPP– Web technologi
Hardware platform: – Pc, laptop (x86 like)
Visualisation: Web browser– SZTAKI will provide a communication
module that will call the module functions provided by the partners.
Software platform: – C++, OpenCv, IPP– Web technologi
Hardware platform: – Pc, laptop (x86 like)
PartnersPartners
ACV
BILKENT
UPC
SZTAKI
ACV
BILKENT
UPC
SZTAKI
Tracking, pedestrian detection
Multimodal human actions, HMM
2D Body actions, motion fields
Unusual event detection, annotation process, statistical analysis, shadow removing
Tracking, pedestrian detection
Multimodal human actions, HMM
2D Body actions, motion fields
Unusual event detection, annotation process, statistical analysis, shadow removing
BILKENTBILKENT
(formerly ARC)
Distribution of workMovingCam.
MovingCam.
StaticCam.StaticCam.
mosaicingmosaicing
ForegroundDetect.
ForegroundDetect.
shilouettesshilouettes
HMM class.HMM class.
Body modelBody modelMotion features
PeriodicityPeriodicity
Pedestriandetection
Pedestriandetection
TrackingTracking
soundsound
classificationclassification
Unusual event
Unusual event
Region alertRegion alert
Sztaki
ACV
BILKENT
UPC
Contribution of ACV
Non-parametric clustering of moving objects in difference imagesOcclusion handling for interacting targetsKernel-based tracking using motion features for multiple targetsVideo data set and evaluation of the motion detection and tracking performance (Benchmark competition)
Details on the algorithmic modules
Human detection by clustering and model-based verification
VIDEO
Kernel-based human tracking using motion information
VIDEO
Occlusion handling
VIDEO
Tracking evaluation (comparison to manual
ground truth)
Evaluation video datasets (street scenarios)
Sequence Street_01.avi: 720x576 pixels, 8628 frames (tracking ground truth available for 1040 frames)
Sequence Street_02.avi: 720x576 pixels, 763 frames (tracking ground truth available for 763 frames)
Contribution of Bilkent Motion and silhoutte based person detector
– detect motion and moving blocks and observe periodicity in bounding boxes of moving blocks in video.
– use silhouttes to classify moving objects in video– combine the results of periodicity and silhoutte based
detectorIn this way,
Determine the number of people in the scene.– HMM classification (fight, fall or simply walk)– Record the sounds and classify the sounds to (car
sounds, walking person, and loud screems)– Combine the results of 3 and 4 to reach a final decision.
BILKENTBILKENT
Human Recognition in Video Human Recognition in Video
Utilizes objects’ silhouettes for different poses
Silhouettes are extracted using contour tracing
Utilizes objects’ silhouettes for different poses
Silhouettes are extracted using contour tracing
Compare silhouette signature functions using wavelet energy signaturesCompare silhouette signature functions using wavelet energy signatures
BILKENTBILKENT
Observation: Walking and falling personObservation: Walking and falling person
Falling Person Detection using Motion Clues (visual)Falling Person Detection using Motion Clues (visual)
T1T1
T2T2
BILKENTBILKENT
Contribution of UPC
Foreground detection and automatic features extraction– motion history descriptors– simple body model
Apply the integrated system to different environments– crowded scenes in automatic stairs
Motion AnalysisMotion Analysis
Motion History and Motion Energy descriptors introduced by Bobick et al. in 2D and Canton et al. in 3D allows robust motion analysis
Motion History and Motion Energy descriptors introduced by Bobick et al. in 2D and Canton et al. in 3D allows robust motion analysis
MEVMEV
MHVMHV
Model Based AnalysisModel Based Analysis
Analyzing input data by means of a Human Body Model, allows retrieving information about limbs positions
Analyzing input data by means of a Human Body Model, allows retrieving information about limbs positions
Scene capture
User segmentation
CoG computation
Creation of the geodesic distance map
Contour tracking
Creation of the distance/silhouette border position function
H-maxima operation on the function
Local maxima extraction
Morphological skeleton computation and crucial point labeling
Scene capture
User segmentation
CoG computation
Creation of the geodesic distance map
Contour tracking
Creation of the distance/silhouette border position function
H-maxima operation on the function
Local maxima extraction
Morphological skeleton computation and crucial point labeling
Pixel position
Geodesi
c D
ista
nce
Silhouette analysis for detection of body extremitiesSilhouette analysis for detection of body extremities
Contribution of SZTAKI
Foreground detectionView region surveillanceAlert event generationEvent History– Search & display
Contribution of SZTAKI
Foreground detection in moving camera
Contribution of SZTAKI
Mosaicing
Contribution of SZTAKI
Usual – non usual motionUsual – non usual motion
Pixel-wise motion estimationPixel-wise motion estimationblack: right, white: leftblack: right, white: left
Motion statisticsMotion statistics
InputInputActual motion Actual motion masked with usual masked with usual motionmotion
Contribution of SZTAKISG based unusuality detector on – motion fields– motion tracks
Software Environment– Interface module to user dll/lib/module
• Separates and bridge modules– Server
• Serves image/video streams• Transcodes images• Forward requests to modules
– DB server• Metadata store & search
– Webserver• Generate html pages with links to Server (later)
– Client dynamic web • Javascript/flash based graphics display• Mozilla native mjpeg stream + SVG
Web PageWeb Page
DBmetadata
DBmetadata
tcp/ipSERVERSERVER
WebserverWeb
server
MatlabMatlab
C++C++
DLL/LIBDLL/LIBComm.
Interfacejson
tcp/ip
tcp/ipmjpg
json
Html
User modules
Contribution of SZTAKI - architecture
Data SourceData Source
Controller
Comm.Interface
Comm.Interface
Comm.InterfaceModule Register
Streams
Internet