Multimedia Systems and Communication Research Department of Electrical and Computer Engineering Multimedia Systems Lab University of Illinois at Chicago Chicago, Illinois, USA Ashfaq Khokhar
Jan 17, 2018
Multimedia Systems and Communication Research
Department of Electrical and Computer Engineering Multimedia Systems Lab
University of Illinois at ChicagoChicago, Illinois, USA
Ashfaq Khokhar
Major Related Research ThrustsMultimedia Representation, Analysis, Communication,
and Manipulation (Ansari, Schonfled and Khokhar) Content based Indexing and Retrieval Classification of Spatio-Tempral Image and Video Events Motion Tracking Digital Right Management Parallel Implementations on GPU and multicore processors
Heterogeneous Sensor Networks (Ansari, Zefran, and Khokhar) Approximate Spatio-Temporal Query Processing, Information Fusion, and Triggers Motion Control Algorithms Cross Layer Power Efficient Routing Solutions
2
3
Multimedia Representation, Analysis, Communication, and Manipulation Event/object retrieval and classification from
video databases is an extremely challenging problem.
Query and/or stored video data undergo transformation due to camera or object motion (e.g. affine mapping).
Query and/or stored video data contain partial information (e.g. due to video occlusions).
4
Our Worko Scalable content based indexing and retrieval
system for video events, images, and audio clips.
o Classification of motion events, facial expressions, gestures
o Tracking of multiple moving objectso Localized Null Spaceo Kernel Particle Filterso Hierarchical Distributed Indexing Structureso Distributed Hidden Markov Models
Proposed Localized Null Space
Zero elements
Zero elements
N-3
N
Traditional Null Space
5
Structure of Localized Null Space
Illustration of the structure of the traditional Null Space and the proposed Localized Null Space.
Zero elements
Zero elements
3 Non-Zero elements
N-3
Zero elements
Zero elements
K-3Non-Zero elements for W1
K
N-K-3Non-Zero elements for W2
3
Zero elements
Zero elements
N-K
6
Benefits of LNS
)ˆ(,,
ixiuiu rf Can be viewed as consisting of multiple
subspace, therefore can be dynamically split for retrieval of partial queries.
Can be used to merge multiple Null Spaces into an integrated Null Space.
Has the same complexity as the traditional null space.
7
Trajectory and part of the rotated trajectory with identical localized null space representations.
LNS Example
8
Application of LNS in Face Recognition
24 different poses used for each face from the UMIST database.
9
Application of LNS in Face Recognition
Visual illustration of classification accuracy based on Localized Null Space Invariants when the query image is missing vertical or horizontal sections.
Multi-foveation videos
Pixel foveation
DCT foveation
Cyclic Motion Tracking
(Click to play)
Full body, Background clutter Occlusion
Heterogeneous Sensor Networks
Joint work with Northwestern Univ.
12
Proposed SolutionHierarchical Novel Scalable
AbstractionsHybrid StructureRank Order Filters for Value Field
AbstractionMulti-resolution Binary Maps for
Sensor Location Abstraction
Sensor Networks: In-network Hybrid Query Processing
Example Query: Retrieve all the prairie regions in DuPage county that are near river and have between 15% and 45% of salinity decline.
Solution RequirementsLess CommunicationLess Maintenance CostLess StorageLess Query LatencyMore Accurate Results
Existing distributed solutions are incapable of handling value and location queries with equal efficiency!
Our Solution: Novel Hierarchical Abstractions
9 6 5 12 3 18 3 17Sensed Values
Sensor node
9 6 5 12 3 18 3 17Sensed Values
Sensor node
Local cluster head
9 6 5 12 3 18 3 17
9 6 5 12 3 18 3 17Gathering data
Sensed Values
Sensor node
Local cluster head
9 6 5 12 3 18 3 17
3 3 5 6 9 12 17 18
9 6 5 12 3 18 3 17Gathering data
Sorting gathered data
Sensed Values
Sensor node
Local cluster head
9 6 5 12 3 18 3 17
3 5 12 18
3 3 5 6 9 12 17 18
9 6 5 12 3 18 3 17Gathering data
Sorting gathered data
Regular sampling
Sensed Values
Sensor node
Local cluster head
3 5 12 18 4 9 21 25Data sample
Intermediate Level i+1 node
Intermediate Level i nodes
|..|..|..|
3 5 12 18 4 9 21 25
|..|..|..|CompressionData sample
Intermediate Level i+1 node
Intermediate Level i nodes
|..|..|..|
3 5 12 18 4 9 21 25
|..|..|..|
|..|..|..||..|..|..|
CompressionData sample
Gather samples
Intermediate Level i+1 node
Intermediate Level i nodes
|..|..|..|
3 5 12 18 4 9 21 25
|..|..|..|
|..|..|..||..|..|..|
3 5 12 18 4 9 21 25
CompressionData sample
Decompression
Gather samples
Intermediate Level i+1 node
Intermediate Level i nodes
|..|..|..|
3 5 12 18 4 9 21 25
|..|..|..|
|..|..|..||..|..|..|
3 5 12 18 4 9 21 25
3 4 5 9 12 18 21 25
CompressionData sample
Decompression
Gather samples
Merge samples
Intermediate Level i+1 node
Intermediate Level i nodes
|..|..|..|
3 5 12 18 4 9 21 25
|..|..|..|
|..|..|..||..|..|..|
3 5 12 18 4 9 21 25
3 4 5 9 12 18 21 25
3 5 18 25
CompressionData sample
Decompression
Gather samples
Merge samples
Regular samplingIntermediate
Level i+1 node
Intermediate Level i nodes
• Small and fixed size update messages across the hierarchical structure
• Immediate exact response for extreme values (minimum and maximum)
• Low latency, error bounded responses for range queries.
• Small and fixed size update messages across the hierarchical structure
• Fast response for coarse view queries• Low latency, energy efficient responses for
fine detailed queries
What Can be Done for Nokia
Parallel implementation of complete image processing pipeline on GPUs and multi-core platforms
Scalable solutions for recognition/classification, and content based indexing and retrieval of images, audio, and video events.Solutions will work under affine transformations
In network indexing and querying solutions for approximate query processing
16