A Segmentation based Sequential Pattern Matching for Efficient Video Copy Detection

A Segmentation based Sequential Pattern Matching for

Efficient Video Copy Detection

• Introduction • Motivation • Problem Statement • Aim and Objectives • Literature Survey • System architecture• Proposed System• Improvements• References

Content

Introduction Rapid growth of the Internet , Easiness in digital media acquiring and

distributing.

As digital videos can be copied and modified easily, protecting the copyright of the digital media has become matter of concern.

Criteria for selection of copy detection algorithm :- Accuracy, measured in terms of false positive and false negative rates

Computational Requirements, processing time & storage

3

Definition-“Exclusive rights granted by the State for inventions, new and original designs, trademarks, new plant varieties and artistic and literary works”.

Goals For IPR Security:-Detection and retrieval of authentic content.Protection of content from fraudulent

alterations.

Common types of intellectual property rights include copyright ,trademarks, patents, industrial design rights.

4

Intellectual Property Rights

A point in an image which has a well-defined position and can be robustly detected.

Local features vs. Global FeaturesTypes of local feature - Edges, Corner, Blobs.Associated with a significant change of one or more image

properties (e.g.intensity,colors). Used to find corresponding points between images which is

very useful for numerous applications!

What is an Interest Point/feature ?

Courtesy:- Kristen Grauman

Local features: Main Components

Typical Photometric & Geometric Transformations

Motivation• Main Concern-• A considerable number of videos are illegal copies or manipulated versions of

existing media, making copyright management a complicated process. • Call for Change:-• Today’s widespread video copyright infringement calls for the development of

fast and accurate copy-detection algorithms. • As video is the most complex type of digital media, it has so far received the

least attention regarding copyright management.• Protect Data:- • Content-based copy detection (CBCD) ,a promising technique for video

monitoring and copyright protection.

Problem statement

To design a copy-detection algorithm which is sufficiently robust to detect severely deformed copies with high accuracy to localize copy segment.

Aim and Objectives of the project

• The aim of the project is to provide security to multimedia content and provide platform to safeguard copyright of the digital media.

• Objective of video copy detection:-

• To decide whether a query video segment is a copy of a video from the video data set.

• If a system finds a matching video segment, then to retrieve the name of copy video in the video database and the time stamp where the query was copied from.

Architecture for Video Copy Detection• Main Components:-

1. Change-based Threshold for Video Segmentation2. Feature extraction with SIFT from keyframes3. Similarity-based matching between SIFT feature point sets4. Graph-based Video Sequence Matching5. Evaluation Criteria:

• Copy Location Accuracy• Computational Time Cost• Recall & Precision

Architecture for Video Copy Detection

2

1

1

2 3

4

1. Change-based Threshold for Video Segmentation2. Feature extraction with Binary SIFT from keyframes3. Similarity-based matching between SIFT feature point sets4. Graph-based Video Sequence Matching

1.Change-based Threshold for Video Segmentation

Video Time Direction

Reference Video Clip

Th

Segment 1

Segment 2

Method cuts continuous video frames into video segments by eliminating temporal redundancy of the visual information of continuous video frames.• Threshold for detecting abrupt changes of visual information of frames , TH= µ

+α σ , µ and σ are mean and standard deviation of difference values between consecutive frames , α suggested between 5 and 6.

• Threshold for detecting gradual changes of visual information of frames , TL = b * Th , where b is selected from the range 0.1-0.5

2. Feature Extraction with SIFT from Keyframes

Segment 1

Segment 2

SIFT Detection:-1. Find Scale-Space Extrema2. Keypoint Localization & Filtering

– Improve keypoints,throw out bad ones

SIFT Description:-3. Orientation Assignment

– Remove effects of rotation and scale4. Create descriptor

– Using histograms of orientations

Keyframe

3. Similarity-based matching between SIFT feature sets

• For Binary SIFT descriptor extracted compute its nearest neighbor in the dictionary.

• Cluster the set of descriptors (using k-means for example) to k clusters. The cluster centers act as dictionary’s visual words.

• Given a test feature(Binary SIFT),Hierarchical k-NN search is used to find out nearest visual word.

4. Graph-based Video Sequence Matching

Time direction consistency: For Mij and Mlm, if there exists (i – j)*(l-m)> 0, then Mij and Mlm satisfy the time direction consistency.

Time jump degree: For Mij and Mlm ,the time jump degree between them is defined as,

If the following two conditions are satisfied, there exists an edge between two vertexes:1. The two vertexes should satisfy time direction consistency.

2. The time jump degree ∆t < T( T is a preset threshold).

Evaluation Criteria:-• Copy location accuracy:

• This measure aims to assess the accuracy of finding the exact extent of the copy in the reference video.

• What percent of your predictions were correct?

Precision :

What percent of positive predictions were correct?

Recall:

What percent of the positive cases did you catch?

Improvements In Existing System• Segmentation based on Dual Threshold:

• Basic Problems:-• Matching based on SIFT descriptor is computationally expensive for large number of points and its high

dimension. • To reduce the computational complexity, use the dual-threshold method to segment the videos into segments with

homogeneous content and extract keyframes from each segment.

• Binary SIFT Descriptor for Feature Matching:• Basic Problems:-

• Global or Local Descriptor?• SIFT not only has good tolerance to scale changes, illumination variations, and image rotations, but also is robust to

change of viewpoints, and additive noise,logo insertion, shifting or cropping, complicated edit.• Compared with methods based on global descriptor, methods based on local descriptor(SIFT) have a better

detection performance .• Memory cost of binary SIFT is low, making it feasible to store the whole binary SIFT in the index list.

• Graph-based Video Sequence Matching:• Basic Problems:-

• Hard threshold ?• Exhaustive search ?

• To resolve these problems, Graph-based video sequence matching method has the advantages of high accuracy in locating copies, reducing detection time costs, and being able to simultaneously locate more than one copy in two comparing video sequences.

References

• “A Segmentation And Graph-based Video Sequence Matching Method For Video Copy Detection”,hong Liu, Hong Lu, And Xiangyang Xue, IEEE Transactions On Knowledge And Data Engineering, Vol. 25, No. 8, August 2013

• “Visual word expansion and BSIFT verification for large-scale image search”,Wengang Zhou • Houqiang Li • Yijuan Lu •Meng Wang • Qi Tian,Springer,2013

• “Content-based Video Copy Detection Using Discrete Wavelet Transform”, Gitto George Thampi, D. Abraham Chandyc., Proceedings Of IEEE Conference On Information And Communication Technologies,2013

• “Fast And Accurate Content-based Video Copy Detection Using Bag-of-global Visual Features”,yusuke Uchida, Koichi Takagi, Shigeyuki Sakazawa,IEEE,2012

• “Fast And Robust Short Video Clip Search For Copy Detection” Junsongyuan, Ling-yu Duan, Qi Tian, Surendra Ranganath, And Changsheng Xu1,2005

• “Distinctive Image Features From Scale-invariant Keypoints,” Int’l J. Computer Vision, D.G.Lowe,Vol. 60, No. 2, Pp. 91-110, 2004.