Reconstructing and Watermarking Stereo Vision Systems Osama Hosam Osama Hosam Supervisor: Professor Sun Xingming Supervisor: Professor Sun Xingming
Reconstructing and Watermarking Stereo Vision Systems-PhD Presentation
Jun 19, 2015
We have solved the correspondence problem by applying the matching process in two levels, the first level is Feature based matching, in which we have extracted the features of both images by creating multi-resolution images and applying histogram segmentation. The resulting features are region features; a comparison is done between the regions in the first image with the regions of the second image to get the disparity map.
The second level is Area-based matching in which we applied the Wavelet transform to get an expected window size as a search area for each pixel. We have joined the two levels to obtain more accurate pixel by pixel correspondence. We also obtained an adaptive search range and window size for each pixel to reduce the mismatches. Our procedure introduced high accuracy results and denser depth information.
The depth information is used to get the final 3D model – using only pair of images will create 2.5D model, using more than pair of images will create 3D model, we will refer to 3D model as a general output of stereo reconstruction– After reconstructing the model, in some applications it is needed to be published online. For example suppose the reconstructed model is a model for Sphinx – Famous statue in Egypt – The reconstruction for the model can be done in many days or months; then the model will be published online to let Internet users around the world watch the model. Therefore, techniques should be used to protect the copyright for that model. We have applied new fragile watermarking technique to secure the 3D reconstructed model and protect its copyright.
The second level is Area-based matching in which we applied the Wavelet transform to get an expected window size as a search area for each pixel. We have joined the two levels to obtain more accurate pixel by pixel correspondence. We also obtained an adaptive search range and window size for each pixel to reduce the mismatches. Our procedure introduced high accuracy results and denser depth information.
The depth information is used to get the final 3D model – using only pair of images will create 2.5D model, using more than pair of images will create 3D model, we will refer to 3D model as a general output of stereo reconstruction– After reconstructing the model, in some applications it is needed to be published online. For example suppose the reconstructed model is a model for Sphinx – Famous statue in Egypt – The reconstruction for the model can be done in many days or months; then the model will be published online to let Internet users around the world watch the model. Therefore, techniques should be used to protect the copyright for that model. We have applied new fragile watermarking technique to secure the 3D reconstructed model and protect its copyright.
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Reconstructing and Watermarking Stereo Vision Systems
Osama HosamOsama Hosam
Supervisor: Professor Sun XingmingSupervisor: Professor Sun Xingming
OutlineOutline
Stereo imagesStereo images
• Stereo Image definition.Stereo Image definition.
• The problems of the reconstruction process. The problems of the reconstruction process.
• The proposed Stereo Image Watermarking technique.The proposed Stereo Image Watermarking technique.
•Disparity estimation.Disparity estimation.
•Watermark Embedding and ExtractionWatermark Embedding and Extraction
• Experimental Results Experimental Results
•The proposed 3D Watermarking technique The proposed 3D Watermarking technique
•The problems of 3D WatermarkingThe problems of 3D Watermarking
• 3D Watermarking approach with high visual quality. 3D Watermarking approach with high visual quality.
• Experimental Results. Experimental Results.
• Conclusion Conclusion
OutlineOutline
Stereo imagesStereo images
• Stereo Image definition.Stereo Image definition.
• The problems of the reconstruction process. The problems of the reconstruction process.
• The proposed Stereo Image Watermarking technique.The proposed Stereo Image Watermarking technique.
•Disparity estimation.Disparity estimation.
•Watermark Embedding and ExtractionWatermark Embedding and Extraction
• Experimental Results Experimental Results
•The proposed 3D Watermarking technique The proposed 3D Watermarking technique
•The problems of 3D WatermarkingThe problems of 3D Watermarking
• 3D Watermarking approach with high visual quality. 3D Watermarking approach with high visual quality.
• Experimental Results. Experimental Results.
• Conclusion Conclusion
Stereo image definition
Stereo images work as the human eyes, since the pair of eyes take pair of images of the same view from different angles.
The relation between Disparity and Depth
Disparity is the shift of the objects in both images, or the difference between the object positions in both images.
Disparity of Sl
Centroid of Sr
Centroid of Sl
Sl SlSr
Left Image Right Image
????
Depth is inversely proportional to the disparity, so the objects near the camera will be shifted more than the objects which is far from the camera
d : Disparityf : distance from the image plan to the camera T : The baseline of the two cameras Z : the third dimension or Depth.
The problems of the reconstruction process
Segmentation Matching
Disparity map
3D scene
- Finding the best segmentation method - The correspondence problem - The occluded regions - The density of the 3D scene
The problems of the reconstruction process
Segmentation
?Match
Segmentation
For every feature in For every feature in the left image we the left image we search for the search for the correspondent feature correspondent feature in the entire right in the entire right image featuresimage features
A distinctiveA distinctive descriptor for the descriptor for the features are needed features are needed to reduce the to reduce the mismatches and mismatches and obtain high accuracy.obtain high accuracy.
OutlineOutline
Stereo imagesStereo images
• Stereo Image definition.Stereo Image definition.
• The problems of the reconstruction process. The problems of the reconstruction process.
• The proposed Stereo Image Watermarking technique.The proposed Stereo Image Watermarking technique.
•Disparity estimation.Disparity estimation.
•Watermark Embedding and ExtractionWatermark Embedding and Extraction
• Experimental Results Experimental Results
•The proposed 3D Watermarking technique The proposed 3D Watermarking technique
•The problems of 3D WatermarkingThe problems of 3D Watermarking
• 3D Watermarking approach with high visual quality. 3D Watermarking approach with high visual quality.
• Experimental Results. Experimental Results.
• Conclusion Conclusion
The Proposed Stereo Image WatermarkingApproach
1. Stereo Images are reconstructed (make disparity estimation) to obtain disparity map.
2. Disparity map is used as a watermark and is embedded into the left image of the stereo image pair.
3. The host image is transferred through insecure channel to the receiver.
Internet(Insecure)
DisparityEstimation
WatermarkEmbedding
Receiver
StereoImages
Stereo server(Disparity)Watermark
Key
Disparity Estimation
Disparity Estimation is done in two levels
First: Feature based matching, the first level is to extract the feature of the pair of images and then compare these features together. The comparison is done according to specific properties for each feature
Second: Wavelet-based matching, in this level we are going to make dense matching by comparing the pair of images window by window leading to more accurate matching.
Feature-basedmatching
SSDCorrespondence
Wavelet Decomposition Wavelet Decomposition
Left Image Right Image
(Disparity map)Watermark
Disparity Estimation
Disparity Estimation (Feature Based Matching)
Histogram segmentation: can be viewed as an operation that involves tests against a function T of the form
)3()],(),,(,,[ yxfyxpyxTT
Left Image Right Image
)4(
255),(3
3),(2
2),(1
1),(0
4
3
2
1
),(
yxfTif
TyxfTif
TyxfTif
Tyxfif
g
g
g
g
yxPixl
Disparity Estimation (Feature Based Matching)
For each group a series of (low, medium and high) resolution images will be created, It is created by reduction in size by a factor of two using Gaussian convolution filter h(x).
)5(2)(2222 xAexh
A binary map for each of multi-resolution images will indicate, for each pixel, whether it belongs to the group or not, as shown by equation (6) .
)6(.),(0
),(1),(
Tyxfif
Tyxfifyxg
Thus the pixels labeled 1 correspond to the region segment, whereas pixels labeled 0 correspond to the background
Disparity Estimation (Wavelet Based Matching)
For each pixel in the stereo images, - Execute a convolution by an adaptive window size.- The correspondence search range will be limited by using the disparity map gained from feature based matching. - The correspondence will be estimated by using SSD,
),( ),(
22
),(
),(),(
)],(),([
yxAIyxAI
ydxAIyxAI
Crl
rl
The disparity estimate for a pixel x,y is the one which minimizes SSD error:
d(x,y) = arg min C(x,y,d)
LSBEmbedding
Left Image Degraded Left Image
Watermark
Watermarked degradedImage
Watermarked degradedImage
LSB WatermarkExtraction
Watermark
Left Image
ArnoldTransform
Anti-ArnoldTransform
Key
Key
Watermark Embedding and Extraction
Experimental Results(Image acquisition techniques)
The camera
The object
(a) (b)
the image acquisition techniques, (a) The Object Registration device. Source: Borg Al Arab City for Scientific Research and Technology Applications, Informatics Research Institute, Alexandria, Egypt (b) Aircraft Stereo images
Experimental Results(Disparity Estimation)
Due to the two levels approach we have reduced the correspondence mismatches
AlgorithmAlgorithm Number of matching Number of matching levelslevels
MismatchesMismatches
Quantization and Modulation Quantization and Modulation 1 level1 level 23 %23 %
DCT based DCT based 1 level1 level 21 %21 %
DWT based DWT based 1 level 1 level 29 %29 %
Fractional Fourier transform (FrFT) Fractional Fourier transform (FrFT) 1 level 1 level 19 %19 %
Genetic Algorithm (GA) Genetic Algorithm (GA) 1 level1 level 16 %16 %
Our Algorithm Our Algorithm 2 levels 2 levels 4 %4 %
Experimental Results(Random LSB Stereo Image Watermarking)
The PSNR value is used to measure the quality of watermarking in our proposed stereo image watermarking. The PSNR value between host image I and the host image I’ is calculated as
RMSEPSNR
255log20 10
Where
m
i
n
j
jiIjiInm
RMSE1 1
22 ),('),(*
1
Lower values of PSNR mean less invisibility of the watermark, while higher PSNR represents better invisibility of the watermark in the host image.
Experimental Results(Disparity Estimation)
The proposed disparity estimation algorithm results applied on Tsukuba image, (a) The original image (b) the disparity map obtained in [7] (c) disparity map obtained in [9] (d) The disparity map obtained in [6] (e) The disparity map obtained in [10] (f) The disparity map obtained by our proposed approach
(a) (b) (c)
(d) (e) (f)
OutlineOutline
Stereo imagesStereo images
• Stereo Image definition.Stereo Image definition.
• The problems of the reconstruction process. The problems of the reconstruction process.
• The proposed Stereo Image Watermarking technique.The proposed Stereo Image Watermarking technique.
•Disparity estimation.Disparity estimation.
•Watermark Embedding and ExtractionWatermark Embedding and Extraction
• Experimental Results Experimental Results
•The proposed 3D Watermarking technique The proposed 3D Watermarking technique
•The problems of 3D WatermarkingThe problems of 3D Watermarking
• 3D Watermarking approach with high visual quality. 3D Watermarking approach with high visual quality.
• Experimental Results. Experimental Results.
• Conclusion Conclusion
3D Watermarking ( Main idea )
Localized embeddingUsing ratios of 2-D and 3-D geometrical measures
3D Triangular mesh model
3D Watermarking ( Attacks 1/2 )
Rotation, translation, and uniform scaling This kind of operations disables the watermark detector by changing the orientation, location and/or scale of the model to make the detector not able to locate the watermark.
Polygon simplification This kind of operations changes the coordinates of vertex and the topology of the model by reducing the number of faces in a dense mesh while minimally perturbing the shape.
Mesh smoothing This operation attacks the watermark by deleting some high-frequency components of the model and smoothing the model surface.
3D Watermarking ( Attacks 2/2 )
Re-meshing This kind of operations attacks the watermark through using another sampling mesh to representing the same model.
Reordering of points This operation scrambles the vertices in the 3D model to destroy the watermark information embedded in the store sequence of the vertices.
Cropping This operation removes parts of the model to destroy the watermark
Problems of 3D watermarking
The location of a former processed vertex will be changed by perturbing the latter processed vertex
1 - The Causality Problem
V1
V2V3
V1
V2 V3
V1
V2
V3
We have solved this problem by proposing The Contagious Diffusion Technique for traversing the model (details later)
Problems of 3D watermarking
In embedding stage, it required to perturb all the pixels in the model to every pixel value index equal to the pixel’s location index. In practice, some vertices should be perturbed more and more until it satisfy the requirements
2 - The Convergence Problem
We have solved this problem by proposing our NNM technique (details later)
The Proposed Algorithm (1/5)
Our proposed algorithm steps
Extract VBT (Vertex Body Table)
Extract PNT from VBT
Navigate by using Contagious Diffusion Technique.
Embed the watermark by changing topology
The watermarked 3d model (3ds file)
The original 3d model ( 3ds file)
1. Extract VBT from the saved model file.
2. Extract PNT from the saved model file.
3. Navigate through the model by using Contagious Diffusion Technique.
4. Embed the watermark by using NNM (Nearest Neighbor Move) technique.
The Proposed Algorithm (2/5)
1 -Extracting VBT from 3DS file.
The 3ds file is saved on the hard disk as chunks; each chunk has a unique address. Our algorithm extracts the chunks that represent the model vertices. The vertices are then saved into a database table
The Proposed Algorithm (3/5)
2 -Extracting PNT from VBT
For each polygon we search for a polygon sharing the same edge with the current polygon. We notice this way is a time consuming since the search will be done by comparing each edge (two vertices) with each other two vertices in the table. When a matching edge is found, this means the edge (two vertices) belongs to the same polygon and the tow polygons are neighbors
The Proposed Algorithm (4/5)
3 -Contagious Diffusion Technique
In The Contagious Diffusion Technique the polygon has three statuses,Suspected, means it has no embedded data, and its entire neighborhood has no embedded data. Infected, means some of its neighbor are infected others are not.Immune means all the polygon neighbors are entirely infected and have embedded data.
The Proposed Algorithm (5/5)
4 -The embedding Procedure (Nearest Neighbor Move NNM)
1 0 1 0 1 0 1 1 0 1 0 1 0 1base edge
Vc
Vc
Vb
Va
Va
Vb
(a) (b)
)2()( 121 xxL
hxx
)3()( 121 yyL
hyy
)4()( 121 zzL
hzz
where L is the Euclidean distance between vertices Va and Vb, which can be calculated with
)5()()()( 212
212
212 zzyyxxL
Experimental Results
Tools
1 – 3D Max Software to build up our models2 – C++ as a programming language
3 – OpenGL Library to Embed and Extract the secret data into the 3DS files
ModelModel
NameName SpaceshipSpaceship CubesCubes HeartHeart Chess pawnChess pawn
VerticesVertices649649 17281728 861861 10361036
PolygonsPolygons500500 25922592 17171717 20682068
Experimental Results
Model Spaceship Cubes Helmet Heart Chess Pawn
Vertices 649 1728 859 861 1036
Polygons 500 2592 452 1717 2068
Embedded bits 321 623 321 604 913
Distortion(Cheng et. al. [58])
1.213 E-06 4.021 E-06 1.241 E-06 1.414 E-06 3.452 E-06
Distortion (Kwangteak et. al. [59])
1.112 E-07 3.951 E-07 6.213 E-07 8.852 E-07 7.256 E-07
Distortion(Our Method)
0.962 E-07 1.001 E-07 1.289 E-07 2.011 E-07 0.911 E-07
Comparison between previous techniques and NNM technique
Experimental Results
The effect of changing on the visual quality of the watermarked model. The experiment is done on three models, Helmet, Chesspawn and Spaceship.
Experimental Results
(a),(b) and (c) are the cover models for Chesspawn, Spaceship and Helmet respectively. (d),(e) and (f) are the watermarked 3D models.
Experimental Results
Screenshot of our software
OutlineOutline
Stereo imagesStereo images
• Stereo Image definition.Stereo Image definition.
• The problems of the reconstruction process. The problems of the reconstruction process.
• The proposed Stereo Image Watermarking technique.The proposed Stereo Image Watermarking technique.
•Disparity estimation.Disparity estimation.
•Watermark Embedding and ExtractionWatermark Embedding and Extraction
• Experimental Results Experimental Results
•The proposed 3D Watermarking technique The proposed 3D Watermarking technique
•The problems of 3D WatermarkingThe problems of 3D Watermarking
• 3D Watermarking approach with high visual quality. 3D Watermarking approach with high visual quality.
• Experimental Results. Experimental Results.
• Conclusion Conclusion
Conclusions & Future Work
The process of stereo image reconstruction is enhanced by applying two levels of disparity estimation
A highly secure stereo image watermarking system has been implemented. The Random LSB is applied together with a key to increase the security of the watermarked images.
High visual quality of 3D models is obtained after Watermarking the 3D model. The two common problems of 3D watermarking has been solved, namely the causality problem and the convergence problem.
In Future Work, We are planning to perform the watermarking for 3D models in the frequency domain and evaluate the results.
Questions Questions
??
Related Documents
