Applying Security and Privacy Enhancements to a Visual Localization and Tracking Systemcs.uccs.edu/~gsc/pub/master/falrashe/doc/report.pdf · 2015-04-14 · ii Abstract -- Sending
Post on 31-May-2020
1 Views
Preview:
Transcript
Applying Security and Privacy Enhancements to a Visual
Localization and Tracking System
By
Fahad Alrasheedi
B.S in ITC Arab Open University, 2010
A project
submitted to the Graduate Faculty of the University of Colorado at Colorado Springs
in partial fulfillment of the requirements for the degree of
Master of Engineering in Information Assurance
Department of Computer Science
2015
This project for the Master of Engineering degree by Fahad Alrasheedi has been approved for the Department of Computer Science by
_______________________________________________________ Dr. Jonathan Ventura, Advisor
_______________________________________________________ Dr. Dr. Jia Rao, Committee member
_______________________________________________________ Al Brouillette, Committee member
_______________________________ Date
ii
Abstract -- Sending data over the Internet without using security measures to keep the data
protected from modification and even from eavesdropping is a dangerous practice and not
recommended at all. The situation is even worse if the purpose of the communication is to send
sensitive data that demands a high level of security such as bank transactions, military operations,
localization and tracking systems, or even private communications with family members and friends.
Localization and tracking systems that use the Internet as a communication medium between the
person ( client ) and a localizer (server ) should take into an account the security part of the process
otherwise it will be easy for an attacker to read the message content including the query from the
client and the response from the server . The advancement of technology and the rapid growth of
solutions including commercial or open source endeavors, are intended to tackle such security-
related problems and network vulnerabilities to help us keep our communications over the Internet
more secure and reliable. In this paper, we will review the common network attacks which may
happen to the system proposed by Ventura and T. Höllerer [1], which uses a normal TCP
connection, and describe methods an attacker could use to exploit such a kind of unencrypted
connection . Then , we will implement and evaluate different levels of security to enhance the
system and make it more secure.
iii
ACKNOWLEDGMENTS
I would like to express my special appreciation and thanks to my advisor Professor Dr. Ventura, you
have been a tremendous mentor for me. I would like to thank you for encouraging my research and
for allowing me to grow as a research scientist.
I would also like to thank my committee members, professor Dr.Rao, and Mr.Brouillette for serving
as my committee members even at hardship. I also want to thank you for letting my defense be an
enjoyable moment, and for your brilliant comments and suggestions, thanks to you.
A special thanks to my family. Words cannot express how grateful I am to my mother , father for all
of the sacrifices that you’ve made on my behalf. Your prayer for me was what sustained me thus far.
At the end I would like express appreciation to my beloved wife Hanadi who spent sleepless nights
with and was always my support in the moments when there was no one to answer my queries.
iv
LIST OF TABLES
Table 1 : AES parameters . source [9] ……………………………………………………………………………………………17
Table 2: Comparison average total time between DB1 and DB2………………………………………………………………..22
Table 3 : Comparison between CP and GPU when dealing with DB1………………………………………………………….23
Table 4: Comparison between CP and GPU when dealing with DB2…………………………………………………………..24
v
LIST OF FIGURES
Figure 1 : Overview …………………………………………………………………………………………………………………1
Figure 2 : Camera Matrix , source [10] ……………………………………………………………………………………………5
Figure 3 Flowshart to build 3D from VRLT………………………………………………………………………………………8
Figure 4 : Feature Extraction………………………………………………………………………………………………………8
Figure 5 : Matching features linearly………………………………………………………………………………………………9
Figure 6 : Making tracks……………………………………………………………………………………………………………9
Figure 7 Bootstrapping two images…………………………………………………………………………………………………9
Figure 8 : Adding the remaining images ……………………………………………………………………………………….….9
Figure 9 : Reconstruction the 3 D…………………………………………………………………………………………………..9
Figure 10 : Certifacet Signing………………………………………………………………………………………………………10
Figure 11 : Image bytes in Hexadecimal……………………………………………………………………………………………12
Figure 12 : Wireshark catpture for normal TCP…………………………………………………………………………………13
Figure 13 : Wireshark catpture for TCP/SSL…………………………………………………………………………………….14
Figure 14 : a square matrix of plaintext……………………………………………………………………………………………16
Figure 15 : Overview of AES algorithm source [9] ………………………………………………………………………………20
vi
TABLE OF CONTENTS
CHAPTER PAGE ABSTRACT ................................................................................................................................... ii ACKNOWLEDGMENTS ............................................................................................................ iii LIST OF TABLES ........................................................................................................................ iv LIST OF FIGURES .........................................................................................................................v CHAPTERS CHAPTER 1 – Introduction .............................................................................................................1
1.1 Overview ...............................................................................................................1 1.2 Why to Secure VRLT and Research question ......................................................2 1.3 Common Possible TCP Attacks ............................................................................3
CHAPTER 2 – Related Work Related Work and Literature Review 4 ..........................................4 CHAPTER 3 – Creating 3D Model Reconstruction ........................................................................5 3.1 Camera Calibration ...............................................................................................5 3.3 Taking Photos for the Environment ......................................................................7 3.3 Building 3D Model Reconstruction ......................................................................7 CHAPTER 4 – Applying Different Security Levels ......................................................................10 4.1 Security First Level .............................................................................................10 4.1.1 User Authentication ..................................................................................10 4.1.2 Using X.509 Certificates ...........................................................................10 4.2 Security Second Level ........................................................................................11 4.2.1 Encryption of Connection Stream .............................................................11 4.3 Security Third Level ...........................................................................................14 4.3.1 Database Encryption .................................................................................14 4.3.2 Using AES Algorithm ...............................................................................15 CHAPTER 5 – Evaluation and Results .......................................................................................21 5.1 Mutual User Authentication ................................................................................21 5.2 Securing TCP Connection with SSL ...................................................................21 5.3 AES Encryption of Database ..............................................................................22 CHAPTER 6 – Discussion and Future Work ..............................................................................24 REFERENCES .............................................................................................................................27 APPENDICES Appendix A ...................................................................................................................................28 Appendix B ...................................................................................................................................30 Appendix C ...................................................................................................................................32 Appendix D ...................................................................................................................................33 Appendix E ...................................................................................................................................34
vii
Chapter 1 : Introduction
1- Problem Overview
Figure 15 : Overview
The system proposed by Ventura and T. Höllerer [1] is used to localize the client in a real time
when he takes a picture and sends it to a remote server which in turn extracts the features from the
sent picture and matches these features with a 3D reconstruction of a previously stored pictures of
the same environment. Next, the server estimates the best pose and sends the result back to the
client.
The main disadvantage of this system is that it uses an unsecure TCP connection between the
client and the sever. In other words, there is no secure connection area applied above the TCP layer
like SSL or TSL. If this system is used for police , military purposes or any sensitive work to
localize and track team members, it would be easy to exploit the system by the attacker to eavesdrop
2
on the communication between the two parties and possibly modify it for certain unauthorized
reasons .
Such systems should function in a more secure environment even if the system performance is
downgraded by the security and privacy enhancements integrated into the system. Moreover,
security and privacy should be addressed and taken into an account when designing localization and
tracking systems for military work or any environment which demands secure and private operation.
The system could be enhanced with different levels of security to prevent the possible threats that
may happen when using an unsecured TCP connection between the client and the remote server. The
connection stream between the two parties can be encrypted so the attacker cannot detect what is
being sent, even if he can eavesdrop on the connection. The system also could be modified in such a
way it could authenticate the two parties mutually before even starting to exchange data. There are
different ways to apply the user authentication, such as using a username and password to
authenticate the client by the server. Also, certificates for both client and server could be used to
mutually authenticate one another. The latter is our option to achieve user authentication in our
modified system.
2- Why to Secure VRLT , and Project Research Question :
• Localization and tracking system itself should be one of security products , for that reason it
should work in a very secure environment .
• One of the beigest reason as a motivation to enhance VRLT with more security levels is
terrorism . If such system is adopted by military or police forces in its current status , it will
be an easier for terrorists to capture the request and the response being transmitted , and
possibly alter the data .
• In general , it is good practice to secure the connection if a public channel is used as medium
of communication .
Research question : What are the overheads , in term of time consumption , of security
enhancements to a Visual Localization and Tracking systems .
3
3- Common Possible TCP Attacks : There are two types of attack ( active and passive) :
3.1 Active Attacks :
3.1.1 TCP Sequence Number Prediction Attack: When Client C and Server S creates a new
connections, an initial sequence number (ISN) generator is employed which selects a new 32 bit
ISN. This sequence number will be use for counting and tracking TCP segments being transmitted
during the session . These ISN’s will be agreed on upon the successful 3 Handshaking involved in
TCP connection creation Pan,Xiao [12] . It is easy to predict the TCP sequence number by the
attacker due to the natural of TCP/IP protocol , and with a help of sniffer software Pan,Xiao [12] ,
this attack could lead to connection hijacking attack .
3.1.2 TCP Session Hijacking Attack :It is possible for this attack to happen to TCP connection ,
attacker A could sniff the TCP connection between the client C and the server S , then A can inject
the connection with a segment to the server S spoofing the source address as the client C .
Eventually , the server S will accept the segment and update acknowledgment numbers . Also , the
attacker could apply Denial of service attack to the client to increase the chance of injecting the
connection with his segment . TCP session hijacking could also happen without sniffing the
connection between the client C and the server S , the attacker simply will guess the sequence
number in the current TCP segment and the port number , such hijacking is called a blind hijacking
Pan,Xiao [12] .
3.1.3 IP Spoofing : is the act of concealing the source IP address by forging the packet header with
different IP address , so it appears to the recipient of the packet that the sender is the latter address
not from the true location so the reply coming from the recipient will be routed to that IP address
instead of going to the real sender [13].
3.2 Passive Attacks :
3.2.1 Eavesdropping : is one of the basic attacks in order to track an individual user of his online
behavior. This involves the tracker capturing packets sent by the user in a particular network using a
packet sniffer software like Wireshark. Modio, TCPDump, Snort, Packet Filter (PF) etc. The tracker
can analyze the sender’s IP address and the destination IP address, the packet payload, which might
contain sensitive information like names, addresses and even phone numbers (Risso, F., &
Degioanni, L. 2001).
4
Chapter 2 : Related Work and Literature Review
Localization and tracking systems can be: indoors or outdoors. There have been a number of
researches conducted in this area. H. Kawaji , K. Hatada , T. Yamasaki , K. Aizawa [6] is an
example of indoor localization and Ventura and T. Höllerer [1] is an example of outdoor
localization.
Moreover, the tracking systems can be done through GPS signal like the system proposed by S.
Panzieri, F. Pascucci, and G. Ulivi, [8], Wi-Fi access points or using computer vision techniques for
example [ 2 , 3 , 4 , 5 ]. In our literature review we will consider existing computer vision-based
tracking systems to see if any previous papers addressed security and privacy in the system design.
To narrow down the research domain, we will not focus on all previosu works that apply to the
computer vision techniques ,since not all of them use a client-server concept to do tracking and
localization task. For example, the system of C. Arth, D. Wagner, M. Klopschitz, A. Irschara, and
D. Schmalstieg [7] performs image capture and localization on the same device. Instead, we will
focus on the systems which have two devices where one is the client and the other is the server.
To the best of our knowledge, none of these existing systems of interest apply or implement any
security and privacy techniques. For this reason, we intend to improve the system of Ventura and T.
Höllerer [1] in order to afford some security and privacy layers that will be proposed and
implemented. We will implement several measures: user authentication, encryption and decryption
during data transmission, database encryption, and extracting the features at the client side instead of
sending the whole picture to the server and letting the server extract the features .
When designing such systems with consideration for security, an important question is how secure
the designers want their systems to be . In other words, some of them may need only to authenticate
the users , others may need only to encrypt the connection , or they may only need to keep their
database encrypted . However, we will try to cover all such security considerations and measure how
they are affecting the performance of the system proposed by Ventura and T. Höllerer [1] .
5
Chapter 3 : Creating 3D Model Reconstruction
3.1 Camera Calibration : iPhone 6 plus’s camera was used in this project to take photos for the
outdoor environment . Most of today’s cameras come with some significant distortions , fortunately
such distortions are constant and could be corrected and remapped by a calibration [10] .
After the iPhone camera is calibrated , the following camera matrix should be obtained as an output:
Figure 16 : Camera Matrix , source [10]
As for fx and fy , they are the camera focal lengths . However , commonly axe x and axe y use the
same focal length with a certain ratio :
fy = fx * ratio ……………………….. ( 1 )
Ratio is usually equal to one so fy will be equal to fx [10] . That is , the focal length for both axes in
the iPhone 6 plus camera will be a single focal length . As for cx and cy , they are the optical centers ,
Xcenter and Ycenter , and they are expressed in pixel coordinates .
The calibration program basically calculates the camera matrix through geometric equations [10] .
The calibration accepts three different types of objects to do the calculations of the camera matrix :
1- Classical black-white chessboard
2- Symmetrical circle pattern
3- Asymmetrical circle pattern
The first type of the three above objects was chosen to calibrate the iPone 6 plus camera, several
photos of the chessboard have been taken , at least 30 photos , from different distance , angles . The
calibration program will read the configuration files where the number of inner corners ( cross of
6
rows and columns ) is define , the original chessboard that is provided by OpenCV has a width
(rows) six and the height (columns) nine .
Moreover , the configuration file has the option to choose which kind of object will be used to
calibrate the camera , in our case CHESSBOARD . Also , it contains where the photos are stored so
the calibration program can read them . Additionally , the number of how many photos to be
calibrated should be specified , at least 13 photos out of 30 photos should be calibrated to come up
with a good result of camera matrix .
The calibration program will try to detects the corners in the photo of CHESSBOARD based on the
configuration file . However , it is common the that the calibration could fail to detect such corners
in the photo , for this reason we should supply an enough number of photos to calibrate 13 photos (
in our case , we supplied 30 photos ) .
After running the calibration program with the supplied photos and the right parameters in the
configuration file , XML file will be created and it will contain the Camera Matrix as it is shown in
the following part of calibration program output ( the output_camera_calibration.xml ) :
<?xml version="1.0"?> <opencv_storage> <calibration_Time>"Mon Mar 16 06:05:54 2015"</calibration_Time> <nrOfFrames>16</nrOfFrames> <image_Width>3264</image_Width> <image_Height>2448</image_Height> <board_Width>6</board_Width> <board_Height>8</board_Height> <square_Size>30.</square_Size> <FixAspectRatio>1.</FixAspectRatio> <!-- flags: +fix_aspectRatio +fix_principal_point +zero_tangent_dist --> <flagValue>14</flagValue> <Camera_Matrix type_id="opencv-matrix"> <rows>3</rows> <cols>3</cols> <dt>d</dt> <data> 3.3407986643686490e+03 0. 1.6315000000000000e+03 0. 3.3407986643686490e+03 1.2235000000000000e+03 0. 0. 1.</data></Camera_Matrix>
7
3.2 Taking Photos for the Environment : Building database of 3D model is done as offline stage .
The 3D model database could be created using some tools that VRLT provides . Two different sizes
of database have been created in purpose of examining the effect of database growth on the
localization service that VRLT provides .
Regent Cir , the area between Kraemer Family Library and Lot 224 , in University of Colorado
at Colorado Springs was chosen to be an outdoor environment . The following steps show how
photos have been taken form the outdoor environment :
1- The start point was the west north corner of Lot 224 .
2- Walking on Regent Cir toward Lot 222 .
3- After each four steps of walk , three photos were taken ( appendix G gives an example for
these three photos for one stop :
• The first photo (Center ) was taken when looking straight .
• The second photo ( Right ) : step to right from the center , then the camera was
moved 45 degree to the right to cover more area in the right side , however there is a
30-40 percent common between the second photo and the first photo .
• The third photo ( Left ) : step left from the center . Then the camera was moved 45
degree to the left to cover more area in the left side , however there is a 30-40 percent
common between the third photo and the first photo.
4- Number of stops in the first database is 38 stops ( 114 photos ) , the second database adds
another 38 stops which increases the number of photos with another 114 photos ( double size of the
first database ) .
3.3 Building 3D Model Reconstruction : Figure (3) shows the flowchart of the steps that have been
followed to build the 3D model from VRLT :
8
Figure 17 Flowshart to build 3D from VRLT
• Process 1 : Extracting the features from each photo : The system uses SIFT to extract the
descriptors and detectors in each photo for feature matching . The tool used in this step is
ExtractSIFT . Here is an example for using this tool :
Figure 18 : Feature Extraction
• Process 2 : Matching the features : In this step , the features extracted in one photo is
matched to the same feature in another photo . The feature matching is done linearly , the
tool used in this step is LinearMatch . Here is an example for using this tool :
9
Figure 19 : Matching features linearly
• Process 3 : The matching features are then organized into tracks . The tool used for this step
is MakeTrack . Here is an example for using this tool :
Figure 20 : Making tracks
• Process 4 : Bootstrapping two photos together using the tool Bootstrap to start reconstructing
our database . Here is an example for using this tool :
Figure 21 Bootstrapping two images
• Process 5 : Adding the remaining photos to the bootstrapped photos . The tool used for this
step is AddImages . Here is an example for using this tool :
Figure 22 : Adding the remaining images
• Process 5 : the normal vectors are estimated to come up with our 3D model reconstruction .
The tool used for this step is EstimateNormals . Here is an example for using this tool :
Figure 23 : Reconstruction the 3 D .
10
Chapter 4 : Applying Different Security Levels :
4.1 Security First Level :
4.1.1 User Authentication :
VRLT uses no authentication mechanisms at all . That is , as long as the server used in the system to
localize and track the user is running , it can accept any request from any user in the Internet . It can
be easily seen that the system is not secure since it accepts and serves any request from the Internet .
However , VRLT can be enhanced with any authentication mechanism in order to be assure that the
server is serving the right user . Also , users looking for localization service from a specific server
want to be assured they are connected to the right server .
There are several options that can provide VRLT system with a user authentication mechanism such
as : authenticating the users based on their IP addresses , Authenticating the users using usernames
and passwords , and authenticating the users through certificates signed by a trusted third party . The
latter option is applied to VRLT as user authentication mechanism . In the following section ,
certificate of type X.509 is explained in more details .
4.1.2 Using X.509 Certificates :
Most of the network security applications such as : S/MIME , IP sec , and SSL use X.509
certificates to present their identities to each other Stalling[9] . The main purpose of using X.509
certificates is to authenticate the users when they want to be communicating securely through a
public channel .
X.509 certificates are public key cryptosystem so they are inefficient to be used as direct encryption
to a big data Stalling[9] . However , they are used to encrypt a relatively small blocks , and to
exchange the secret keys between the communicating parties after they become authenticated to each
other . The secret keys exchanged can be then used to encrypt the data by one of symmetric
cryptosystems such Advanced Encryption System AES .
11
Figure 24 : Certifacet Signing
Each certificate has three main sections : (1) User information including the public key and ID , (2)
Information about Certificate Authority ,CA (3) and the Signature . Each user willing to have a
signed certificate has to create a pair of two keys ( private and public key ) , and sends only his
public key along with other public information such as the owner name and his ID to one of known
CAs . Then , CA in turn will hash the user information and encrypt the hash code by the CA private
key . The encryption of the hash code is actually the signature , the third section in each certificate ,
attached along with the other two sections in the certificate .
4.2 Security Second Level :
4.2.1 Encryption of Connection Stream: In this level , the connection between the client and the
server is secured from eavesdropping attacks. The security-enhanced VRLT has a client application
and server application that were built to provide two versions : The first one is to send the image
from the client to the server using a normal TCP connection , the second version is to encrypt the
connection between the client and the server by OpenSSL
Encryption algorithm can be defined to be used for the connection , however , the default encryption
algorithm of OpenSSL is used to encrypt the connection stream. The default algorithm is AES 256
SHA . Wireshark is used to capture the connection when it is unencrypted ( normal TCP ) and
encrypted (SSL above the TCP) . The following the an example of the encrypted and unencrypted
packet .
User Information ( Publick key , ID )
Hashing the user
information
Encrypt the the hashing code by the
CA
12
First , reading the image in a hexadecimal as in shown in figure 11 . ( the first 94 bytes from the
image are highlighted :
Figure 25 : Image bytes in Hexadecimal
Second : Figure 12 is the screenshot of Wireshark of first bytes of the image when it was not
encrypted ( normal TCP connection ) :
13
Figure 26 : Wireshark catpture for normal TCP
Third : Figure 13 is the screenshot of Wireshark of the first bytes of the image when it was encrypted
( TCP/SSL connection ) :
14
Figure 27 : Wireshark catpture for TCP/SSL
4.3 Security Third Level :
4.3.1 Database Encryption :
The database contains photos that were taken for the outdoor environment , and the features
extracted by the system along with the key files . In terms of security , it is important to keep the
database protected from unauthorized accesses . Two different sizes of database were created as
follows :
• Database 1 (DB1): has 114 photos , 114 features data files , 114 keys data files ,and 5 xml
15
files .
• Database 2 (DB2): has 228 photos 228 photos , 228 features data files , 228 keys data files
,and 5 xml files .
The VRLT system will read the 3D model through a reconstruction file which is an xml file . The
database is always kept in an encrypted form, and they will be decrypted as long as the user who is
trying to get connected to the server is authenticated by the server . AES algorithm is used as an
de/encryption system .
The encryption and decryption process take much time when the database gets larger . For this
reason , two different sizes of database (DB1 , DB2 ) are used to examine the effect of database
encryption on the performance of the system .
Moreover , two versions of encryption and decryption are used for our database encryption . The
first version is CPU based , whereas the second version is GPU based . The GPU version is used to
see how far the performance , in term of time consumption , could be improved when dealing with
large data .
4.3.2 Using AES Algorithm :
AES is a symmetric block cipher that was accepted to be published by National Institute of Stansards
Technology (NIST) in 2001 , and it was chosen to replace the DES as the approved standard for a
wide range of applications Stallings[9] .
The AES structure is complicated in comparison to the public key cryptosystems such as RSA …. .
AES structure has a number of rounds , the number of rounds depends on the key length . The key
length that AES could be used is : h16 bytes (128 bits ) , 24 bytes (192 bits) , or 32 bytes (256 bits) .
The number of rounds is 10 , 12 , or 14 when the key length is 16 bytes , 24 bytes , 32 bytes
respectively . Each round has number of stages that do the transformation ( permutation and
substitution ) of the input data , the types of stages will be explained in a little bit details in this
section . There is also an initial round which is sometimes called round 0 , this round has only one
stage called ADD ROUND KEY .
16
The input of plaintext to the AES is always a block of 16 bytes ( 128 bit-block ) , these 16 bytes are
arranged in a square matrix of 4 columns and 4 rows , the first four bytes will occupy the first
column , the second four bytes will occupy the second column , the third four bytes will occupy the
third column , and the fourth four bytes will occupy the last column as shown in the following figure
(14) :
Figure 28 : a square matrix of plaintext
The key will be expanded based on the key length itself , as mentioned previously that the key length
specifies the number of rounds . A length of 16 bytes from the key expansion will be supplied to
each round . In case of 16 byte length , the key will be expanded to 176 bytes whereas 24 bytes and
32 bytes will be expanded to 208 bytes , and 240 byes respectively . The Table … shows all the
possible parameters that AES can take :
17
Table 1 : AES parameters . source [9]
Each round has four stages ( Stage 1 : Substitute Bytes , Stage 2 : Shift Rows , Stage 3: Mix
Columns , and Stage 4 : Add Round Key ) unless the initial round has only Add Round Key and the
last stage has three stages (Stage 1 : Substitute Bytes , Stage 2 : Shift Rows , and Stage 3: Add
Round Key ) .
In term of Security , Add Round Key stage adds security to the AES that is why the AES starts and
ends with this stage . The other stages dose not use the key , and they are reversible without
knowledge of key , these stages add no security however they together provide confusion , diffusion
, and nonlinearity Stallings[9] .
The stages in each round are used for permutation and substitution What the four stages do in each
round as described as follows :
1- Substitute bytes: This stage is called SubBytes S-box is used in this stage to perform a byte-by-
byte substitution of the block Stalling[9]
The corresponding function prototype for this task as follows :
A . In CPU version :
18
B. In GPU version :
2- Shift Rows : This stage is called ShiftRows , and it is a simple permutation Stalling [9].
The corresponding function prototype for this task as follows :
A . In CPU version :
B. In GPU version :
3- Mix Columns: This stage is called MixColumns , and it is a substitution that makes use of
arithmetic over GF(28) Stallings[9] .
The corresponding function prototype for this task as follows :
A . In CPU version :
B. In GPU version :
19
4- Add Round Key : This stage is called AddRoundKey , and it is a simple bitwise XOR of the
current block with a portion of the expanded key Stallings [9] .
The corresponding function prototype for this task as follows :
A . In CPU version :
B. In GPU version :
The AES decryption process has the same number of rounds as the encryption process has .
However , each round has the following stages ( Stage 1 : Inverse Shift Rows , Stage 2 : Inverse
substitute Bytes , Stage 3 : Add Round Key , and Stage 4 : Inverse Mix Columns ) unless the initial
round has only Add Round Key and the last stage has three stages (Stage 1 : Inverse Shift Rows ,
Stage 2 : Inverse substitute Bytes , Stage 3 : Add Round Key) .
The AES decryption uses the key expansion in reverse . That means the last 16 bytes in the key
expansion will be used in the initial round in the decryption process , and the first 16 bytes in key
expansion will be used in the last round .
The four stages in each round in the decryption process are simply the inverse to the stages in the
decryption process . The following figure (15) shows the whole picture of how the encryption and
decryption process work in AES . The encryption is on the left side and starts from up to down
whereas the decryption process is on the right side and starts from down to up .
20
Figure 15 : Overview of AES algorithm source [9]
21
Chapter 5 : Evaluation and Results :
Due to the variation of the network measurement factors ( available bandwidth , processing time at
server and client , and transmission time ) , measuring the time overhead of each level that is applied
to VRLT is not easy , and requires to run the system several times , then calculating the average time
. The average time for VRLT to run without any of these security levels is 1636.76 when using DB1
whereas it is 1831.8 when using DB2 . The following sections will talk about each level in more
details .
5.1 Mutual User Authentication :
In this step , no encryption is applied , hence the connection is still plaintext and easily could be
eavesdropped by the attacker . However , the authentication for both peers ( client and server ) is
applied by using SSL connection . Authentication process happens during the 3 Handshaking phase
in SSL connection ( Authenticating the client at the server side and the server at the client ) . The
average time of authentication process when using DB1 is 37.58 ms and 31.72 ms when using DB2 (
the difference is due to the status of the network though they should be the same ) . Equation (2) is
used to calculate the overhead of 3Handshaking phase to the time needed to complete the
localization request in normal TCP .
Overhead = 3 Handshaking average time / The average time needed for the whole process in normal TCP
Equation * 100 ……………………………………………………………………………………………………… (2)
As a result , the overhead of authentication process for DB1 and DB2 is still minimal and has small
effects on the whole localization process as it can be seen in the following :
DB1 : Overhead = (37.58/1636.76)*100 = 2.29 %
DB2 : Overhead = ( 31.72 / 1831.8 ) * 100 = 1.7 %
5.2 Securing TCP Connection with SSL :
After the client is authenticated by the server as well as the server is authenticated at the client side ,
the data being transmitted between the client and the server is secured “encrypted” . Surprisingly ,
this level of security has almost no effect on the localization process . The VRLT has been tested so
many times to see the difference between sending the data securely and not securely . As a result ,
22
there is about 22 times of 50 times that normal TCP connection (unencrypted) took much time than
SSL connection ( encrypted ) . On average , sending the data in normal TCP when DB1 is used is
56.12 ms whereas in SSL is 61.13 , hence the difference is about 5 ms . Based on Equation (2) , the
overhead of adding this security level to the normal localization process when DB1 is used is :
Overhead = (5/1636.76)*100 = 0.3 %
Moreover , the average time of sending data in normal TCP when DB2 is used is 60.24 ms whereas
it is 63.34 ms in SSL connection ( the difference between DB1 and DB2 is due to the status of the
network though they should be the same ) . Based on Equation (2) , the overhead of adding this
security level to the normal localization process when DB2 is used is :
Overhead = (3/1831.8)*100 = 0.1 %
DB2 is actually created to examine the effect of database growth on the localization service that
VRLT provides . DB2 is a double size of DB1 , the average total time when DB2 is used is
compared to the average total time when DB1 is used . Anything else (Sending time , 3 Handshaking
) is the same in either cases ( DB1 or DB2) . The following table shows the comparison between the
average time to handle a localization request in DB1 and DB2 used in VRLT system :
Table 2: Comparison average total time between DB1 and DB2
Appendix A shows the comparison between the Normal TCP connection and the SSL connection for
the 50 runs . In Appendix A , Sending T means the time needed to send the data from peer to the
other peer, Total means the total time needed to make a request from the client till the pose sent back
to the client from the server. 3HS means 3 Handshaking for SSL Connection .
TCP TCP/SSL
Database 1 ( DB1) 1636.76 ms 1656.26 ms
Database 2 ( DB2) 1831.8 1877.45 ms
23
Appendix B shows the times needed for the 50 runs to complete the localization request for both
DB1 and DB2 in TCP connection and SSL connection .
5.3 AES Encryption of Database :
Encrypting and decrypting the database definitely come with overhead. This overhead will be
increased as long as the database increases as we will see in the following results . Table 3 shows the
average times of 20 runs for ( decryption process , decryption process , and total time needed to
handle the request ) when DB1 is used . Tables 4 shows the average time of another 20 runs when
DB2 is used . The average total time to handle the request dose not include the encryption time since
it only needs to consider the decryption time because the database is already encrypted and needs to
be decrypted to complete the localization process. After, serving the request , the database will be
encrypted again but it has no effects on measuring the localization process .
Table 3 : Comparison between CP and GPU when dealing with DB1
Version/process Encryption Decryption Total Time
CPU 1010.9 ms 2572.16 ms 4252.82 ms
GPU 383.975 ms 551.35 ms 2305 ms
24
Table 4: Comparison between CP and GPU when dealing with DB2
From table 3 and table 4 : it is clear that when the database is increased double as in table 4 (DB2),
it needs roughly an addition time of 54 % of the average time that DB1 needs to handle the request
when CPU is used in database decryption , whereas it needs 20 % as an addition time of the average
time that DB1 needs to handle the request when GPU is used . The addition times are calculated
based on the following equation
Addition time ={ ( Database 2 total time – Database 1 total time ) / Database 1 total time } * 100
……………………………………………………………………………………………………………………………… (3)
As result, We observed the following :
1- GPU is about 3 times faster than CPU in encryption process , and it is also 4 times faster than
CPU in decryption process when DB1 is used .
2- GPU is about 2.5 times faster than CPU in encryption process , and it is also 5 times faster
than CPU in decryption process when DB2 is used .
Appendix C shows the details of 20 runs for de/encrypting DB1 , whereas appendix D shows the
details of 20 runs when DB2 is used .
Chapter Six : Discussion and Future Work :
We can summaries the overhead as the following (our default database is DB1):
Version/process Encryption Decryption Total Time
CPU 1931.97 ms 4812.69 ms 6563.795 ms
GPU 719.27 ms 921.2 ms 2781.16 ms
25
• VRLT with a normal TCP connection needs 1636.76 ms to finish the localization service .
• VRLT needs another 37.58 ms for mutual user authentication , and negotiation between the
server and the client for which symmetric cipher to use for data transmission encryption .
The overhead is 2.29 % as an addition time will be needed to the normal TCP version .
• VRLT needs an addition of 5 ms to send the data securely through SSL ( encrypted ) . The
overhead of sending the data as encrypted to the normal TCP connection with the first
security level ( user authentication ) is 0.3 % .
• VRLT needs 2572.16 ms to decrypt the database when CPU is used , whereas it needs only
551.35 ms when GPU is used . The overhead of database decryption to the (
authentication + connection stream encryption )-enhanced VRLT is as follows: in CPU ,
it is 155 % , in GPU , it is 33 % .
However , here is the effects of the database growth on the localization process , and also how far
the GPU could improve the last security level ( Database Encryption ):
• VRLT with a normal TCP connection needs 1831.8 ms to finish the localization service .
• VRLT needs another 31.72 ms for mutual user authentication , and negotiation between the
server and the client for which symmetric cipher to use for data transmission encryption .
The overhead is 1.7 % as an addition time will be needed to the normal TCP version .
• VRLT needs an addition of 3 ms to send the data securely through SSL ( encrypted ) . The
overhead of sending the data as encrypted to the normal TCP connection with the first
security level ( user authentication ) is 0.1 % .
• VRLT needs 4812.69 ms to decrypt the database when CPU is used , whereas it needs 921.2
ms when GPU is used . The overhead of database decryption to the ( authentication +
connection stream encryption )-enhanced VRLT as follows : in CPU , it is 262 % , in
GPU , it is 50 % .
It is clear , The first two levels ( user mutual authentication and connection stream encryption ) add
minimal overhead to the normal TCP version , and this overhead decreases when the database grows
up , however the system is secured with these two essential security levels . As for the database
encryption , it adds much overhead specially when CPU version is used and the database grows .
However using GPU helps reduce the time overhead . Based on these results , it is strongly
recommended to add the first two security levels to VRLT ( user mutual authentication and
26
connection stream encryption ) since it is has small effects on the localization service and this effect
decreases when the database is increases . As for the third security level , it is obvious that it adds so
much overhead , however GPU version will be needed in case of demand of security is so high and it
is OK to make the response delayed .
As for future work , the optional security level ( extracting the features at the client side and sending
the features to the server instead of sending the image itself ) could add more security to the system
, also GPU could be used to extract the image features instead of using CPU version which will
improve the system in term of time consumption . Moreover , the database could be organized in any
DBMS instead of using a directory where every thing is saved in that directory and read through an
xml file .
27
References :
1-‐ J. Ventura and T. Ho ̈llerer, “Wide-‐area Scene Mapping for Mobile Visual Tracking,” in Int. Symposium on Mixed and Augmented Reality (ISMAR), 2012
2-‐ D. Wagner, A. Mulloni T. Langlotz, and D. Schmalstieg. Real-‐Time Self-‐Localization from Panoramic Images on Mobile Devices. In VR , pages 211-‐218
3-‐ G. Klein and D. Murray. Parallel Tracking and Mapping on a Camera Phone. In ISMAR, pages 83–86, 2009.
4-‐ R. Behringer, “Registration for Outdoor Augmented Reality Applications Using Computer Vision Techniques and Hybrid Sensors,” Proc. Virtual Reality Ann. Int’l Symp., pp. 244-‐251, 1999.
5-‐ I. Skrypnyk and D. Lowe, “Scene Modeling, Recognition and Tracking with Invariant Image Features,” Proc. Int’l Symp. Mixed and Augmented Reality (ISMAR ’04), pp. 110-‐119, 2004.
6-‐ H. Kawaji , K. Hatada , T. Yamasaki , K. Aizawa “Image-‐based Indoor Positioning System: Fast Image Matching using Omnidirectional . Panoramic Images” Proceedings of the 1st ACM international workshop on Multimodal pervasive video analysis Pages 1-‐4?
7-‐ C. Arth, D. Wagner, M. Klopschitz, A. Irschara, and D. Schmalstieg. Wide area localization on mobile phones. In Mixed and Augmented Reality, 2009. ISMAR 2009. 8th IEEE International Symposium on, pages 73–82. IEEE, 2009.
8-‐ S. Panzieri, F. Pascucci, and G. Ulivi, “An outdoor navigation system using GPS and inertial platform,” IEEE/ASME Trans. Mechatron., vol. 7, no. 2, pp. 134–142, Jun. 2002.
9-‐ W. Stallings “ CRYPTOGRAPY AND NETWORK SECURITY PRINCIPLES AND PRACTICE “ , 5th ed . Upper Saddle River: Pearson Education,INc, 2006 .
10-‐ www.opencv.rog , “Camera Calibration with OpenCV”
11-‐ I. Dubrawsky, HOW TO CHEAT AT Securing Your Network, Burlington: Amorette Pedersen , 2007 .
12 – Y. Pan, Y. Xiao , Security in Distributed and Networking Systems , 1st ed . Singapore: World Scientific Publishing Co. Pte. Ltd 2007 .
13 – Z. Duan , X. Yuan , J. Chandrashekar “ Controlling IP Spoofing Through Inter-‐Domain Packet Filters “ IEEE INFOCOM , 2006 .
28
APPENDEX A
TCP Conn With DB1
SSL Conn With DB1 Sending T Total
3 HS Sending T Total
61.304 1617.64
21.814 40.555 1578.15 42.273 1706.4
15.218 42.521 1779.17
44.161 1721.62
14.183 54.751 1591.94 49.732 1632.24
25.997 54.87 1621.06
30.803 1473.56
25.34 43.191 1624.55 44.009 1547.87
20.355 84.513 1781.34
38.171 1612.07
27.088 59.512 1617.05 62.532 1664.16
32.199 50.261 1653.78
63.477 1625.09
51.114 92.866 1635.72 39.593 1535.28
30.873 49.095 1732.56
38.674 1636.15
41.075 80.758 1665.53 34.602 1627.3
31.245 77.18 1722.47
64.625 1614.63
111.401 82.135 1783.32 43.322 1613.8
34.262 60.771 1635.99
44.749 1650.37
51.895 47.717 1725.12 79.801 1609.9
33.596 56.066 1732.97
47.828 1636.88
52.008 74.04 1616.24 36.64 1822.57
45.87 58.172 1719.26
31.633 1510.26
60.504 78.241 1736.75 88.106 1724.32
40.157 61.025 1620.65
52.621 1611.79
24.791 51.913 1701.93 47.18 1634.78
187.372 98.131 1639.07
64.559 1653.52
42.638 42.077 1716.03 83.479 1705.58
61.647 28.468 1721.38
37.946 1606.75
16.548 51.47 1593.89 63.426 1818.03
28.178 40.302 1685.34
73.541 1486.39
30.935 91.713 1625.99 45.279 1610.18
14.637 67.51 1770.39
74.53 1731.46
19.438 48.751 1601.81 48.781 1720.07
11.95 46.409 1495.28
78.352 1721.28
40.996 71.873 1636.09 88.206 1599.37
43.668 74.53 1561.39
50.823 1605.92
12.986 42.423 1591.61 48.93 1713.91
23.922 63.734 1616.76
46.025 1579.06
22.244 52.899 1658.53 67.471 1613.4
58.041 68.529 1650.55
46.167 1618.77
14.35 50.656 1671.14 39.387 1616.03
15.339 70.605 1676.54
66.012 1631.68
37.613 46.34 1726.1 68.269 1653.07
35.866 57.543 1694.53
29
92.044 1674.27
67.328 55.082 1634.49 62.289 1615.28
20.745 44.893 1608.84
64.132 1641.7
24.529 25.157 1591.88 60.156 1600.02
43.81 21.231 1517.91
62.499 1623.24
30.874 54.854 1621.82 51.991 1693.16
13.838 52.94 1597.34
79.9 1614.61
33.536 44.681 1619.67 65.404 1554.51
33.381 48.415 1644.29
44.027 1688.1
63.626 73.969 1745.82 46.975 1620.15
38.311 72.827 1622.93
56.12872 1636.7638 AVERAGE 37.58662 58.1633 1656.2592 AVERAGE
30
APPENDEX B
TCP Conn With DB1
SSL Conn With DB1 Sending T Total
3 HS Sending T Total
61.304 1617.64
21.814 40.555 1578.15 42.273 1706.4
15.218 42.521 1779.17
44.161 1721.62
14.183 54.751 1591.94 49.732 1632.24
25.997 54.87 1621.06
30.803 1473.56
25.34 43.191 1624.55 44.009 1547.87
20.355 84.513 1781.34
38.171 1612.07
27.088 59.512 1617.05 62.532 1664.16
32.199 50.261 1653.78
63.477 1625.09
51.114 92.866 1635.72 39.593 1535.28
30.873 49.095 1732.56
38.674 1636.15
41.075 80.758 1665.53 34.602 1627.3
31.245 77.18 1722.47
64.625 1614.63
111.401 82.135 1783.32 43.322 1613.8
34.262 60.771 1635.99
44.749 1650.37
51.895 47.717 1725.12 79.801 1609.9
33.596 56.066 1732.97
47.828 1636.88
52.008 74.04 1616.24 36.64 1822.57
45.87 58.172 1719.26
31.633 1510.26
60.504 78.241 1736.75 88.106 1724.32
40.157 61.025 1620.65
52.621 1611.79
24.791 51.913 1701.93 47.18 1634.78
187.372 98.131 1639.07
64.559 1653.52
42.638 42.077 1716.03 83.479 1705.58
61.647 28.468 1721.38
37.946 1606.75
16.548 51.47 1593.89 63.426 1818.03
28.178 40.302 1685.34
73.541 1486.39
30.935 91.713 1625.99 45.279 1610.18
14.637 67.51 1770.39
74.53 1731.46
19.438 48.751 1601.81 48.781 1720.07
11.95 46.409 1495.28
78.352 1721.28
40.996 71.873 1636.09 88.206 1599.37
43.668 74.53 1561.39
50.823 1605.92
12.986 42.423 1591.61 48.93 1713.91
23.922 63.734 1616.76
46.025 1579.06
22.244 52.899 1658.53 67.471 1613.4
58.041 68.529 1650.55
46.167 1618.77
14.35 50.656 1671.14 39.387 1616.03
15.339 70.605 1676.54
66.012 1631.68
37.613 46.34 1726.1 68.269 1653.07
35.866 57.543 1694.53
31
92.044 1674.27
67.328 55.082 1634.49 62.289 1615.28
20.745 44.893 1608.84
64.132 1641.7
24.529 25.157 1591.88 60.156 1600.02
43.81 21.231 1517.91
62.499 1623.24
30.874 54.854 1621.82 51.991 1693.16
13.838 52.94 1597.34
79.9 1614.61
33.536 44.681 1619.67 65.404 1554.51
33.381 48.415 1644.29
44.027 1688.1
63.626 73.969 1745.82 46.975 1620.15
38.311 72.827 1622.93
56.12872 1636.7638 AVERAGE 37.58662 58.1633 1656.2592 AVERAGE
32
APPENDEX C
CPU / DB1
GPU / DB1
Total Decryption Encryption
Total Decryption Encryption
4385.03 2639.36 1029.43
2106.2 539.08 360.112 4477.83 2835.67 1052.17
2199.18 599.166 377.008
4533.51 2685.04 1019.77
2200.48 494.888 367.934 4350.62 2719.68 1101.06
2186.47 569.598 403.979
4350.44 2723.43 1079.88
2238.2 493.411 420.392 4182.66 2560.39 1094.09
2487.75 529.319 405.558
4299.18 2651.67 1029.53
2328.23 577.154 376.874 4315.7 2504.88 992.637
2283 560.295 372.826
4194.18 2435.98 949.142
2473.6 583.867 388.734 4228.64 2686.57 981.562
2087.67 551.428 341.011
4348.74 2477.8 991.907
2297.61 499.05 368.409 4237.07 2582.36 972.267
2202.53 551.656 328.274
4100.63 2452.27 948.555
2219.54 555.562 364.976 4029.42 2463.58 970.697
2248.78 579.137 393.403
4000.67 2459.16 998.682
2370.21 577.297 397.543 4141.32 2479.67 957.753
2654.4 506.995 407.862
4597.94 2612.97 950.968
2348.96 584.228 402.149 4027.45 2456.67 1071.76
2577.82 561.959 379.046
4028.6 2454.44 994.832
2449.12 532.701 407.148 4226.9 2561.76 1031.53
2141.15 580.212 416.266
4252.8265 2572.1675 1010.9111 AVERAGE 2305.045 551.35015 383.9752 AVERAGE
33
APPENDEX D
CPU / DB2
GPU / DB2
Total Decryption Encryption
Total Decryption Encryption
6571.35 4739.34 1980.14
2673.15 909.437 742.516 6509.48 4733.23 1926.76
2769.92 971.824 620.587
6691.22 4813.73 2163.22
2853.2 869.072 652.781 6609.46 4918.26 1871.07
2613.24 856.677 746.193
6368.9 4750.58 1853.36
2813.67 907.251 758.37 6552.12 4780.52 1860.09
2959.81 907.076 715.443
6448.41 4730.78 1845.99
2804.66 920.212 753.906 6471.17 4738.87 1831.31
2722 975.859 723.778
6477.66 4820.85 2200.94
2591.31 983.095 621.91 7003.25 5210.09 2023.29
2698.75 900.031 703.321
6838.14 5036.27 2011.13
2758.63 881.452 747.811 6429.99 4779.86 1855.76
2594.16 868.075 722.938
6407.86 4694.59 1875.08
2985.07 997.91 714.49 6476.85 4789.82 1866.8
2956.11 948.994 732.845
6389.33 4750.87 1825.6
2741.87 920.804 756.211 6567.86 4744.67 1882.65
3083.74 930.135 768.618
6576.57 4766.81 1842.31
2764.34 957.661 741.406 6507.84 4822.78 1992.7
2643.99 874.091 666.164
6788.55 4857.4 2025.17
2852.84 948.892 758.692 6589.89 4774.47 1905.94
2742.69 895.642 737.482
6563.795 4812.6895 1931.9655 AVERAGE 2781.1575 921.2095 719.2731 AVERAGE
34
APPENDEX E
The firs Photo ( Center )
The second photo ( Left )
The third photo ( Right )
top related