-
596 IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY,
VOL. 9, NO. 4, APRIL 2014
Data Hiding in Encrypted H.264/AVC VideoStreams by Codeword
Substitution
Dawen Xu, Rangding Wang, and Yun Q. Shi, Fellow, IEEE
Abstract Digital video sometimes needs to be stored andprocessed
in an encrypted format to maintain security andprivacy. For the
purpose of content notation and/or tamper-ing detection, it is
necessary to perform data hiding in theseencrypted videos. In this
way, data hiding in encrypted domainwithout decryption preserves
the confidentiality of the content.In addition, it is more
efficient without decryption followed bydata hiding and
re-encryption. In this paper, a novel schemeof data hiding directly
in the encrypted version of H.264/AVCvideo stream is proposed,
which includes the following threeparts, i.e., H.264/AVC video
encryption, data embedding, anddata extraction. By analyzing the
property of H.264/AVC codec,the codewords of intraprediction modes,
the codewords of motionvector differences, and the codewords of
residual coefficients areencrypted with stream ciphers. Then, a
data hider may embedadditional data in the encrypted domain by
using codewordsubstitution technique, without knowing the original
video con-tent. In order to adapt to different application
scenarios, dataextraction can be done either in the encrypted
domain or in thedecrypted domain. Furthermore, video file size is
strictly pre-served even after encryption and data embedding.
Experimentalresults have demonstrated the feasibility and
efficiency of theproposed scheme.
Index Terms Data hiding, encrypted domain, H.264/AVC,codeword
substituting.
I. INTRODUCTION
CLOUD computing has become an important technologytrend, which
can provide highly efficient computationand large-scale storage
solution for video data. Given thatcloud services may attract more
attacks and are vulnerableto untrustworthy system administrators,
it is desired that thevideo content is accessible in encrypted
form. The capabilityof performing data hiding directly in encrypted
H.264/AVCvideo streams would avoid the leakage of video content,
which
Manuscript received November 12, 2013; accepted January 21,
2014. Dateof publication January 27, 2014; date of current version
March 4, 2014.This work was supported in part by the Natural
Science Foundation ofChina under Grants 61301247 and 61170137, in
part by Zhejiang ProvincialNatural Science Foundation of China
under Grant LY13F020013, and inpart by Ningbo Natural Science
Foundation under Grant 2013A610059. Theassociate editor
coordinating the review of this manuscript and approving itfor
publication was Dr. Adnan M. Alattar.
D. Xu is with the School of Electronics and Information
Engineer-ing, Ningbo University of Technology, Ningbo 315016, China
(e-mail:[email protected]).
R. Wang is with the CKC Software Laboratory, Ningbo University,
Ningbo315211, China (e-mail: [email protected]).
Y. Q. Shi is with the Department of Electrical and Computer
Engineering,New Jersey Institute of Technology, Newark, NJ
07102-1982 USA (e-mail:[email protected]).
Color versions of one or more of the figures in this paper are
availableonline at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TIFS.2014.2302899
can help address the security and privacy concerns with
cloudcomputing [1]. For example, a cloud server can embed
theadditional information (e.g., video notation, or
authenticationdata) into an encrypted version of an H.264/AVC video
byusing data hiding technique. With the hidden information,
theserver can manage the video or verify its integrity
withoutknowing the original content, and thus the security and
pri-vacy can be protected. In addition to cloud computing,
thistechnology can also be applied to other prominent
applicationscenarios. For example, when medical videos or
surveillancevideos have been encrypted for protecting the privacy
of thepeople, a database manager may embed the personal
informa-tion into the corresponding encrypted videos to provide
thedata management capabilities in the encrypted domain.
Till now, few successful data hiding schemes in theencrypted
domain have been reported in the open literature.In [2], a
watermarking scheme in the encrypted domainusing Paillier
cryptosystem is proposed based on the secu-rity requirements of
buyer-seller watermarking protocols.A Walsh-Hadamard transform
based image watermarkingalgorithm in the encrypted domain using
Paillier cryptosystemis presented in [3]. However, due to the
constraints of the Pail-lier cryptosystem, the encryption of an
original image resultsin a high overhead in storage and
computation. Note that,several investigations on reversible data
hiding in encryptedimages are reported in [4][8] recently. The
encryption is per-formed by using bit-XOR (exclusive-OR) operation.
In thesemethods, however, the host image is in an
uncompressedformat. In [9], a robust watermarking algorithm is
proposed toembed watermark into compressed and encrypted
JPEG2000images.
As said the above mentioned works have been focusedon image.
With the increasing demands of providing videodata security and
privacy protection, data hiding in encryptedH.264/AVC videos will
undoubtedly become popular in thenear future. Obviously, due to the
constraint of the underlyingencryption, it is very difficult and
sometimes impossible totransplant the existing data hiding
algorithms to the encrypteddomain. To the best of our knowledge,
there has been noreport on the implementation of data hiding in
encryptedH.264/AVC video streams. Only few joint data-hiding
andencryption approaches that focus on video have been proposed.For
example, in [10], during H.264/AVC compression, theintra-prediction
mode (IPM), motion vector difference (MVD)and DCT coefficients
signs are encrypted, while DCT coeffi-cients amplitudes are
watermarked adaptively. In [11], a com-bined scheme of encryption
and watermarking is presented,
1556-6013 2014 IEEE. Personal use is permitted, but
republication/redistribution requires IEEE permission.See
http://www.ieee.org/publications_standards/publications/rights/index.html
for more information.
-
XU et al.: DATA HIDING IN ENCRYPTED H.264/AVC VIDEO STREAMS
597
which can provide the access right as well as the
authenticationof video content simultaneously. The IPMs of 4 4
luminanceblock, the sign bits of texture, and the sign bits of
MVDsare encrypted, while IPM is used for watermarking. However,the
watermarked bitstream is not fully format-compliant as aresult a
standard decoder may crash since it cannot parse awatermarked
bitstream. Concretely, the value 2 of IPMdoes not exist in the
actual standard. In summary, in theexisting related technologies
[10][11], encryption and dataembedding are implemented almost
simultaneously duringH.264/AVC compression process. However, to
meet the afore-mentioned application requirements, its necessary to
performdata hiding directly in the encrypted domain. In addition,
theapproaches in [10] and [11] do not operate on the
compressedbitstream. That is, encryption and watermark embedding
areaccomplished in the encoding process, while decryption
andwatermark detection are completed in the decoding processThe
compression/decompression cycle is time-consuming andhampers
real-time implementation. Besides, encryption andwatermark
embedding would lead to increasing the bit-rateof H.264/AVC
bitstream.
Therefore, it becomes highly desirable to develop datahiding
algorithms that work entirely on encoded bitstreamin the encrypted
domain However, there are some signifi-cant challenges for data
hiding directly in compressed andencrypted bitstream. The first
challenge is to determine whereand how the bitstream can be
modified so that the encryptedbitstream with hidden data is still a
compliant compressedbitstream. The second challenge is to insure
that decryptedvideos containing hidden data can still appear to be
of highvisual fidelity. The third challenge is to maintain the
filesize after encryption and data hiding, which requires that
theimpact on compression gain is minimal. The fourth challengeis
that the hidden data can be extracted either from theencrypted
video stream or from the decrypted video stream,which is much more
applicable in practical applications.
Based on the analysis given above, we propose a novelscheme to
embed secret data directly in compressed andthen encrypted
H.264/AVC bitstream. Firstly, by analyzingthe property of H.264/AVC
codec, the codewords of IPMs,the codewords of MVDs, and the
codewords of residualcoefficients are encrypted with a stream
cipher. The encryptionalgorithm is combined with the Exp-Golomb
entropy codingand Context-adaptive variable-length coding (CAVLC)
[12],which keeps the codeword length unchanged. Then, datahiding in
the encrypted domain is performed based on anovel codeword
substituting scheme. In contrast to the existingtechnologies
[10][11] discussed above, the proposed schemecan achieve excellent
performance in the following threedifferent prospects.
The data hiding is performed directly in encryptedH.264/AVC
video bitstream.
The scheme can ensure both the format compliance andthe strict
file size preservation.
The scheme can be applied to two different applicationscenarios
by extracting the hidden data either from theencrypted video stream
or from the decrypted videostream.
The remainder of the paper is organized as follows. InSection
II, we describe the proposed scheme, which includesthree parts,
i.e., H.264/AVC video encryption, data embeddingand data
extraction. Experimental results are presented inSection III.
Discussion is shown in Section IV. Finally inSection V, conclusion
is drawn.
II. PROPOSED SCHEME
In this section, a novel scheme of data hiding in theencrypted
version of H.264/AVC videos is presented, whichincludes three
parts, i.e., H.264/AVC video encryption, dataembedding and data
extraction. The content owner encryptsthe original H.264/AVC video
stream using standard streamciphers with encryption keys to produce
an encrypted videostream. Then, the data-hider (e.g., a cloud
server) can embedthe additional data into the encrypted video
stream by usingcodeword substituting method, without knowing the
originalvideo content. At the receiver end, the hidden data
extractioncan be accomplished either in encrypted or in
decryptedversion. The diagram of the proposed framework is shown
inFig. 1, where the encryption and data embedding are depictedin
part (a), and the data extraction and video decryption areshown in
part (b).
A. Encryption of H.264/AVC Video Stream
Video encryption often requires that the scheme be timeefficient
to meet the requirement of real time and formatcompliance. It is
not practical to encrypt the whole compressedvideo bitstream like
what the traditional ciphers do becauseof the following two
constraints, i.e., format compliance andcomputational cost.
Alternatively, only a fraction of video datais encrypted to improve
the efficiency while still achievingadequate security. The key
issue is then how to select thesensitive data to encrypt. According
to the analysis givenin [13], it is reasonable to encrypt both
spatial information(IPM and residual data) and motion information
(MVD) duringH.264/AVC encoding.
In this paper, an H.264/AVC video encryption schemewith good
performance including security, efficiency, andformat compliance is
proposed. By analyzing the prop-erty of H.264/AVC codec, three
sensitive parts (i.e., IPMs,MVDs, and residual coefficients) are
encrypted with streamciphers. Compared with [13], the proposed
encryption algo-rithm is performed not during H.264/AVC encoding
but inthe H.264/AVC compressed domain. In this case, the bit-stream
will be modified directly. Selective encryption in theH.264/AVC
compressed domain has been already presentedon context-adaptive
variable length coding (CAVLC) [14] andcontext-adaptive binary
arithmetic coding (CABAC) [15]. Inthis paper, we have improved and
enhanced the previousproposed approach by encrypting more syntax
elements. Weencrypt the codewords of IPMs, the codewords of
MVDs,and the codewords of residual coefficients. The
encryptedbitstream is still H.264/AVC compliant and can be decoded
byany standard-compliant H.264/AVC decoder, but the encryptedvideo
data is treated completely different compared to plain-text video
data. In fact, performing the format-compliant
-
598 IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY,
VOL. 9, NO. 4, APRIL 2014
Fig. 1. Diagram of proposed scheme. (a) Video encryption and
data embedding at the sender end. (b) Data extraction and video
display at the receiver endin two scenarios.
encryption directly on the compressed bitstream is
extremelycomplicated as the internal states of the encoder have to
bepreserved [16], otherwise the remaining data is
interpretedfalsely which may easily lead to format violations.
1) Intra-Prediction Mode (IPM) Encryption: According toH.264/AVC
standard, the following four types of intra codingare supported,
which are denoted as Intra_4 4, Intra_1616,Intra_chroma, and I_PCM
[12]. Here, IPMs in the Intra_44and Intra_16 16 blocks are chosen
to encrypt.
Four intra prediction modes (IPMs) are available in theIntra_16
16. The IPM for Intra_16 16 block is spec-ified in the mb_type
(macroblock type) field which alsospecifies other parameters about
this block such as codedblock pattern (CBP). Table I is the list of
mb_type valueswith their meanings which are taken from the standard
[17].In H.264/AVC baseline profile, the mb_type is encoded withthe
Exp-Golomb code. To maintain standard-compliance ofthe encrypted
bitstream, we can encrypt the codeword ofan IPM without modifying
the CBP. In addition, to keepthe codewords length unchanged, the
encrypted codewordshould have the same size as the original
codeword. It can
be observed that the combination of CBP is the same in everyfour
lines, and the codewords have the same length in everytwo
consecutive lines, as shown in Table I. For example,the codewords
corresponding to 3 and 4 are 00100and 00101, respectively, which
have the same length andthe same value of CBP. Thus for Intra_16 16
block, theIPM encryption is performed by applying a bitwise
XORoperation between the last bit of the codewords and a bit ofthe
pseudorandom sequence to keep the value of CBP andthe length of
codeword unchanged [18]. The pseudorandomsequence is generated via
a standard secure cipher (e.g., RC4)determined by an encryption key
E_Key1.
In H.264/AVC, each Intra_4 4 luminance block is pre-dicted from
its spatially neighboring samples. Specifically,H.264/AVC offers
nine prediction modes (0-8) for Intra_44luminance blocks. The
choice of prediction mode for eachIntra_4 4 luminance block must be
signalled to the decoderand this could potentially require a large
number of bits.To efficiently compress the prediction-mode data,
the predic-tive coding technique is applied to signal prediction
modes.For each currently considered block E , the most probable
-
XU et al.: DATA HIDING IN ENCRYPTED H.264/AVC VIDEO STREAMS
599
TABLE I
MACROBLOCK TYPES FOR I SLICES AND VARIABLE LENGTH OF
CODEWORD IN H.264/AVC [17]
mode (M P ME ) is defined as the smaller of the predictionmodes
of the spatially adjacent upper block A and leftblock B [19]. If
either of these adjacent blocks is not available,the corresponding
value is set to 2, indicating DC predictionmode.
The prediction mode of the currently considered block E
isdenoted as ModeE . If ModeE is equal toM P ME , only onebit is
needed to signal the prediction mode. When ModeE andM P ME are
different, the codeword is composed of one flagbit 0and three bits
fixed-length code [19].
For Intra_4 4 block, if ModeE is equal to M P ME ,the codeword
is kept unchanged. Otherwise, three bits fixed-length code (denoted
as X) [13] in each codeword is encryptedwith a pseudorandom
sequence which is generated via astandard secure cipher (e.g., RC4)
determined by an encryptionkey E_Key2. Bitwise XOR operation is
still utilized as theencryption scheme.
From what described above, it is obvious that the length ofthe
encrypted codeword is the same as the original one. Forthe format
compliance in the decoding process, the encryptedIPMs of blocks in
the first row and/or in the first columnshould have the decodable
value, since not all modes areavailable along the top and the left
borders of each framedue to the lack of neighbors. In our scheme,
if the IPMafter encryption is not available for a border block,
then theIPM encryption of this block will be skipped. This
further
TABLE II
MVDs AND CORRESPONDING EXP-GOLOMB CODEWORDS
indicates that IPM encryption is not secure enough in
somespecific locations and should be used in combination withother
encrypting method. In summary, IPM encryption implieschanging the
actual mode to another one without violating thesemantics and
bitstream compliance.
2) Motion Vector Difference (MVD) Encryption: In order toprotect
both texture information and motion information, notonly the IPMs
but also the motion vectors should be encrypted.In H.264/AVC,
motion vector prediction is further performedon the motion vectors,
which yields MVD. In H.264/AVCbaseline profile, Exp-Golomb entropy
coding [19] is used toencode MVD. The codeword of Exp-Golomb is
constructedas[M zeros] [1] [I N FO ], where I N FO is an M-bit
fieldcarrying information.
Table II shows the values of MVDs and correspondingExp-Golomb
codewords. The last bit of the codeword isencrypted by applying the
bitwise XOR operation with astandard stream cipher, which is
determined by an encryptionkey E_Key3. According to Table II, the
last bit encryptionmay change the sign of MVD, but does not affect
the lengthof the codeword and satisfies the format compliance [10].
Thatis, the resulting ciphertexts are still valid Exp-Golomb
codes.For example, the codewords corresponding to 2 and 2are 00100
and 00101, respectively, which have the samelength. It should be
noted that when the value of MVD isequal to 0, its corresponding
codeword 1 keeps unchangedduring the encryption process.
3) Residual Data Encryption: In order to keep high
security,another type of sensitive data, i.e., the residual data in
bothI-frames and P-frames should be encrypted. In this section,
anovel method for encrypting the residual data based on
thecharacteristics of codeword is presented in detail.
-
600 IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY,
VOL. 9, NO. 4, APRIL 2014
In H.264/AVC baseline profile, CAVLC entropy codingis used to
encode the quantized coefficients of a residualblock [19]. Each
CAVLC codeword can be expressed as thefollowing format:
{Coe f f token, Sign of T railingOnes,Level, T otal zeros, Run
bef ore}
The specific function of each syntax element is describedin
[19]. To keep the bitstream compliant, not all syntaxelements can
be modified during encryption process. Forexample,Coef f token, T
otal zeros, and Run bef ore shouldremain unchanged [14]. Therefore,
residual data encryp-tion can be accomplished by modifying the
codewords ofSign_of_TrailingOnes and Level.
The Sign_of_TrailingOnes is encoded with a single bit. Bit0 is
assigned for +1 and bit 1is assigned for 1. Thecodeword of
Sign_of_TrailingOnes is encrypted by applyingthe bitwise XOR
operation with a standard stream cipher,which is determined by an
encryption key E_Key4.
The codeword for each Level is made up of a prefix(level_prefix)
and a suffix (level_suffix) as
Level codeword = [level pre f i x], [level su f f i x]Table III
shows Levels with different suffixLength and corre-
sponding codewords. The last bit of the codeword is encryptedby
applying the bitwise XOR operation with a standard streamcipher,
which is determined by an encryption key E_Key5.According to Table
III, the last bit encryption may changethe sign of Levels, but does
not affect the length of thecodeword and satisfies the format
compliance. For example,when suffixLength is equal to 1, the
codewords correspondingto 2 and -2 are 010 and 011, respectively,
which havethe same length. It should be noted that when
suffixLength isequal to 0, the codewords should keep unchanged
during theencryption process.
B. Data Embedding
Although few methods have been proposed to embed datainto
H.264/AVC bitstream directly [20][21], however, thesemethods cannot
be implemented in the encrypted domain.In the encrypted bitstream
of H.264/AVC, the proposed dataembedding is accomplished by
substituting eligible codewordsof Levels in Table III. Since the
sign of Levels are encrypted,data hiding should not affect the sign
of Levels. Besides,the codewords substitution should satisfy the
following threelimitations. First, the bitstream after codeword
substitutingmust remain syntax compliance so that it can be decoded
bystandard decoder. Second, to keep the bit-rate unchanged,
thesubstituted codeword should have the same size as the
originalcodeword. Third, data hiding does cause visual
degradationbut the impact should be kept to minimum. That is,
theembedded data after video decryption has to be invisible toa
human observer. So the value of Level corresponding tothe
substituted codeword should keep close to the value ofLevel
corresponding to the original codeword. In addition,the codewords
of Levels within P-frames are used for datahiding, while the
codewords of Levels in I-frames are remained
TABLE III
LEVELS AND CORRESPONDING CODEWORDS
unchanged. Because I-frame is the first frame in a groupof
pictures (GOPs), the error occurred in I-frame will bepropagated to
subsequent P-frames.
According to the analysis given above, we can see that thereare
no corresponding substituted codewords when suffixLengthis equal to
0 or 1, as shown in Table III. When suffixLengthis equal to 0, we
cannot find a pair of codewords with thesame size. When
suffixLength is equal to 1, one codeword alsocannot be substituted
by another codeword with the same size,since this substitution
would change the sign of Level. Thenthe codewords of Levels which
suffixLength is 2 or 3 would be
-
XU et al.: DATA HIDING IN ENCRYPTED H.264/AVC VIDEO STREAMS
601
Fig. 2. CAVLC codeword mapping. (a) su f f i x Length = 2&
Level > 0.(b) su f f i x Length = 2& Level > 0. (c) su f
f i x Length = 3& Level > 0.(d) su f f i x Length = 3&
Level < 0.
divided into two opposite codespaces denoted as C0and C1as shown
in Fig. 2. The codewords assigned in C0and C1 areassociated with
binary hidden information 0 and 1.
Suppose the additional data that we want to embed is abinary
sequence denoted as B = {b(i)|i = 1, 2, , L, b(i) {0, 1}}. Data
hiding is performed directly in encrypted bit-stream through the
following steps.
Step1. In order to enhance the security, the additionaldata is
encrypted with the chaotic pseudo-random sequenceP = {p(i)|i = 1,
2, , L, p(i) {0, 1}}[22] to generate theto-be-embedded sequence W =
{w(i)|i = 1, 2, , L, w(i) {0, 1}}. The sequence P is generated by
using logistic mapwith an initial value [22], i.e., the data hiding
key. It is verydifficult for anyone who does not retain the data
hiding keyto recover the additional data.
Step2. The codewords of Levels are obtained by parsing
theencrypted H.264/AVC bitstream.
Step3. If current codeword belongs to codespaces C0orC1,the
to-be-embedded data bit can be embedded by codewordsubstituting.
Otherwise, the codeword is left unchanged. Thedetailed procedure of
codeword substituting for data hiding isshown in Fig. 3. For
example, when Level is positive 1 and itssufflxLength is 3, then
its corresponding codeword is 1000which belongs to C0 as shown in
Fig. 2(c). If the data bit1 needs to be embedded, the codeword 1000
should bereplaced with 1010. Otherwise, if the data bit 0 needs
toembedded, the codeword 1000 will keep unchanged.
Step4. Choose the next codeword and then go to Step3 fordata
hiding. If there are no more data bits to be embedded,the embedding
process is stopped.
Suppose the to-be-embedded data is 1001, the CAVLCcodeword of
Level parsing from H.264/AVC bitstream is
Fig. 3. The procedure of codeword mapping.
01 010 00100 00100 0001011 0000100 and the encryptionstream is
10111, an example of data embedding based oncodeword mapping is
shown in Fig. 4(a).
C. Data Extraction
In this scheme, the hidden data can be extracted either
inencrypted or decrypted domain, as shown in Fig. 1(b).
Dataextraction process is fast and simple. We will first discuss
theextraction in encrypted domain followed by decrypted domain.
1) Scheme I: Encrypted Domain Extraction. To protectprivacy, a
database manager (e.g., cloud server) may only getaccess to the
data hiding key and have to manipulate datain encrypted domain.
Data extraction in encrypted domainguarantees the feasibility of
our scheme in this case.
In encrypted domain, as shown in Fig. 1(b), encrypted videowith
hidden data is directly sent to the data extraction module,and the
extraction process is given as follows.
Step1: The codewords of Levels are firstly identified byparsing
the encrypted bitstream.
Step2: If the codeword belongs to codespace C0, theextracted
data bit is 0. If the codeword belongs to codespaceC1, the
extracted data bit is 1.
Step3: According to the data hiding key, the same
chaoticpseudo-random sequence P that was used in the
embeddingprocess can be generated. Then the extracted bit
sequencecould be decrypted by using P to get the original
additionalinformation. Since the whole process is entirely operated
inencrypted domain, it effectively avoids the leakage of
originalvideo content.
An example of data extraction in encrypted domain is shownin
Fig. 4(b).
2) Scheme II: Decrypted Domain Extraction. In scheme I,both
embedding and extraction of the data are performedin encrypted
domain. However, in some cases, users wantto decrypt the video
first and extract the hidden data fromthe decrypted video. For
example, an authorized user, whichowned the encryption key,
received the encrypted video withhidden data. The received video
can be decrypted usingthe encryption key. That is, the decrypted
video still includesthe hidden data, which can be used to trace the
source of thedata. Data extraction in decrypted domain is suitable
for this
-
602 IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY,
VOL. 9, NO. 4, APRIL 2014
Fig. 4. An example of data embedding and extraction. (a) Data
embedding.(b) Data extraction in encrypted domain. (c) Data
extraction in decrypteddomain.
case. As shown in Fig. 1(b), the received encrypted video
withhidden data is first pass through the decryption module.
Thewhole process of decryption and data extraction is given
asfollows.
Step1: Generate encryption streams with the encryption keysas
given in encryption process.
Step2: The codewords of IPMs, MVDs,Sign_of_TrailingOnes and
Levels are identified by parsing theencrypted bitstream.
Step3: The decryption process is identical to the
encryptionprocess, since XOR operation is symmetric. The
encrypted
codewords can be decrypted by performing XOR operationwith
generated encryption streams, and then two XOR opera-tions cancel
each other out, which renders the original plain-text. Since the
encryption streams depend on the encryptionkeys, the decryption is
possible only for the authorized users.After generating the
decrypted codewords with hidden data,the content owner can further
extract the hidden information.
Step4: According to Table III, the last bit encryption maychange
the sign of Level. However, as shown in Fig. 2, theencrypted
codeword and the original codeword are still in thesame codespaces.
If the decrypted codeword of Level belongsto codespace C0, the
extracted data bit is 0. If the decryptedcodeword of Level belongs
to codespace C1, the extracted databit is 1.
Step5: Generate the same pseudo-random sequence P thatwas used
in embedding process according to the data hidingkey. The extracted
bit sequence should be decrypted to get theoriginal additional
information.
An example of data extraction in decrypted domain is shownin
Fig. 4(c).
III. EXPERIMENTAL RESULTS
The proposed data hiding scheme has been implementedin the
H.264/AVC reference software version JM-12.2. Sixwell-known
standard video sequences (i.e., Stefan, Table,Tempete, Mobile,
Hall, and News) in QCIF format (176 144)at the frame rate 30
frames/s are used for simulation. The first100 frames in each video
sequence are used in the experiments.The GOP (Group of Pictures)
structure is IPPPP: one I framefollowed four P frames.
A. Security of Encryption Algorithm
For the proposed video encryption scheme, the securityincludes
both cryptographic security and perceptual security.Cryptographic
security denotes the security against crypto-graphic attacks, which
depends on the ciphers adopted by thescheme. In the proposed
scheme, the secure stream cipher(e.g., RC4) is used to encrypt the
bitstream, and chaoticpseudo-random sequence generated by logistic
map is usedto encrypt the additional data. They have been proved
tobe secure against cryptographic attacks. Perceptual
securityrefers to whether the encrypted video is unintelligible or
not.Generally, it depends on the encryption schemes properties.For
example, encrypting only IPM cannot keep secure enough,since the
encrypted video is intelligible [10]. The proposedscheme encrypts
IPM, MVD and residual coefficients, whichkeeps perceptual security
of the encrypted video. The demon-stration is shown in Figs. 5 and
6. An original frame from eachvideo is depicted in Fig. 5, and
their corresponding encryptedresults are depicted in Fig. 6. Other
frames have a similareffect of encryption. Due to space
limitations, we do not listthe results of all frames. It should be
mentioned that not everyvideo can be degraded to the same extent.
The perceptual qual-ity of high-motion videos with a complex
textured backgroundbecomes much more scrambled after encryption
than that oflow-motion videos with a static background. The reason
is thatthere are less residual coefficients and MVDs in
low-motion
-
XU et al.: DATA HIDING IN ENCRYPTED H.264/AVC VIDEO STREAMS
603
Fig. 5. Original video frames.
Fig. 6. Encrypted video frames.
videos that are available for encryption. In general,
scramblingperformance of the described encryption system is more
thanadequate.
B. Visual Quality of Stego Video
The encrypted video containing hidden data provided by theserver
should be decrypted by the authorized user. Therefore,the visual
quality of the decrypted video containing hiddendata is expected to
be equivalent or very close to that ofthe original video. By
modifying the compressed bitstreamto embed additional data, the
most important challenge is tomaintain perceptual transparency,
which refers to the modifi-cation of bitstream should not degrade
the perceived contentquality. In this paper, only the codewords of
Levels withinP-frames are modified for data hiding. Simulation
results havedemonstrated that we can embed the additional data
witha large capacity into P-frames while preserving high
visualquality. The encrypted and decrypted video frames with
hiddendata are shown in Figs 7 and 8 respectively. In the
experiments,no visible artifacts have been observed in all of the
decryptedvideo frames with hidden data.
Besides subjective observation, PSNR (Peak Signal to
NoiseRatio), SSIM (Structural Similarity Index), and VQM
(VideoQuality Measurement) have been adopted to evaluate
theperceptual quality [22]. PSNR is widely used objective
videoquality metric. However, it does not perfectly correlate with
a
Fig. 7. Encrypted video frames with hidden data.
Fig. 8. Decrypted video frames with hidden data.
perceived visual quality due to nonlinear behavior of
humanvisual system. The SSIM index lies in the range between0 and
1, where 1 indicates the reference image is identicalthan the
target image. Since H.264/AVC is lossy compression,in order to
better illustrate the data hiding on the video quality,the visual
quality of non-stego video stream should be tested.The video
sequence obtained by decompressing non-stegovideo stream is used as
the target sequence, while the originaluncompressed video sequence
is used as the reference videosequence. Similarly, in order to test
the visual quality of stegovideo stream, the video sequence
obtained by encrypting, datahiding, decrypting, and decompressing
process is used as thetarget sequence. That is, in this case, the
target video containshidden data. The VQM is another approach to
measure videoquality that correlates more with the human visual
system.In general, the lower VQM value indicates higher
perceptualvideo quality, and zero indicates excellent quality. The
experi-mental results are shown in Table IV. As can be seen, a
higherQP (quantization parameter) will result in lower video
quality.The visual quality degradation of decrypted video
containinghidden data is very low even for large payloads, i.e., it
isgenerally hard to detect the degradation in video quality
causedby data hiding.
C. Embedding Capacity
Data hiding payload can be assessed in kilobits per
second(kbits/s) [23]. The maximum payload capacity in each
video
-
604 IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY,
VOL. 9, NO. 4, APRIL 2014
TABLE IV
EMBEDDING CAPACITY, PSNR, SSIM, AND VQM IN DIRECTLY DECRYPTED
VIDEOS (CAVLC CODEWORD MAPPING)
encoded with different QP values is given in Table IV. Payloadof
the proposed scheme depends on type of video content andthe QP
values. The reason for this is that each video streamhas a
different number of qualified codewords. Embeddingcapacity is
determined by the number of qualified codewords.Obviously, high
motion sequence (e.g., Stefan, Table) hasmuch larger number of
qualified codewords. This is be causedby high motion sequence has
more qualified Levels, sincedata hiding is only performed within
P-frames. Specifically,payload decreases with increase in QP value.
When QP valueincreases, the number of residual coefficients
decreases, andthen the qualified Levels will be less.
D. Bit Rate Variation
To further evaluate the performance of the proposed scheme,bit
rate variation BR_var caused by encryption and data hidingis also
introduced.
B R var = B R em B R origB R orig
100%
where BR_em is the bit rate generated by encryption and
dataembedding encoder, and BR_orig is the bit rate generated bythe
original encoder. The bit rate of the encrypted and stegovideo
remains unchanged. This is because the encryption anddata hiding
are all performed by replacing a suitable codewordto another
codeword with the same length, as described inSection II (A) and
(B). In summary, the encryption processand data hiding process do
not affect compression efficiencyof encoders, since compression
efficiency is typically definedby the bit rate.
Fig. 9. Exp-Golomb codeword mapping. (a) MV D > 0. (b) MV D
< 0.
IV. DISCUSSION
In addition to the codewords of Levels, the Exp-Golombcodewords
of MVDs can also be used for data hiding. FromTable II, we can see
that there are no corresponding substitutedcodewords when MVD is
equal to 0, because we cannotfind a pair of codeword with the same
size. In addition, thecodewords 010 and 011should not be modified
during datahiding, since they are encrypted codeword pair. Then the
restedcodewords would be divided into two opposite
codespacesdenoted as C0and C1as shown in Fig. 9. The
codewordsassigned in C0and C1 are associated with binary
hiddeninformation 0 and 1.
Data hiding and extraction procedure are the same as
thepreviously described in Section II (B) and (C), respectively.The
experiment results are shown in Table V. According toTable V, for
high motion sequences (such as Stefan, Table)and high texture
sequences (such as Tempete and Mobile), thedegradation in video
quality caused by MVDs Exp-Golombcodeword substituting is more
serious than the previous Levels
-
XU et al.: DATA HIDING IN ENCRYPTED H.264/AVC VIDEO STREAMS
605
TABLE V
EMBEDDING CAPACITY, PSNR, SSIM, AND VQM IN DIRECTLY DECRYPTED
VIDEOS (EXP-GOLOMB CODEWORD MAPPING)
TABLE VI
TEST RESULTS BASED ON THE COMBINATION OF CAVLC CODEWORD MAPPING
AND EXP-GOLOMB CODEWORD MAPPING
codeword substituting method. For this type of video, it
isappropriate to embed data using the codewords of Levels.According
to Table IV, for low motion sequence (such as Hall,News), the
embedding capacity is low if only the codewordsof Levels are used
for data hiding. In this case, both theCAVLC codewords of Levels
and the Exp-Golomb codewordsof MVDs can be used for data hiding. As
depicted in Table V,for this type of video, the degradation in
video quality causedby data hiding is quite small. So the
combination is entirelyfeasible. The test results based on the
combination of theCAVLC codewords of Levels and the Exp-Golomb
codewordsof MVDs are also given in Table VI. Compared with Table
IV,the embedding capacity is improved obviously, but the
videoquality degradation is also negligible. Based on the
aboveanalysis, we can make a flexible choice of embedding
carrieraccording to the situation.
To the best of our knowledge, till now, there is no algorithmto
embed additional data directly in encrypted H.264/AVCvideo stream.
Therefore, no detailed experimental comparisons
are given in the paper. As described in Section I, in the
existingrelated technologies [10][11], encryption and data
hidingare accomplished almost simultaneously during
H.264/AVCencoding process. In addition, encryption and data
embeddingwould lead to increasing the bit-rate of H.264/AVC
bitstream.On the contrary, our proposed scheme can encrypt
H.264/AVCvideo stream directly and then embeds data into
encryptedH.264/AVC video stream to meet the
privacy-preservingrequirements. The bit-rate of the encrypted
H.264/AVC videostream containing hidden data is exactly the same as
theoriginal H.264/AVC video stream.
V. CONCLUSION
Data hiding in encrypted media is a new topic that hasstarted to
draw attention because of the privacy-preservingrequirements from
cloud data management. In this paper, analgorithm to embed
additional data in encrypted H.264/AVCbitstream is presented, which
consists of video encryption,data embedding and data extraction
phases. The algorithm
-
606 IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY,
VOL. 9, NO. 4, APRIL 2014
can preserve the bit-rate exactly even after encryption anddata
embedding, and is simple to implement as it is directlyperformed in
the compressed and encrypted domain, i.e.,it does not require
decrypting or partial decompression ofthe video stream thus making
it ideal for real-time videoapplications. The data-hider can embed
additional data into theencrypted bitstream using codeword
substituting, even thoughhe does not know the original video
content. Since data hidingis completed entirely in the encrypted
domain, our method canpreserve the confidentiality of the content
completely. Withan encrypted video containing hidden data, data
extractioncan be carried out either in encrypted or decrypted
domain,which provides two different practical applications.
Anotheradvantage is that it is fully compliant with the
H.264/AVCsyntax. Experimental results have shown that the
proposedencryption and data embedding scheme can preserve
file-size,whereas the degradation in video quality caused by data
hidingis quite small.
REFERENCES
[1] W. J. Lu, A. Varna, and M. Wu, Secure video processing:
Problems andchallenges, in Proc. IEEE Int. Conf. Acoust., Speech,
Signal Processing,Prague, Czech Republic, May 2011, pp.
58565859.
[2] B. Zhao, W. D. Kou, and H. Li, Effective watermarking scheme
inthe encrypted domain for buyer-seller watermarking protocol, Inf.
Sci.,vol. 180, no. 23, pp. 46724684, 2010.
[3] P. J. Zheng and J. W. Huang, Walsh-Hadamard transform in the
homo-morphic encrypted domain and its application in image
watermarking,in Proc. 14th Inf. Hiding Conf., Berkeley, CA, USA,
2012, pp. 115.
[4] W. Puech, M. Chaumont, and O. Strauss, A reversible
datahiding method for encrypted images, Proc. SPIE, vol. 6819,pp.
68191E-168191E-9, Jan. 2008.
[5] X. P. Zhang, Reversible data hiding in encrypted image, IEEE
SignalProcess. Lett., vol. 18, no. 4, pp. 255258, Apr. 2011.
[6] W. Hong, T. S. Chen, and H. Y. Wu, An improved reversible
datahiding in encrypted images using side match, IEEE Signal
Process.Lett., vol. 19, no. 4, pp. 199202, Apr. 2012.
[7] X. P. Zhang, Separable reversible data hiding in
encryptedimage, IEEE Trans. Inf. Forensics Security, vol. 7, no.
2,pp. 826832, Apr. 2012.
[8] K. D. Ma, W. M. Zhang, X. F. Zhao, N. Yu, and F. Li,
Reversible datahiding in encrypted images by reserving room before
encryption, IEEETrans. Inf. Forensics Security, vol. 8, no. 3, pp.
553562, Mar. 2013.
[9] A. V. Subramanyam, S. Emmanuel, and M. S. Kankanhalli,
Robustwatermarking of compressed and encrypted JPEG2000 images,
IEEETrans. Multimedia, vol. 14, no. 3, pp. 703716, Jun. 2012.
[10] S. G. Lian, Z. X. Liu, and Z. Ren, Commutative encryption
andwatermarking in video compression, IEEE Trans. Circuits Syst.
VideoTechnol., vol. 17, no. 6, pp. 774778, Jun. 2007.
[11] S. W. Park and S. U. Shin, Combined scheme of encryption
andwatermarking in H.264/scalable video coding (SVC), New
DirectionsIntell. Interact. Multimedia, vol. 142, no. 1, pp.
351361, 2008.
[12] T. Wiegand, G. J. Sullivan, G. Bjontegaard, and A. Luthra,
Overviewof the H.264/AVC video coding standard, IEEE Trans.
Circuits Syst.Video Technol., vol. 13, no. 7, pp. 560576, Jul.
2003.
[13] S. G. Lian, Z. X. Liu, Z. Ren, and H. L. Wang, Secure
advancedvideo coding based on selective encryption algorithms, IEEE
Trans.Consumer Electron., vol. 52, no. 2, pp. 621629, May 2006.
[14] Z. Shahid, M. Chaumont, and W. Puech, Fast protection of
H.264/AVCby selective encryption of CAVLC and CABAC for I and P
frames,IEEE Trans. Circuits Syst. Video Technol., vol. 21, no. 5,
pp. 565576,May 2011.
[15] M. N. Asghar and M. Ghanbari, An efficient security system
forCABAC bin-strings of H.264/SVC, IEEE Trans. Circuits Syst.
VideoTechnol., vol. 23, no. 3, pp. 425437, Mar. 2013.
[16] T. Stutz and A. Uhl, A survey of H.264 AVC/SVC encryp-tion,
IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 3,pp.
325339, Mar. 2012.
[17] Advanced Video Coding for Generic Audiovisual Services,
ITU, Geneva,Switzerland, Mar. 2005.
[18] J. G. Jiang, Y. Liu, Z. P. Su, G. Zhang, and S. Xing, An
improvedselective encryption for H.264 video based on intra
prediction modescrambling, J. Multimedia, vol. 5, no. 5, pp.
464472, 2010.
[19] I. E. G. Richardson, H.264 and MPEG-4 Video Compression:
VideoCoding for Next Generation Multimedia. Hoboken, NJ, USA:
Wiley,2003.
[20] D. K. Zou and J. A. Bloom, H.264 stream replacement
watermarkingwith CABAC encoding, in Proc. IEEE ICME, Singapore,
Jul. 2010,pp. 117121.
[21] D. W. Xu and R. D. Wang, Watermarking in H.264/AVC
compresseddomain using Exp-Golomb code words mapping, Opt. Eng.,
vol. 50,no. 9, p. 097402, 2011.
[22] D. W. Xu, R. D. Wang, and J. C. Wang, Prediction mode
modulateddata-hiding algorithm for H.264/AVC, J. Real-Time Image
Process.,vol. 7, no. 4, pp. 205214, 2012.
[23] T. Shanableh, Data hiding in MPEG video files using
multivariateregression and flexible macroblock ordering, IEEE
Trans. Inf. ForensicsSecurity, vol. 7, no. 2, pp. 455464, Apr.
2012.
Dawen Xu received the M.S. degree in communica-tion and
information system from Ningbo University,Ningbo, China, and the
Ph.D. degree in computerapplied technology from Tongji University,
Shang-hai, China, in 2005 and 2011, respectively. He is anAssociate
Professor with the School of Electronicsand Information
Engineering, Ningbo University ofTechnology, Ningbo. His current
research interestsinclude digital watermarking and information
hiding,and signal processing in the encrypted domain. Hehas been a
Technical Paper Reviewer for IEEE
conferences, journals, and magazines.
Rangding Wang received the M.S. degree fromNorthwestern
Polytechnical University, Xian,China, in 1987, and the Ph.D. degree
from TongjiUniversity, Shanghai, China, in 2004. He is aProfessor
with the Faculty of Information Scienceand Engineering, Ningbo
University, Ningbo,China. His current research interests
includemultimedia information security, digital speechprocessing,
digital watermarking, steganography,and steganalysis. He is the
author or coauthor ofmore than 120 research papers and two books.
He
holds more than 20 patents.
Yun Q. Shi (M90SM93F05) received the B.S.and M.S. degrees from
Shanghai Jiao Tong Uni-versity, Shanghai, China, and the M.S. and
Ph.D.degrees from the University of Pittsburgh, Pitts-burgh, PA,
USA. He has been a Professor with theDepartment of Electrical and
Computer Engineer-ing, New Jersey Institute of Technology,
Newark,NJ, USA, since 1987. His current research inter-ests include
digital data hiding, steganalysis, foren-sics and information
assurance, and visual signalprocessing and communications. He is
the author or
coauthor of more than 300 research papers, one book, and five
book chapters,and an Editor of more than ten books. He holds 28
U.S. patents.
Dr. Shi received the Innovators Award 2010 from New Jersey
Inventors Hallof Fame for Innovations in Digital Forensics and
Security. His U.S. patent7 457 341 titled System and Method for
Robust Reversible Data Hiding andData Recovery in the Spatial
Domain won the 2010 Thomas Alva EdisonPatent Award from the
Research and Development Council of New Jersey.He served as an
Associate Editor for two IEEE TRANSACTIONS and a fewother
journals.
/ColorImageDict > /JPEG2000ColorACSImageDict >
/JPEG2000ColorImageDict > /AntiAliasGrayImages false
/CropGrayImages true /GrayImageMinResolution 150
/GrayImageMinResolutionPolicy /OK /DownsampleGrayImages true
/GrayImageDownsampleType /Bicubic /GrayImageResolution 600
/GrayImageDepth -1 /GrayImageMinDownsampleDepth 2
/GrayImageDownsampleThreshold 1.50000 /EncodeGrayImages true
/GrayImageFilter /DCTEncode /AutoFilterGrayImages false
/GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict >
/GrayImageDict > /JPEG2000GrayACSImageDict >
/JPEG2000GrayImageDict > /AntiAliasMonoImages false
/CropMonoImages true /MonoImageMinResolution 400
/MonoImageMinResolutionPolicy /OK /DownsampleMonoImages true
/MonoImageDownsampleType /Bicubic /MonoImageResolution 1200
/MonoImageDepth -1 /MonoImageDownsampleThreshold 1.50000
/EncodeMonoImages true /MonoImageFilter /CCITTFaxEncode
/MonoImageDict > /AllowPSXObjects false /CheckCompliance [ /None
] /PDFX1aCheck false /PDFX3Check false /PDFXCompliantPDFOnly false
/PDFXNoTrimBoxError true /PDFXTrimBoxToMediaBoxOffset [ 0.00000
0.00000 0.00000 0.00000 ] /PDFXSetBleedBoxToMediaBox true
/PDFXBleedBoxToTrimBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ]
/PDFXOutputIntentProfile (None) /PDFXOutputConditionIdentifier ()
/PDFXOutputCondition () /PDFXRegistryName () /PDFXTrapped
/False
/Description >>> setdistillerparams>
setpagedevice