Reversible data hiding exploiting spatial correlation ...hklee.kaist.ac.kr/publications/Pattern Recognition(Kim and Lee 2009...Data hiding Histogram modification Reversible data hiding

Pattern Recognition 42 (2009) 3083 -- 3096

Contents lists available at ScienceDirect

Pattern Recognition

journal homepage: www.e lsev ier .com/ locate /pr

Reversible data hiding exploiting spatial correlation between sub-sampled images

Kyung-Su Kima,∗, Min-Jeong Leea, Hae-Yeoun Leeb, Heung-Kyu Leea

aSchool of Electrical Engineering and Computer Science, Division of Computer Science, Korea Advanced Institute of Science and Technology (KAIST), 335 Gwahangno, Yuseong-gu,Daejeon 305-701, South KoreabSchool of Computer and Software Engineering, Kumoh National Institute of Technology, 1, Yanghodong, Gumi, Gyeongbuk, South Korea

A R T I C L E I N F O A B S T R A C T

Article history:Received 1 August 2008Received in revised form 4 March 2009Accepted 4 April 2009

Keywords:Content authenticationData hidingHistogram modificationReversible data hidingSub-samplingWatermarking

Reversible data hiding enables host media to be restored from marked media without any loss of hostinformation. Since this reversibility helps to make right decision during image analysis, it is highlydesired in quality-sensitive imagery where even the minimal distortion introduced by embedding datais unacceptable. In this paper, we propose a reversible data hiding method that modifies the differencehistogram between sub-sampled images. It exploits the high spatial correlation inherent in neighboringpixels to achieve high capacity and imperceptible embedding. On various test images including 16-bitimages, we demonstrate the validity of our proposed method by comparing to other existing reversibledata hiding algorithms. Experimental results support that our method provides high embedding capacitywhile keeping the distortions at a low level.

© 2009 Elsevier Ltd. All rights reserved.

1. Introduction

Data hiding is a technique that embeds secret information calleda mark into host media to provide various purposes such as copy-right protection, broadcast monitoring, authentication, and so on.Although cryptography is another way to protect the digital content,it only protects the content in transit. If the content is decryptedonce, it has no further protection. Moreover, cryptographic tech-niques cannot give sufficient integrity for content authentication.Data hiding technique can be used in a wide variety of applications,each of which has own specific requirements: different payload, per-ceptual transparency, robustness, and security [1–4].

Digital watermarking is a form of data hiding. From the applica-tion point of view, digital watermarking methods can be classifiedinto two categories: robust watermarking and fragile watermarking[1]. Robust watermarking aims at making a watermark robust to allpossible distortions that preserve the value of the contents. On theother hand, fragile watermarking makes watermark invalid even af-ter the slightest modification of the contents, so it is useful to controlcontent integrity and authentication. Reversible data hiding, or so-called lossless data hiding, invertible data hiding, is a kind of fragilewatermarking. For content authentication and tamper proofing, this

∗ Corresponding author. Tel.: +82423505566; fax: +82423508144.E-mail addresses: [email protected] (K.-S. Kim), [email protected]

(M.-J. Lee), [email protected] (H.-Y. Lee), [email protected] (H.-K. Lee).

0031-3203/$ - see front matter © 2009 Elsevier Ltd. All rights reserved.doi:10.1016/j.patcog.2009.04.004

enables exact recovery of the original image from the watermarkedimage after watermark removal [5]. The hash value of the originalcontent, as well as electronic patient records (ERPs) and metadataregarding the content can be represented as watermark. In quality-sensitive applications such as medical imaging, military imaging, lawenforcement, and remote sensing where a slight modification canlead to significant difference in final decision making process, theoriginal image without any modification is required during imageanalysis. Even if the modification is quite small and imperceptibleto human eyes, they do not accept because it may affect the rightdecision and lead to legal problems. In multimedia archives, contentproviders do not want to waste their storage keeping both the orig-inal image and the watermarked one due to cost and maintenanceproblems [6].

In this paper, we propose a reversible data hiding method thatmodifies the difference histogram between sub-sampled images. Thehigh spatial correlation inherent in neighboring pixels is exploited toachieve high capacity and imperceptible embedding. On various testimages including 16-bits images, we validate the proposed methodby comparing to other reversible data hiding algorithms. This pa-per is organized as follows. In Section 2, reversible data hiding algo-rithms are reviewed and analyzed in terms of capacity, visual quality,and complexity. Section 3 presents our reversible data embeddingand extraction algorithm including a solution to the over/underflowproblem. Also, the lower bound of the peak signal-to-noise ratio(PSNR) is evaluated. Experimental results are shown in Sections 4and 5 concludes.

3084 K.-S. Kim et al. / Pattern Recognition 42 (2009) 3083 -- 3096

2. Related works

The performance of a reversible data hiding algorithm is mea-sured in the following aspects: (1) embedding capacity, (2) visualquality, and (3) computational complexity. Reversible data hidingaims at developing a method that increases the embedding capacityas high as possible while keeping the distortion and the computa-tional complexity at a low level. According to embedding strategies,reversible data hiding can be classified into three types in our liter-ature.

Type-I algorithms losslessly compress selected features from animage to obtain enough space, which is then filled up with the mes-sage to be hidden. Fridrich et al. [7] used a JBIG lossless compressionscheme for compressing a proper bit-plane that offered minimumredundancy and embedded image hash by appending it to the com-pressed bit-stream. However, a noisy image may force to embed thehash in higher bit-plane, and hence it caused visual artifacts. Celiket al. [8] used a CALIC lossless compression algorithm and achievedhigh capacity by using generalized least significant bit embedding(G-LSB) technique, but the capacity depended on image structure.

Type-II algorithms are performed in transform domain such asdiscrete cosine transform (DCT) or discrete wavelet transform (DWT)where message bits are embedded into the corresponding coeffi-cients. Yang et al. [9] proposed a reversible data hiding algorithmbased on integer DCT coefficients of image blocks. The capacity andvisual quality were adjusted by selecting different numbers of ACcoefficients in different frequencies. Xuan et al. [10] employed inte-ger wavelet transform. Message bits were embedded into a middlebit-plane of the integer wavelet coefficients in high frequency sub-band. Lee et al. [6] applied integer-to-integer wavelet transform toimage blocks and embedded message bits into the high frequencywavelet coefficients of each block.

Type-III algorithms can be grouped into two categories: differ-ence expansion and histogram modification. A difference expansion(DE) technique was proposed by Tian [11], where an integer Haarwavelet transform was used to obtain high-pass components con-sidered as the differences of pixel pairs. Message bits were embed-ded by expanding these differences. The main advantage was highembedding capacity, but disadvantages were undesirable distortionat low capacities and lack of capacity control due to embedding ofa location map which contained the location information of all se-lected expandable difference values. Alattar developed the DE tech-nique for color images using triplets [12] and quads [13] of adjacentpixels and generalized DE for any integer transform [14]. Kamstraand Heijmans [15] improved the DE technique by employing low-pass components to predict which location will be expandable, sotheir scheme was capable of embedding small capacities at low dis-tortions. To overcome the drawbacks of the DE technique, Thodi andRodriguez [16] presented a histogram-shifting technique to embeda location map for capacity control and suggested a prediction errorexpansion approach utilizing the spatial correlation in the neighbor-hood of a pixel.

Histogram modification techniques use image histogram to hidemessage bits and achieve reversibility. Since most histogram-basedmethods do not apply any transform, all processing is performed inspatial domain, and thus the computational cost is moderately lowerthan type-I and type-II algorithms. Ni et al. [17] utilized a zero pointand a peak point of a given image histogram where the amount ofembedding capacity was a number of pixels with the peak point. Ver-saki et al. [18] also proposed a reversible scheme using a peak pointand zero point. One drawback of these algorithms is that it requiresthe information of the histogram peak point or zero point to recoveran original image. In [19,20], they extended Ni's scheme and appliedlocation map to reverse without the knowledge of the peak pointand zero point. Tsai et al. [23] achieved a higher embedding capac-

ity than the previous histogram-based methods by using a residueimage indicating a difference between a basic pixel and each pixelin a non-overlapping block. However, since they require a peak andzero point information per each block for being reversible, that in-formation should be attached to message bits and that makes actualembedding capacity lower. Lee et al. [21] explored the peak point indifference image histogram and embedded data into locations wherethe values of the difference image were −1 and +1. Lin et al. [22]divided the image into non-overlapping blocks and generated a dif-ference image for block by block. Then, message bits were embed-ded by modifying the difference image of each block after making anempty bin through histogram shifting. Although this technique is ahigh capacity reversible method using a multi-level hiding strategy,it suffers from transmitting peaks information of all blocks. When adegree of hiding level goes up, the amount of overhead informationis considerably high. This is why we should focus on actual embed-ding capacity EC =W −O, where W and O denote whole embeddingcapacity and overhead information, respectively. Lin et al.'s schemehas to embed O more bits together with the embedded message.

In type-I algorithms, the embedding capacity varies according tothe characteristic of the image and the performance highly dependedon the applied lossless compression algorithm. Type-II algorithmsshow satisfactory results, but require converting into transform do-main at additional computational cost. The DE technique in type-IIIalgorithms suffers from capacity control due to the embedding ofthe location map. Although histogram-based methods simply workthrough histogram modification, overhead information should be assmall as possible.

3. Reversible data hiding

This section presents a histogram-based reversible data hidingmethod for images in spatial domain, which satisfies high embedding

y

x

image

P1 P2 P3

P4 P5 P6

P7 P8 P9

pixels

M

N

S1

P1

M/2

N/2

P3

P7 P9

S2

P2

P8

S3

P4 P6

S4

P5

P1

S1

P2

S2

P3

S3

P4

S4

P5

S5

P6

S6

P7

S7

P8

S8

P9

S9

N/3

M/3

Fig. 1. Sub-sampling examples at different sampling factors: (a) original; (b)u = v = 2; and (c) u = v = 3.

K.-S. Kim et al. / Pattern Recognition 42 (2009) 3083 -- 3096 3085

Original image I

Sub-sampling

Sub-sampled images

S1 S2

S3S4

Determine

reference subimage

S1

Create difference

histogram

H2H1 H3

Shift difference

histogram

Modify difference

histogram

Message W

H'S2H'S1

H'S3

Marked image Iw

HS

600050004000300020001000

0

600050004000300020001000

0

600050004000300020001000

0

600050004000300020001000

0-100 -50 0 50 100 -100 -50 0 50 100

-100 -50 0 50 100

-100 -50 0 50 100 -100 -50 0 50 100 -100 -50 0 50 100

600050004000300020001000

0

600050004000300020001000

0

Fig. 2. Overall data embedding procedure.

capacity, high visual quality, and low computational complexity.After sub-sampling is explained, embedding and extraction schemesare presented. Then, the way to prevent over/underflow is explained.

3.1. Sub-sampling

Sampling is the process of selecting units (e.g., pixels, coefficients)from an image. Suppose that an image of size N×M pixels is denotedby I(x, y), where x = 0, . . . ,M − 1 and y = 0, . . . ,N − 1. Two samplingfactors, u and v set the desired sub-sampling intervals in a rowand column direction, respectively. As illustrated in Fig. 1, the 2-Dimage is sampled at uniform intervals. This process is called as sub-

sampling and each sub-sampled image Sk of size N/u × M/v isobtained as follow:

Sk(i, j) = I(i · v + floor

(k − 1u

), j · u + ((k − 1)modu)

)(1)

Where i=0, . . . ,M/v−1, j=0, . . . ,N/u−1, and k=1, . . . ,u×v. IfN/u or M/v is not an integer value, we slightly modify the size ofall sub-sampled images by flooring. For example, when M=N=512and u = v = 3, the size of the sub-sampled image is set at thenearest integer less than or equal to the number: 512/3 = 170in both width and height. The residue of pixels is neither includedin any sub-sampled images nor targeted embeddable components

3086 K.-S. Kim et al. / Pattern Recognition 42 (2009) 3083 -- 3096

by our data hiding scheme. The basic idea is to utilize horizontal,vertical, and diagonal neighbors of a pixel since they have stronglyspatial correlation and high pixel redundancy.

3.2. Embedding algorithm

Let us assume that the embedded message is a pseudo randombinary sequence and a grayscale image is employed. The messageis embedded by modifying the difference histogram between sub-sampled versions of an image. Fig. 2 describes an overall data em-bedding procedure, which is composed of six steps as follows.

Step 1: Generate sub-sampled versions Sk in Eq. (1) by performingsub-sampling from an original image I using two sampling factors(u,v).

Step 2: Determine a reference sub-sampled image SRef to maxi-mize spatial correlation between the sub-sampled images. We selectSRef from the following Eq. (2). For example, when u = v = 3, S5is determined as the reference one. It is defined as

SRef =(Round

(u2

− 1))

× v + Round(

v2

)(2)

Step 3: Create difference images between the reference SRef andthe other destination sub-sampled images denoted by SDes.

DRef−Des(k1, k2) = SRef (k1, k2) − SDes(k1, k2) (3)

where 0k1M/v − 1, 0k2N/u − 1.Step 4: Prepare empty bins in each histogram H of the difference

images according to an embedding level L, where H= −255, . . . , 255.Depending on the desired degree, L affects the performance ofcapacity and the perceptual quality. In order to achieve this, thenegative differences and the non-negative differences in the outerregions of a selected embedding range should be shifted left andright, respectively. When shifting H, only the pixels in the desti-nation sub-sampled image are modified. The real embedding pro-cedure will use the range [−L, L] in this shifted histogram Hs. Theshifted histogram Hs can be calculated as follows:

Hs =H + L + 1 if HL + 1H − L − 1 if H − L − 1

(4)

Also, this can be obtained by

D′Ref−Des(k1, k2) = SRef (k1, k2) − S′

Des(k1, k2) (5)

where

S′Des(k1, k2) =

SDes(k1, k2) − (L + 1) if HL + 1SDes(k1, k2) + (L + 1) if H − L − 1

(6)

Step 5: Embedmessagew(n) by modifying Hs, wherew(n) ∈ 0, 1.The modified difference image D′ is scanned. Once a pixel with thedifference value of −L or +L is encountered, we check the message tobe embedded. This process is repeated until there are no pixels withthe difference value of ±L. Then the embedding level L decreasesby 1. These scanning and embedding steps are executed until L<0.Likewise, we only modify the pixels in the destination sub-sampledimage. The message embedding can be formulated as follows:

D′′Ref−Des(k1, k2) = SRef (k1, k2) − S′′

Des(k1, k2) (7)

where

S′′Des(k1, k2) =

⎧⎪⎪⎨⎪⎪⎩S′Des(k1, k2) + (L + 1) if D′ = −L,w(n) = 1S′Des(k1, k2) − (L + 1) if D′ = L,w(n) = 1S′Des(k1, k2) + L if D′ = −L,w(n) = 0S′Des(k1, k2) − L if D′ = L,w(n) = 0

(8)

0-6

1

0

1

0

1

0

10

10

-5 -4 -3 -2 -1 1 2 3 4 5 6

0-6 -5 -4 -3 -2 -1 1 2 3 4 5 6

0-6 -5 -4 -3 -2 -1 1 2 3 4 5 6

0-6 -5 -4 -3 -2 -1 1 2 3 4 5 6

0-6 -5 -4 -3 -2 -1 1 2 3 4 5 6

Fig. 3. Histogram modification at the embedding procedure when the embeddinglevel L is 2. At the last step (e), the bin (−1) has no occurrence: (a) original differencehistogram H; (b) histogram shifting; (c) message embedding (L = 2); (d) messageembedding (L = 1); and (e) message embedding (L = 0).

for L>0,

S′′Des(k1, k2) =

S′Des(k1, k2) − 1 if D′ = 0,w(n) = 1S′Des(k1, k2) if D′ = 0,w(n) = 0

(9)

for L = 0.

K.-S. Kim et al. / Pattern Recognition 42 (2009) 3083 -- 3096 3087

600050004000300020001000

0-100 -50 0 50 100

600050004000300020001000

0-100 -50 0 50 100

600050004000300020001000

0-100 -50 0 50 100

600050004000300020001000

0-100 -50 0 50 100

600050004000300020001000

0-100 -50 0 50 100

600050004000300020001000

0-100 -50 0 50 100

Marked Image Iw

Sub-sampling

Sub-sampling images

S1 Sw2

Sw3 Sw4

Determinereference subimage

S1

Create differencehistogram

H’S1 H’S2 H’S3

Extract image MHS

Shift differencehistrogram back H1 H2

H3

Recovered image I

Fig. 4. Overall data extraction and recovery procedure.

Step 6: Finally, obtain the marked image Iw through the inverse ofthe sub-sampling with the unmodified reference sub-sampled imageSRef and the modified destination sub-sampled images S′′

Des.Fig. 3 depicts how the histogram is modified by the proposed

embedding steps. As a result of embedding message, the bin H′s =

−1 becomes empty. Making such an empty bin can be used as theevidence that the marked image to be authentic has been altered ornot before extracting the hidden message at decoder side.

The presented method requires transmitting some overhead in-formation (u,v, L) so that the decoder recovers the marked imageto its original. We simply embed the overhead bits into the least bitplane (LSB) of the selected pixels using the secret key. Those pix-els holding the overhead bits do not participate in the embeddingprocess. Also, the secret key that generates the pseudo random se-quence should be shared. Since our method is based on the private

key scheme, only an authorized party with the same key can decodethe hidden information and recover the marked image to its originalone.

3.3. Extraction and recovery algorithm

Fig. 4 describes an overall procedure to extract the message andrecover the original image from the marked image. Before extractingthe hidden message, the basic authentication step verifies whetherthe received marked image has been altered or not. Since the pre-sented method embeds a fragile mark, the marked image is regardedas inauthentic even if a very small change in pixel values occurred.

Remind that the H′s = −1 bin has no occurrence. If there is more

than one occurrence at H′s = −1, we decide that the marked image

3088 K.-S. Kim et al. / Pattern Recognition 42 (2009) 3083 -- 3096

Iw is tampered and stop further steps. This prompt decision beforerestoring the original image advantages time-critical and low-powerapplications. If authentic, the original image is completely recoveredfrom the marked image after the hidden message is extracted. Thedetailed extraction and recovery steps are as follows:

Step 1: Obtain two sampling factors (u,v) and the embeddinglevel L from the LSB of the selected pixels using the secret key.

Step 2: Generate sub-sampled versions in Eq. (1) by performingsub-sampling from the marked image Iw using the sampling factorsin Step 1.

Step 3: Determine a reference sub-sampled image SRef by Eq. (2).Step 4: Create difference images between the unmarked refer-

ence sub-sampled image SRef and the other marked destination sub-sampled images denoted by Sw.

DRef−w(k1, k2) = SRef (k1, k2) − Sw(k1, k2) (10)

where 0k1M/v − 1, 0k2N/u − 1, and w = 1, . . . ,u ×v, wRef .

Step 5: Extract the hidden message w(n) from each differenceimage. The extraction process is the inverse of embedding process.After a new variable L′ is set to 0, the difference image D is scanned.Once a pixel with the difference value of 1 is encountered, bit 1 isretrieved. If the pixel with the difference value of 0 is encountered,bit 0 is retrieved. This process is repeated until there are no pixelswith the difference value of 0 and 1. After, L′ is increased by 1, thedifference image is scanned again, and themessage is extracted usingthe following rule:

w(n) =0 if D = 2L′ or − 2L′

1 if D = 2L′ + 1 or − 2L′ − 1(11)

This scanning and extracting process is executed until L′ >L.Step 6: Remove the hidden message w(n) from the difference

images.

D′Ref−w(k1, k2) = SRef (k1, k2) − S′

w(k1, k2) (12)

where

S′w(k1, k2) =

Sw(k1, k2) + 1 if D(k1, k2) = 1Sw(k1, k2) otherwise

(13)

for L′ = 0,

S′w(k1, k2) =

⎧⎪⎪⎨⎪⎪⎩Sw(k1, k2) + L′ if D(k1, k2) = 2L′

Sw(k1, k2) + (L′ + 1) if D(k1, k2) = 2L′ + 1Sw(k1, k2) − L′ if D(k1, k2) = −2L′

Sw(k1, k2) − (L′ + 1) if D(k1, k2) = −2L′ − 1

(14)

for 1L′ L.Step 7: Shift each histogram Hs of the difference image back to

obtain its original difference histogram H as follows:

H =Hs − L − 1 if Hs2L + 2Hs + L + 1 if Hs − 2L − 2

(15)

This can be obtained by

D′′Ref−w(k1, k2) = SRef (k1, k2) − S′′

w(k1, k2) (16)

where

S′′w(k1, k2) =

S′w(k1, k2) + (L + 1) if Hs2L + 2S′w(k1, k2) − (L + 1) if Hs − 2L − 2

(17)

Step 8: Finally, obtain the recovered original image I through theinverse of the sub-sampling with the sub-sampled images.

In this way, the original image can be completely recovered fromthe marked image. Fig. 5 shows how the histogram is restored by theproposed extraction and recovery steps. The decisive authentication

0-6 -5 -4 -3 -2 -1 1 2 3 4 5 6

0-6 -5 -4 -3 -2 -1 1 2 3 4 5 6

0-6 -5 -4 -3 -2 -1 1 2 3 4 5 6

0-6 -5 -4 -3 -2 -1 1 2 3 4 5 6

0-6 -5 -4 -3 -2 -1 1 2 3 4 5 6

Fig. 5. Histogram modification at the extraction and recovery procedure whenthe embedding level L is 2: (a) marked difference histogram H′

s; (b) histogrammodification (L′ =0); (c) histogram modification (L′ =1); (d) histogram modification(L′ = 2); and (e) histogram shifting back to its original.

can be simply implemented in such a way that the embedded mes-sage represents the hash of the original one. If the extracted hashand the hash of the recovered image match bit by bit, the markedimage is authentic.

K.-S. Kim et al. / Pattern Recognition 42 (2009) 3083 -- 3096 3089

Fig. 6. 8-Bit 512×512 images: (a) Lena, (b) Baboon, (c) Boat, (d) Airplane, (e) Aerial, (f) Tank, and (g) Trucks. 16-bit 256 × 256 images: (h) MRI 1 and (i) MRI 2.

3.4. Preventing over/underflow

When the message is embedded into the difference histogramby modifying the destination sub-sampled images, overflow or un-derflow can occur in the pixel domain. For an 8-bit gray image, itmeans that the modified grayscale value exceeds its upper bound255 (overflow) or is below lower bound 0 (underflow). To preventthis over/underflow problem, there have been solutions such as us-ing modular-256 addition or using a location map holding the pixellocation where the problem occurred. Although modular-256 ad-dition can avoid over/underflow, the marked image suffered fromsalt-and-pepper visual artifacts when a grayscale value close to 255was flipped to 0 or vice versa. Using a location map seems to be amore effective method because it locates which pixels were flippedto 255–0 or 0–255. However, the location map has to be transmit-ted to the decoder as overhead information or appended it to theembedded bits.

We adopt another modular addition with a cycle to min-imize the visual distortion due to pixel flipping [7]. For ex-ample, we used cycles of length 64 rather than one cycleof length 256: 01 · · ·630, 6465 · · ·12764,128129 · · ·191128, 192193 · · ·255192. Amongseveral intervals, only the intervals of both sides are employed:01 · · ·630 for underflow, 192193 · · ·255192 foroverflow. According to the characteristic of the difference image,the cycle is adaptively determined. The cycle is set to 2 for Lena im-age because the maximum difference value between sub-sampledimages is |123|. The invertible modular addition and subtraction

with the cycle are defined as follows:

SDes(k1, k2) ⊕ i = C⌊iC

⌋+ mod(SDes(k1, k2) + i,C)

SDes(k1, k2) − i = SDes(k1, k2) ⊕ (−i)

= C⌊−iC

⌋+ mod(SDes(k1, k2) + (−i),C) (18)

where C is a cycle. Overflow and underflow during embedding pro-cess is generalized as follows:

Overflow =⎧⎨⎩SDes(k1, k2) + (L + 1)>255 for histogram shiftingS′Des(k1, k2) + (L + 1)>255 for message embedding, w(n) = 1S′Des(k1, k2) + L>255 for message embedding, w(n) = 0

Undeflow =⎧⎨⎩SDes(k1, k2) − (L + 1)<0 for histogram shiftingS′Des(k1, k2) − (L + 1)<0 for message embedding, w(n) = 1S′Des(k1, k2) − L<0 for message embedding, w(n) = 0

(19)

By Eq. (19), when the current variables L = 2, SRef (k1, k2) = 251,S′Des(k1, k2)=253, and to-be-embedded bit w(n)=1 are set, this caseis determined as overflow (253+2+1 = 256> 255) and the modularaddition is adopted by Eq. (18).

To achieve the reversibility in the decoder side, the pixels derivedby modular addition or modular subtraction must be distinguishedfrom common addition or subtraction. For practical situations, weassume that no tremendous variation exists between SRef and Swby considering the characteristics of natural images. If there is asignificant difference between SRef and Sw during the extraction andrecovery algorithm, the pixel value in Sw is recognized as overflow

3090 K.-S. Kim et al. / Pattern Recognition 42 (2009) 3083 -- 3096

S1 S2 S3 S40.23730.23730.23730.23740.23740.23740.23740.23740.23740.23740.2374

Sub−sampled image as a reference

Cap

acity

(bp

p)

S1 S2 S3 S440.91

40.911

40.912

40.913

40.914

40.915

40.916

40.917

40.918

40.919

40.92

Sub−sampled image as a reference

PSN

R (

dB)

S1 S2 S3 S4 S5 S6 S7 S8 S90.24

0.245

0.25

0.255

0.26

0.265

0.27

0.275

0.28

Sub−sampled image as a reference

Cap

acity

(bp

p)

S1 S2 S3 S4 S5 S6 S7 S8 S940.04

40.06

40.08

40.1

40.12

40.14

40.16

40.18

40.2

Sub−sampled image as a reference

PSN

R (

dB)

S1 S2 S3 S4 S5 S6 S7 S8 S9 S10

S11

S12

S13

S14

S15

S16

S17

S18

S19

S20

S21

S22

S23

S24

S25 S1 S2 S3 S4 S5 S6 S7 S8 S9 S10

S11

S12

S13

S14

S15

S16

S17

S18

S19

S20

S21

S22

S23

S24

S25

0.215

0.22

0.225

0.23

0.235

0.24

0.245

0.25

0.255

0.26

Sub−sampled image as a reference

Cap

acity

(bp

p)

39.54

39.56

39.58

39.6

39.62

39.64

39.66

39.68

39.7

Sub−sampled image as a reference

PSN

R (

dB)

Fig. 7. Capacity and quality plots for the selected reference sub-sampled image. S1 (u = v = 2), S5 (u = v = 3), and S13 (u = v = 5) chosen by Eq. (2) show bestperformance: (a) capacity (u=v=2); (b) quality (u=v=2); (c) capacity (u=v=3); (d) quality (u=v=3); (e) capacity (u=v=5); and (f) quality (u=v=5).

or underflow and then reversed by modular operation. It is given by

S′w(k1, k2) =

S′w(k1, k2) ⊕ −(L′ + 1) or (−L′) if |SRef − S′

w|TH1S′w(k1, k2) otherwise

S′w(k1, k2) =

S′w(k1, k2) ⊕ (L′ + 1) or (L′) if |SRef − S′

w|TH2S′w(k1, k2) otherwise

(20)

for removing the message represented as Step 6 in Section 3.3,

S′′w(k1, k2) =

S′′w(k1, k2) ⊕ −(L′ + 1) if |SRef − S′′

w|TH3S′′w(k1, k2) otherwise

S′′w(k1, k2) =

S′′w(k1, k2) ⊕ (L′ + 1) if |SRef − S′

w|TH4S′′w(k1, k2) otherwise

(21)

for shifting the histogram back represented as Step 7 in Section 3.3.

3.5. Low bounds of PSNR

PSNR is a well-known quantitative value to measure the distor-tion between original images and marked images. In the presentedmethod, the following equations represent theoretical low boundsof MSE and PSNR values for different values of u, v, and L:

MSE = (L + 1)2 ×(

u · v − 1u · v

)(22)

PSNR = 10 × log10

(2552

MSE

)(23)

Assume that the embedding level L is 0, two sampling factors uand v are both 2, and there is no pixel with over/underflow duringthe embedding. It is clear that the pixels in the difference images

K.-S. Kim et al. / Pattern Recognition 42 (2009) 3083 -- 3096 3091

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.530

32

34

36

38

40

42

44

46

48

50

Embedding capacity (bpp)

Imag

e qu

ality

(dB

)

LenaBaboonBoatAirplaneAerialTankTrucksMRI 1MRI 2

Fig. 8. Comparison of embedding capacity (bpp) versus distortion (dB) for test images.

are either added or subtracted by 1 except for the reference sub-sampled image. In the worst case, the grayscale values of all pixelsin the destination sub-sampled images are either incremented ordecremented by 1. In this case, the resultant MSE is 0.75 and thus thetheoretical lower bound of the PSNR of the marked image is 49.38dB.

4. Experimental results

In terms of embedding capacity and distortion, the performanceof our algorithm was measured by comparing with other reversibledata hiding schemes. In all experiments, seven grayscale imagesof 512×512 pixels from the USC-SIPI image database [24] and two16-bit medical images of 256×256 pixels were tested as depictedin Fig. 6. The message bits to be embedded were randomly gener-ated using the MATLAB function. Embedding variables u, v, and Lwere adjusted [1, 8], [1, 8], and [0, 9], respectively.

4.1. Embedding capacity and distortion performance

The capacity (bit per pixel) measures the amount of data thatcan be hidden. In order to achieve high embedding capacity, weutilize the fact that the difference values with small magnitudesbetween sub-sampled images occur more frequently. In general, alarge number of pixel values of the difference image have a tendencyto be distributed around 0. The actual embedding capacity EC of theproposed algorithm depends on how many the difference imagesare used and how many pixels having the difference values between−L and +L in each difference image exist. In addition, two samplingfactors and the embedding level affect EC. As discussed in Section 2,EC is calculated by

EC = W − O (24)

where W denotes the whole embedding capacity and O denotes theamount of data used to represent the overhead information. In ourscheme, O would be maximum 12 bits (8 bits for u and v, 4 bitsfor L).

Fig. 7 analyzes that capacity versus distortion performance wheneach sub-sampled image is determined as a reference one in Section3.1. When S1 (u= v= 2), S5 (u= v= 3), and S13 (u= v= 5)were chosen by Eq. (2), the performance was satisfactory amongsub-sampled images in terms of capacity and quality. It is becausepixel redundancy and spatial correlation between the determinedreference sub-sampled image and the other ones are high at theselected sampling factors.

Fig. 8 shows the quality of the marked images at various em-bedding capacities up to 0.5 bpp when the sampling factors u andv are set to 3. From this result, the performance of capacity ver-sus distortion depends on the characteristics of the images. Someimages, especially Airplane and Boat images, containing more low-frequency components than middle and high-frequency componentsachieved high embedding capacity while keeping the PSNR valuelow. For instance, the capacity was 84365 bits (0.32bpp) with PSNRof 43.7dB for Airplane image, whereas the capacity was 19383 bits(0.07bpp) with PSNR of 42.9dB for Baboon image. The presentedmethod achieved the capacity from 6k to 210k and the PSNR valuefrom 50 to 30.27dB for all test images. Especially, 16-bit medical im-ages had large gradient values in the figure because most differencevalues were centered on zero and thus a high embedding capacitywith a same visual quality as compared to that of other images wasachieved. Figs. 9–11 show the original images and the marked im-ages at various embedding capacities. As shown in the figures, thevisual quality of the marked images is satisfactory at moderate em-bedding capacity.

Fig. 12 plots the embedding capacity with the different samplingfactors of (u,v): (2,2), (3,3), (5,5), and (8,8). The performance ofour algorithm was slightly degraded when the factor values weretoo large (8,8). This is due to the fact that the larger the samplingfactors are, the weaker the spatial correlation between SRef and SDesis. For example, for Lena image, the sampling factor (3,3) achieved0.52bpp at the level 5, whereas the sampling factor (8,8) achieved0.47bpp at the same level although it utilized more sub-sampledimages than those of the sampling factor (3,3). In our analysis, the

3092 K.-S. Kim et al. / Pattern Recognition 42 (2009) 3083 -- 3096

Fig. 9. Original and marked images: Lena, Baboon, and Boat images: (a) original Lena; (b) 33.5dB with 0.73bpp; (c) original Baboon; (d) 30.3dB with 0.39bpp; (e) originalBoat; and (f) 33dB with 0.67bpp.

sampling factor (3,3) provided the largest embedding capacity whilekeeping the distortions at a lowest level.

4.2. Performance comparisons with other algorithms

Tables 1 and 2 summarized comparison results with otherhistogram-shifting based algorithms [17–23] for four test images:

Lena, Baboon, Boat, and Airplane. Although the algorithms listed inthe tables resulted in the similar PSNR value about 48dB, the pay-load of our algorithm outperformed that of other algorithms, whichwas achieved about from 20% to 300% performance enhancementfor the test images. Although Tsai et al.'s scheme employing thenegative histogram and non-negative histogram of the residue im-age provided capacity enhancement compared to image histogram

K.-S. Kim et al. / Pattern Recognition 42 (2009) 3083 -- 3096 3093

Fig. 10. Original and marked images: Airplane, Aerial, and Tank images: (a) original Airplane; (b) 34.2dB with 0.74bpp; (c) original Aerial; (d) 31.8dB with 0.56bpp;(e) original Tank; and (f) 32.1dB with 0.66bpp.

Fig. 11. Original and marked images: Trucks, MRI 1, and MRI 2 images: (a) original Trucks; (b) 31dB with 0.51bpp; (c) original MRI 1; (d) 36.44dB with 0.35bpp;(e) original MRI 2; and (f) 35.97dB with 0.25bpp.

3094 K.-S. Kim et al. / Pattern Recognition 42 (2009) 3083 -- 3096

0 1 2 3 4 5 6 7 8 90

0.1

0.2

0.3

0.4

0.5

0.6

Embedding level L

Em

bedd

ing

capa

city

(bp

p)

0 1 2 3 4 5 6 7 8 90

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Em

bedd

ing

capa

city

(bp

p)

0 1 2 3 4 5 6 7 8 90

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Em

bedd

ing

capa

city

(bp

p)

0 1 2 3 4 5 6 7 8 90

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Em

bedd

ing

capa

city

(bp

p)LenaBaboonBoatAirplaneAerialTankTrucksMRI1 MRI2

Embedding level L

Embedding level L Embedding level L

LenaBaboonBoatAirplaneAerialTankTrucksMRI1 MRI2

LenaBaboonBoatAirplaneAerialTankTrucksMRI1 MRI2

LenaBaboonBoatAirplaneAerialTankTrucksMRI1 MRI2

Fig. 12. Comparison of embedding capacity in bpp versus the embedding level L with various sampling factors (u,v). (a) (2,2); (b) (3,3); (c) (5,5); and (d) (8,8).

Table 1Comparison results in terms of the payload (bits) and the PSNR (dB) for Lena andBaboon.

Lena image Baboon image

Payload PSNR Payload PSNR

Ni et al. [17] 5460 48.2 5421 48.2Varsaki et al. [18] 5460 48.2 5421 48.2Hwang et al. [19] 5408 48.2 5208 48.2Kuo et al. [20] 5418 48.2 5352 48.2Tsai et al. [23] 13459 49.3 Not applicable 48.1Proposed 20121 48.9 6499 48.7

In the proposed algorithm, u, v, and L were set to 3, 3, and 0, respectively.In [23], the block size was set to 4×4. “Not applicable” means EC has the mi-nus value due to the lack of embedding peak information into the correspondingimage.

Table 2Comparison results in terms of the payload (bits) and the PSNR (dB) for Boat andAirplane.

Boat image Airplane image

Payload PSNR Payload PSNR

Ni et al. [17] 5394 48.2 16171 48.2Varsaki et al. [18] 5394 48.2 16171 48.2Hwang et al. [19] 5289 48.2 16119 48.2Kuo et al. [20] 5342 48.2 16125 48.2Tsai et al. [23] 13778 49.2 34171 50.2Proposed 21442 48.9 32631 49

In the proposed algorithm, u, v, and L were set to 3, 3, and 0, respectively. In[23], the block size was set to 4×4.

based methods, it suffers from the lack of embedding the pairs ofpeak and zero points for being reversible, especially in an imagecontaining high frequency components like Baboon, thus it is notapplicable to such an image.

In terms of actual embedding capacity (bpp) and image quality(dB), the presented algorithm was compared with RS scheme [7],G-LSB scheme [8], DE scheme [11], Ni et al.'s scheme [17], and Lin etal.'s scheme [22] for the Lena, Baboon, Boat, and Airplane as shown inFig. 13. The embedding capacity means ECwhere the amount of over-head information is subtracted (e.g., peak values for blocks, an orig-inal peak and a zero point in the image histogram, etc.). DE schemeand G-LSB scheme showed relatively high embedding capacity ver-sus the allowable PSNR value, whereas the RS scheme had low em-bedding capacity compared to others. The histogram-based Ni et al.'sscheme showed the fixed PSNR quality, 48.2dB, but the achievablecapacity was a little varied for each image. Note that the embeddingperformance of the Lin et al.'s scheme did not achieve more than0.3bpp for the test images, because the amount of the peak infor-mation for all blocks exceeded the whole embedding capacity whenthe so-called hiding level was much higher up. In other words, itsuffered from the lack of capacity control due to the need for embed-ding all peaks information for blocks. It is important for applicationsto meet the requirement for real-time delivery and achieving syn-chronization, since lightweight transmission protocols are applied tomultimedia communication where the image has to be transmittedthrough the limited network bandwidth. As a result, the extra in-formation should be reduced as much as possible for general appli-cability. Experimental results support that the presented algorithmachieves high embedding capacity with low distortions and outper-forms other reversible data hiding schemes.

K.-S. Kim et al. / Pattern Recognition 42 (2009) 3083 -- 3096 3095

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.820

25

30

35

40

45

50

Embedding capacity (bpp)

Imag

e qu

ality

(dB

)

Proposed schemeDE schemeG−LSB schemeRS schemeNi et al.’s schemeLin et al.’s scheme

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.420

25

30

35

40

45

50

Embedding capacity (bpp)

Imag

e qu

ality

(dB

)

Proposed schemeDE schemeG−LSB schemeRS schemeNi et al.’s scheme

0 0.1 0.2 0.3 0.4 0.5 0.6 0.720

25

30

35

40

45

50

Embedding capacity (bpp)

Imag

e qu

ality

(dB

)

Proposed schemeDE schemeG−LSB schemeNi et al.’s schemeLin et al.’s scheme

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.820

25

30

35

40

45

50

Embedding capacity (bpp)

Imag

e qu

ality

(dB

)

Proposed schemeDE schemeG−LSB schemeNi et al.’s schemeLin et al.’s scheme

Fig. 13. Comparison of actual embedding capacity EC in bpp versus distortion with other reversible schemes: RS scheme [7], G-LSB scheme [8], DE scheme [11], Ni et al.'sscheme [17], and Lin et al.'s scheme [22]: (a) Lena; (b) Baboon; (c) Boat; and (d) Airplane.

5. Conclusion and discussion

A simple and efficient reversible data hiding algorithm was pre-sented, where the difference histogram between sub-sampled im-ages was modified to embed the message. We exploited the fact thatthe difference values having small magnitudes occurred frequentlybecause of the high spatial correlation between sub-sampled images.Under the given embedding level, the proposed algorithm shiftedthe difference histogram and then embedded the message by mod-ifying pixel values. Especially, overflow and underflow were pre-vented during embedding and no overhead information during theretrieval was required. Experimental results supported that our al-gorithm achieved higher embedding capacity than other reversibleschemes while maintaining the distortion at a low level. The pre-sented algorithm can be deployed for applications in the areas of im-age authentication, tamper proofing, medical and military imaging.This algorithm has the following advantages: (1) simple and effec-tive, (2) avoidable salt-and-pepper noise, (3) applicable to commonimages as well as medical and military images, (4) adjustable em-bedding capacity ranges from 6k to 210k according to the require-ment, and (5) usable in applications that need a fragile watermarkto identify any change of the image. The performance of the pre-sented algorithm can be enhanced by deciding optimum samplingfactors considering the characteristics of a given image. For exam-ple, u should make larger than v for the given image containinga large amount of horizontal redundancy and less than v for theone containing a large amount of vertical redundancy. A limitationof the presented algorithm is that it is hard to achieve more than1bpp because we only perform the first round of embedding, but thecapacity of 1bpp is large enough for authentication purpose. Future

work is to achieve multi round embedding by considering capacity-distortion performance.

Acknowledgments

This work was supported by the Korea Science and EngineeringFoundation (KOSEF) grant funded by the Korea government (MEST)(no. R0A-2007-000-20023-0).

References

[1] I.J. Cox, M.L. Millter, J.A. Bloom, J. Fridrich, T. Kalker, Digital Watermarking andSteganography, second ed., Morgan Kaufmann, Burlington, USA, 2008.

[2] H.T. Sencar, M. Ramkumar, A.N. Akansu, Data Hiding Fundamentals andApplications, Elsevier Academic Press, London, UK, 2004.

[3] M. Wu, B. Liu, Multimedia Data Hiding, Springer, New York, USA, 2003.[4] M. Awrangjeb, An overview of reversible data hiding, in: Proceedings of

the Sixth International Conference on Computer and Information Technology,Jahangirnagar University, Bangladesh, December 2003, pp. 75–79.

[5] F. Mintzer, J. Lotspiech, N. Morimoto, Safeguarding digital library contents andusers: digital watermarking, D-Lib Magazine, December 1997.

[6] S. Lee, C.D. Yoo, T. Kalker, Reversible image watermarking based on integer-to-integer wavelet transform, IEEE Transactions on Information Forensics andSecurity 2 (3) (2007) 321–330.

[7] J. Fridrich, J. Goljan, R. Du, Invertible authentication, in: Proceedings of theSPIE, Security and Watermarking of Multimedia Contents, vol. 4314, San Jose,CA, January 2001, pp. 197–208.

[8] M.U. Celik, G. Sharma, A.M. Tekalp, E. Saber, Lossless generalized-LSB dataembedding, IEEE Transactions on Image Processing 14 (2) (2005) 253–266.

[9] B. Yang, M. Schmucker, C.B.W. Funk, S. Sun, Integer DCT-based reversiblewatermarking for images using compounding technique, in: Proceedings of theSPIE, Security, Steganography, and Watermarking of Multimedia Contents, vol.5306, San Jose, CA, January 2004, pp. 405–415.

[10] G. Xuan, Y.Q. Shi, Q. Yao, Z. Ni, C. Yang, J. Gao, P. Chai, Lossless data hidingusing histogram shifting method based on integer wavelets, InternationalWorkshop on Digital Watermarking, Lecture Notes in Computer Science, vol.4283, Springer, Jeju Island, Korea, 2006, pp. 323–332.

3096 K.-S. Kim et al. / Pattern Recognition 42 (2009) 3083 -- 3096

[11] J. Tian, Reversible data embedding using a difference expansion, IEEETransactions on Circuits and Systems for Video Technology 13 (8) (2003)890–896.

[12] A.M. Alattar, Reversible watermark using difference expansion of triplets, in:Proceedings of the IEEE International Conference on Image Processing, vol. 1,Barcelona, Spain, September 2003, pp. 501–504.

[13] A.M. Alattar, Reversible watermark using difference expansion of quads, in:Proceedings of the IEEE International Conference on Acoustics, Speech, andSignal Processing, vol. 3, Montreal, Canada, May 2004, pp. 377–380.

[14] A.M. Alattar, Reversible watermark using the difference expansion of ageneralized integer transform, IEEE Transactions on Image Processing 13 (8)(2004) 1147–1156.

[15] L. Kamstra, H.J.A.M. Heijmans, Reversible data embedding into images usingwavelet techniques and sorting, IEEE Transactions on Image Processing 14 (12)(2005) 2082–2090.

[16] D.M. Thodi, J.J. Rodriguez, Expansion embedding techniques for reversiblewatermarking, IEEE Transactions on Image Processing 16 (3) (2007) 721–730.

[17] Z. Ni, Y.Q. Shi, N. Ansari, W. Su, Reversible data hiding, IEEE Transactions onCircuits and Systems for Video Technology 16 (3) (2006) 354–362.

[18] E. Varsaki, V. Fotopoulos, A.N. Skodras, A reversible data hiding techniqueembedding in the image histogram, Technical Report HOU-CS-TR-2006-08-GR,Hellenic Open University, 2006.

[19] J. Hwang, J.W. Kim, J.U. Choi, A reversible watermarking based on histogramshifting, International Workshop on Digital Watermarking, Lecture Notesin Computer Science, vol. 4283, Springer-Verlag, Jeju Island, Korea, 2006,pp. 348–361.

[20] W.-C. Kuo, D.-J. Jiang, Y.-C. Huang, Reversible data hiding based onhistogram, International Conference on Intelligent Computing, Lecture Notesin Artificial Intelligence, vol. 4682, Springer-Verlag, Qing Dao, China, 2007,pp. 1152–1161.

[21] S.K. Lee, Y.H. Suh, Y.S. Ho, Lossless data hiding based on histogram modificationof difference images, Pacific Rim Conference on Multimedia, Lecture Notes inComputer Science, vol. 3333, Springer, Tokyo, Japan, 2004, pp. 340–347.

[22] C.-C. Lin, W.-L. Tai, C.-C. Chang, Multilevel reversible data hiding based onhistogram modification of difference images, Pattern Recognition 41 (12) (2008)3582–3591.

[23] P. Tsai, Y.-C. Hu, H.-L. Yeh, Reversible image hiding scheme usingpredictive coding and histogram shifting, Signal Processing (2009), doi:10.1016/j.sigpro.2008.12.017.

[24] Usc-sipi image database-miscellaneous (online), Available: 〈http://sipi.usc.edu/database/database.cgi?volume = misc〉.

About the Author—KYUNG-SU KIM received the B.S. degree in Computer Engineering from Inha University, Republic of Korea, in 2005, and the M.S. degree in ComputerScience from Korea Advanced Institute of Science and Technology (KAIST), Republic of Korea, in 2007. He is currently working toward his Ph.D. degree in MultimediaComputing Laboratory, Department of EECS, KAIST. His research interests include image/video watermarking and fingerprinting, error concealment method for wirelesschannel, information security, multimedia signal processing, and multimedia communications.

About the Author—MIN-JEONG LEE received the B.S. degree in Computer Engineering from Kyungpook National University, Republic of Korea, in 2006, and the M.S. degreein Computer Science from Korea Advanced Institute of Science and Technology (KAIST), Republic of Korea, in 2008. She is currently pursuing the Ph.D. degree in MultimediaComputing Laboratory, Department of EECS, Division of CS, KAIST. Her research interests are focused on image/video watermarking and fingerprinting with particularattention to multimedia forensics, and information security.

About the Author—HAE-YEOUN LEE received his M.S. and Ph.D. degrees in Computer Science from Korea Advanced Institute of Science and Technology, Republic of Korea,in 1997 and 2006, respectively. From 2001 to 2006, he was with Satrec initiative, Republic of Korea. Currently, he is a Professor in Kumoh National Institute of Technology,Republic of Korea. His major interests are digital watermarking, image processing, remote sensing and digital rights management.

About the Author—HEUNG-KYU LEE received a B.S. degree in Electronics Engineering from Seoul National University, Seoul, Republic of Korea, in 1978, and M.S. and Ph.D.degrees in Computer Science from Korea Advanced Institute of Science and Technology, Republic of Korea, in 1981 and 1984, respectively. Since 1986 he has been a professorin the Department of Computer Science, KAIST. His major interests are digital watermarking, digital fingerprinting, and digital rights management.