Top Banner
313 ©Copyright 1996 by International Business Machines Corpora- tion. Copying in printed form for private use is permitted without payment of royalty provided that (1) each reproduction is done without alteration and (2) the Journal reference and IBM copyright notice are included on the first page. The title and abstract, but no other portions, of this paper may be copied or distributed royalty free without further permission by computer-based and other infor- mation-service systems. Permission to republish any other portion of this paper must be obtained from the Editor. IBM SYSTEMS JOURNAL, VOL 35, NOS 3&4, 1996 0018-8670/96/$5.00 1996 IBM BENDER ET AL. Data hiding, a form of steganography, embeds data into digital media for the purpose of identification, annotation, and copyright. Several constraints affect this process: the quantity of data to be hidden, the need for invariance of these data under conditions where a “host” signal is subject to distortions, e.g., lossy compression, and the degree to which the data must be immune to interception, modification, or removal by a third party. We explore both traditional and novel techniques for addressing the data-hiding process and evaluate these techniques in light of three applications: copyright protection, tamper- proofing, and augmentation data embedding. igital representation of media facilitates access and potentially improves the portability, effi- ciency, and accuracy of the information presented. Undesirable effects of facile data access include an increased opportunity for violation of copyright and tampering with or modification of content. The moti- vation for this work includes the provision of protec- tion of intellectual property rights, an indication of content manipulation, and a means of annotation. Data hiding represents a class of processes used to embed data, such as copyright information, into vari- ous forms of media such as image, audio, or text with a minimum amount of perceivable degradation to the “host” signal; i.e., the embedded data should be invis- ible and inaudible to a human observer. Note that data hiding, while similar to compression, is distinct from encryption. Its goal is not to restrict or regulate access to the host signal, but rather to ensure that embedded data remain inviolate and recoverable. Two important uses of data hiding in digital media are to provide proof of the copyright, and assurance of content integrity. Therefore, the data should stay hid- den in a host signal, even if that signal is subjected to manipulation as degrading as filtering, resampling, cropping, or lossy data compression. Other applica- tions of data hiding, such as the inclusion of augmen- tation data, need not be invariant to detection or removal, since these data are there for the benefit of both the author and the content consumer. Thus, the techniques used for data hiding vary depending on the quantity of data being hidden and the required invari- ance of those data to manipulation. Since no one method is capable of achieving all these goals, a class of processes is needed to span the range of possible applications. The technical challenges of data hiding are formida- ble. Any “holes” to fill with data in a host signal, either statistical or perceptual, are likely targets for removal by lossy signal compression. The key to suc- cessful data hiding is the finding of holes that are not suitable for exploitation by compression algorithms. A further challenge is to fill these holes with data in a way that remains invariant to a large class of host sig- nal transformations. D Techniques for data hiding by W. Bender D. Gruhl N. Morimoto A. Lu
24
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Techniques for Data Hiding

313

©Copyright 1996 by International Business Machines Corpora-tion. Copying in printed form for private use is permitted withoutpayment of royalty provided that (1) each reproduction is donewithout alteration and (2) theJournal reference and IBM copyrightnotice are included on the first page. The title and abstract, but noother portions, of this paper may be copied or distributed royaltyfree without further permission by computer-based and other infor-mation-service systems. Permission torepublish any other portionof this paper must be obtained from the Editor.

IBM SYSTEMS JOURNAL, VOL 35, NOS 3&4, 1996 0018-8670/96/$5.00 1996 IBM BENDER ET AL.

Data hiding, a form of steganography, embedsdata into digital media for the purpose ofidentification, annotation, and copyright. Severalconstraints affect this process: the quantity ofdata to be hidden, the need for invariance of thesedata under conditions where a “host” signal issubject to distortions, e.g., lossy compression,and the degree to which the data must be immuneto interception, modification, or removal by a thirdparty. We explore both traditional and noveltechniques for addressing the data-hiding processand evaluate these techniques in light of threeapplications: copyright protection, tamper-proofing, and augmentation data embedding.

igital representation of media facilitates accessand potentially improves the portability, effi-

ciency, and accuracy of the information presented.Undesirable effects of facile data access include anincreased opportunity for violation of copyright andtampering with or modification of content. The moti-vation for this work includes the provision of protec-tion of intellectual property rights, an indication ofcontent manipulation, and a means of annotation.Data hiding represents a class of processes used toembed data, such as copyright information, into vari-ous forms of media such as image, audio, or text witha minimum amount of perceivable degradation to the“host” signal; i.e., the embedded data should be invis-ible and inaudible to a human observer. Note that datahiding, while similar to compression, is distinct fromencryption. Its goal is not to restrict or regulate accessto the host signal, but rather to ensure that embeddeddata remain inviolate and recoverable.

Two important uses of data hiding in digital media areto provide proof of the copyright, and assurance ofcontent integrity. Therefore, the data should stay hid-den in a host signal, even if that signal is subjected tomanipulation as degrading as filtering, resampling,cropping, or lossy data compression. Other applica-tions of data hiding, such as the inclusion of augmen-tation data, need not be invariant to detection orremoval, since these data are there for the benefit ofboth the author and the content consumer. Thus, thetechniques used for data hiding vary depending on thequantity of data being hidden and the required invari-ance of those data to manipulation. Since no onemethod is capable of achieving all these goals, a classof processes is needed to span the range of possibleapplications.

The technical challenges of data hiding are formida-ble. Any “holes” to fill with data in a host signal,either statistical or perceptual, are likely targets forremoval by lossy signal compression. The key to suc-cessful data hiding is the finding of holes that are notsuitable for exploitation by compression algorithms.A further challenge is to fill these holes with data in away that remains invariant to a large class of host sig-nal transformations.

D

Techniques for datahiding

by W. BenderD. GruhlN. MorimotoA. Lu

Page 2: Techniques for Data Hiding

BENDER ET AL. IBM SYSTEMS JOURNAL, VOL 35, NOS 3&4, 1996314

Features and applications

Data-hiding techniques should be capable of embed-ding data in a host signal with the following restric-tions and features:

1. The host signal should be nonobjectionallydegraded and the embedded data should be mini-mally perceptible. (The goal is for the data toremainhidden. As any magician will tell you, it ispossible for something to be hidden while itremains in plain sight; you merely keep the personfrom looking at it. We will use the wordshidden,inaudible, imperceivable, and invisible to meanthat an observer does not notice the presence of thedata, even if they are perceptible.)

2. The embedded data should be directly encodedinto the media, rather than into a header or wrap-per, so that the data remain intact across varyingdata file formats.

3. The embedded data should be immune to modifi-cations ranging from intentional and intelligentattempts at removal to anticipated manipulations,e.g., channel noise, filtering, resampling, cropping,encoding, lossy compressing, printing and scan-ning, digital-to-analog (D/A) conversion, and ana-log-to-digital (A/D) conversion, etc.

4. Asymmetrical coding of the embedded data isdesirable, since the purpose of data hiding is tokeep the data in the host signal, but not necessarilyto make the data difficult to access.

5. Error correction coding1 should be used to ensuredata integrity. It is inevitable that there will besome degradation to the embedded data when thehost signal is modified.

6. The embedded data should be self-clocking orarbitrarily re-entrant. This ensures that the embed-ded data can be recovered when only fragments ofthe host signal are available, e.g., if a sound bite isextracted from an interview, data embedded in theaudio segment can be recovered. This feature alsofacilitates automatic decoding of the hidden data,since there is no need to refer to the original hostsignal.

Applications. Trade-offs exist between the quantityof embedded data and the degree of immunity to hostsignal modification. By constraining the degree ofhost signal degradation, a data-hiding method canoperate with either high embedded data rate, or highresistance to modification, but not both. As oneincreases, the other must decrease. While this can beshown mathematically for some data-hiding systems

such as a spread spectrum, it seems to hold true for alldata-hiding systems. In any system, you can tradebandwidth for robustness by exploiting redundancy.The quantity of embedded data and the degree of hostsignal modification vary from application to applica-tion. Consequently, different techniques are employedfor different applications. Several prospective applica-tions of data hiding are discussed in this section.

An application that requires a minimal amount ofembedded data is the placement of a digital watermark. The embedded data are used to place an indica-tion of ownership in the host signal, serving the samepurpose as an author’s signature or a company logo.

Since the information is of a critical nature and thesignal may face intelligent and intentional attempts todestroy or remove it, the coding techniques used mustbe immune to a wide variety of possible modifica-tions.

A second application for data hiding is tamper-proof-ing. It is used to indicate that the host signal has beenmodified from its authored state. Modification to theembedded data indicates that the host signal has beenchanged in some way.

A third application, feature location, requires moredata to be embedded. In this application, the embed-ded data are hidden in specific locations within animage. It enables one to identify individual contentfeatures, e.g., the name of the person on the left versusthe right side of an image. Typically, feature locationdata are not subject to intentional removal. However,it is expected that the host signal might be subjectedto a certain degree of modification, e.g., images areroutinely modified by scaling, cropping, and tone-scale enhancement. As a result, feature location data-hiding techniques must be immune to geometrical andnongeometrical modifications of a host signal.

Trade-offs exist betweenthe quantity of data and

the immunity tomodification.

Page 3: Techniques for Data Hiding

IBM SYSTEMS JOURNAL, VOL 35, NOS 3&4, 1996 BENDER ET AL. 315

Image and audio captions (or annotations) mayrequire a large amount of data. Annotations oftentravel separately from the host signal, thus requiringadditional channels and storage. Annotations stored infile headers or resource sections are often lost if thefile format is changed, e.g., the annotations created ina Tagged Image File Format (TIFF) may not be presentwhen the image is transformed to a Graphic Inter-change Format (GIF). These problems are resolved byembedding annotations directly into the data structureof a host signal.

Prior work. Adelson2 describes a method of data hid-ing that exploits the human visual system’s varyingsensitivity to contrast versus spatial frequency. Adel-son substitutes high-spatial frequency image data forhidden data in a pyramid-encoded still image. Whilehe is able to encode a large amount of data efficiently,there is no provision to make the data immune todetection or removal by typical manipulations such asfiltering and rescaling. Stego,3 one of several widelyavailable software packages, simply encodes data inthe least-significant bit of the host signal. This tech-nique suffers from all of the same problems as Adel-son’s method but creates an additional problem ofdegrading image or audio quality. Bender4 modifiesAdelson’s technique by usingchaos as a means toencrypt the embedded data, deterring detection, butproviding no improvement to immunity to host signalmanipulation. Lippman5 hides data in the chromi-nance channel of the National Television StandardsCommittee (NTSC) television signal by exploiting thetemporal over-sampling of color in such signals. Typi-cal of Enhanced Definition Television Systems, thismethod encodes a large amount of data, but the dataare lost to most recording, compression, and transcod-ing processes. Other techniques, such as Hecht’sData-Glyph,6 which adds abar code to images, areengineered in light of a predetermined set of geomet-ric modifications.7 Spread-spectrum,8-11 a promisingtechnology for data hiding, is difficult to intercept andremove but often introduces perceivable distortioninto the host signal.

Problem space. Each application of data hidingrequires a different level of resistance to modificationand a different embedded data rate. These form thetheoretical data-hiding problem space (see Figure 1).There is an inherent trade-off between bandwidth and“robustness,” or the degree to which the data areimmune to attack or transformations that occur to thehost signal through normal usage, e.g., compression,resampling, etc. The more data to be hidden, e.g., a

caption for a photograph, the less secure the encoding.The less data to be hidden, e.g., a watermark, the moresecure the encoding.

Data hiding in still images

Data hiding in still images presents a variety of chal-lenges that arise due to the way the human visual sys-tem (HVS) works and the typical modifications thatimages undergo. Additionally, still images provide arelatively small host signal in which to hide data. Afairly typical 8-bit picture of 200× 200 pixels pro-vides approximately 40 kilobytes (kB) of data spacein which to work. This is equivalent to only around 5seconds of telephone-quality audio or less than a sin-gle frame ofNTSC television. Also, it is reasonable toexpect that still images will be subject to operationsranging from simple affine transforms to nonlineartransforms such as cropping, blurring, filtering, andlossy compression. Practical data-hiding techniquesneed to be resistant to as many of these transforma-tions as possible.

Despite these challenges, still images are likely candi-dates for data hiding. There are many attributes of theHVS that are potential candidates for exploitation in adata-hiding system, including our varying sensitivityto contrast as a function of spatial frequency and themasking effect of edges (both in luminance and

Figure 1 Conceptual data-hiding problem space

ROBUSTNESSB

AN

DW

IDT

H

EXTENT OF CURRENT TECHNIQUES

Page 4: Techniques for Data Hiding

BENDER ET AL. IBM SYSTEMS JOURNAL, VOL 35, NOS 3&4, 1996316

chrominance). TheHVS has low sensitivity to smallchanges in luminance, being able to perceive changesof no less than one part in 30 for random patterns.However, in uniform regions of an image, theHVSis more sensitive to the change of the luminance,approximately one part in 240. A typicalCRT (cathoderay tube) display or printer has a limited dynamicrange. In an image representation of one part in 256,e.g., 8-bit gray levels, there is potentially room to hidedata as pseudorandom changes to picture brightness.Another HVS “hole” is our relative insensitivity tovery low spatial frequencies such as continuouschanges in brightness across an image, i.e., vignett-ing. An additional advantage of working with stillimages is that they are noncausal. Data-hiding tech-niques can have access to any pixel or block of pixelsat random.

Using these observations, we have developed a varietyof techniques for placing data in still images. Sometechniques are more suited to dealing with smallamounts of data, while others to large amounts. Sometechniques are highly resistant to geometric modifica-tions, while others are more resistant to nongeometricmodifications, e.g., filtering. We present methods thatexplore both of these areas, as well as their combina-tion.

Low bit-rate data hiding

With low bit-rate encoding, we expect a high level ofrobustness in return for low bandwidth. The emphasisis on resistance to attempts of data removal by a thirdparty. Both a statistical and a perceptual technique arediscussed in the next sections on Patchwork, texture,and applications.

Patchwork: A statistical approach

The statistical approach, which we refer to asPatch-work, is based on a pseudorandom, statistical process.Patchwork invisibly embeds in a host image a specificstatistic, one that has a Gaussian distribution. Figure 2shows a single iteration in the Patchwork method.Two patches are chosen pseudorandomly, the first A,the second B. The image data in patch A are lightenedwhile the data in patch B are darkened (exaggeratedfor purposes of this illustration). This unique statisticindicates the presence or absence of a signature.Patchwork is independent of the contents of the hostimage. It shows reasonably high resistance to mostnongeometric image modifications.

For the following analysis, we make the followingsimplifying assumptions (these assumptions are notlimiting, as is shown later): We are operating in a 256level, linearly quantized system starting at 0; allbrightness levels are equally likely; all samples areindependent of all other samples.

The Patchwork algorithm proceeds as follows: takeany two points,A and B, chosen at random in animage. Leta equal the brightness at pointA andb thebrightness at pointB. Now, let

(1)

Theexpected value ofS is 0, i.e., the average value ofS after repeating this procedure a large number oftimes isexpected to be 0.

S a b–=

Figure 2 A single iteration in the Patchwork method(photograph courtesy of Webb Chapel)

Page 5: Techniques for Data Hiding

IBM SYSTEMS JOURNAL, VOL 35, NOS 3&4, 1996 BENDER ET AL. 317

Although theexpected value is 0, this does not tell usmuch about whatS will be for a specific case. This isbecause the variance is quite high for this procedure.The variance ofS, σs is a measure of how tightly sam-ples ofS will cluster around the expected value of 0.To compute this, we make the following observation:SinceS= a − b anda andb are assumed independent,

can be computed as follows (this, and all otherprobability equations are from Drake12):

(2)

where for a uniformS is:

(3)

Now, sincea andb are samples from thesame set, taken with replacement. Thus:

(4)

which yields a standard deviationσS ≈ 104. Thismeans that more than half the time,S will be greaterthan 43 or less than− 43. Assuming a Gaussian clus-tering, a single iteration does not tell us much. How-ever, this is not the case if we perform the proceduremany times.

Let us repeat this proceduren times, lettingai andbi

be the valuesa andb take on during theith iteration,Si. Now letSn be defined as:

(5)

Theexpectedvalue ofSn is:

(6)

This makes intuitive sense, since the number of timesai is greater thanbi should be offset by the number oftimes the reverse is true. Now the variance is:

(7)

And the standard deviation is:

(8)

Now, we can computeS10000 for a picture, and if it var-ies by more than a few standard deviations, we can befairly certain that this did not happen by chance. Infact, since as we will show laterS′n for largen has aGaussian distribution, a deviation of even a fewσS′sindicates to a high degree of certainty the presence ofencoding (see Table 1).

The Patchwork method artificially modifiesS for agiven picture, such thatS′n is many deviations awayfrom expected. To encode a picture, we:

1. Use a specific key for a known pseudorandomnumber generator to choose (ai, bi). This is impor-tant, because the encoder needs to visit the samepoints during decoding.

2. Raise the brightness in the patchai by an amountδ,typically in the range of 1 to 5 parts in 256.

3. Lower the brightness inbi by this same amountδ(the amounts do not have to be the same, as long asthey are in opposite directions).

4. Repeat this forn steps (n typically ~10 000).

Now, when decoded,S′n will be:

(9)

or:

(10)

So each step of the way we accumulate an expectationof 2 × δ. Thus aftern repetitions, we expectS′n to be:

σS2

σS2 σa

2 σb2+=

σa2

σa2 5418≈

σa2 σb

2=

σS2 2 σ× a

2 2255 0–( )2

12------------------------× 10836≈ ≈=

Sn Sii 1=

n

∑ ai bi–i 1=

n

∑= =

Sn n S× n 0× 0= = =

σSn

2 n σS2×=

σSnn σ× n 104×≈=

Sn′ ai δ+( ) bi δ–( )–

i 1=

n

∑=

Sn′ 2δn ai bi–( )

i 1=

n

∑+=

Table 1 Degree of certainty of encoding given deviationfrom that expected in a Gaussian distribution(δ = 2)

StandardDeviations

Away

Certainty n

0123

50.00%84.13%97.87%99.87%

0679

27136104

Page 6: Techniques for Data Hiding

BENDER ET AL. IBM SYSTEMS JOURNAL, VOL 35, NOS 3&4, 1996318

(11)

As n or δ increases, the distribution ofS′n shifts overto the right (Figure 3 and Table 1). In Figure 3, asδ orn increases, the distribution ofS′n shifts further to theright. If we shift it far enough, any point that is likelyto fall into one distribution is highly unlikely to benear the center of the other distribution.

While this basic method works well by itself, we havemade a number of modifications to improve perfor-mance including:

1. Treatingpatches of several points rather than sin-gle points. This has the effect of shifting the noiseintroduced by Patchwork into the lower spatial fre-quencies, where it is less likely to be removed bylossy compression and typical Finite ImpulseResponse (FIR) filters.

2. Making Patchwork more robust by using a combi-nation with either affine coding (described later) orsome heuristic based upon feature recognition(e.g., alignment using the interocular line of aface). Patchwork decoding is sensitive to affinetransformations of the host image. If the points inthe picture visited during encoding are offset bytranslation, rotation, or scaling before decoding,the code is lost.

3. Taking advantage of the fact that Patchwork isfairly resistant to cropping. By disregarding pointsoutside of the known picture area, Patchworkdegrades in accuracy approximately as the log ofthe picture size. Patchwork is also resistant togamma and tone scale correction since values ofcomparable luminance move roughly the sameway under such modifications.

Patch shape. The shape of the patches deserves somecomment. Figure 4 shows three possible one-dimen-sional patch shapes, and next to them a very approxi-mate spectrum of what a line with these patchesdropped onto it pseudorandomly would look like. InFigure 4A, the patch is very small, with sharp edges.This results in the majority of the energy of the patchbeing concentrated in the high frequency portion ofthe image spectrum. This makes the distortion hard tosee, but also makes it a likely candidate for removalby lossy compressors. If one goes to the otherextreme, as in Figure 4B, the majority of the informa-tion is contained in the low-frequency spectrum. The

2δnσSn

--------- 0.028δ n≈Figure 3 As δ or n increases, the distribution of S´nshifts further to the right.

0 4 TO 8 STANDARD DEVIATIONS

Figure 4 The contour of a patch largely determineswhich frequencies will be modified by theapplication of Patchwork.

π0

(A)

(B)

(C)

PATCHMODIFIED

π

π

0

0

FREQUENCIES

Figure 5 Patch placement affects patch visibility.

RECTILINEAR HEXAGONAL RANDOM

(A) (B) (C)

Page 7: Techniques for Data Hiding

IBM SYSTEMS JOURNAL, VOL 35, NOS 3&4, 1996 BENDER ET AL. 319

last choice, Figure 4C shows a wide, sharp-edgedpatch, which tends to distribute the energy around theentire frequency spectrum.

The optimal choice of patch shape is dependent uponthe expected image modifications. IfJPEG (Joint Pho-tographic Experts Group) encoding is likely, then apatch that places its energy in the low frequencies ispreferable. If contrast enhancement is to be done,placing energy in higher frequencies would be better.If the potential image modifications are unknown,then spreading the patch energy across the spectrumwould make sense.

The arrangement of patches has an impact on patchvisibility. For illustration, three possibilities are con-sidered (Figure 5). The simplest method is shown inFigure 5A, a simple rectilinear lattice. While simple,this arrangement is often a poor choice if a highn is tobe used. As the grid is filled in, continuous edges ofgradient are formed. TheHVS is very sensitive to suchedges. A second choice, Figure 5B breaks this sym-metry by using hexagons for the patch shape. A pre-ferred solution, shown in Figure 5C, is a completelyrandom placement of patches. An intelligent selectionof patch shape in both the horizontal and verticaldimensions will enhance the effectiveness of patch-work for a given picture.

Uniformity. A simplifying assumption of a uniformluminance histogram was made above, but this is nota requirement of Patchwork. The only assumptionPatchwork makes is that theexpected value ofSi = ai −bi is zero.

It can be shown that this condition is always metthrough the following argument:

1. Letar be the time reversed series ofa.2. Ar = A* by definition (A* is the complex conjugate

of A).3. F(a∗ar) = AA* (F is the Fourier transform).4. AA* is everywhere real by definition of the com-

plex conjugate.5. F–1(AA*) is even by definition.6. Even sequences are symmetric around zero by def-

inition.

An image histogram (Figure 6, top) is a somewhatrandom distribution. The result of taking the complexconjugate (Figure 6, bottom) is symmetric aroundzero.

Figure 6 A histogram of Figure 2 and its autocorrelation

Figure 7 A histogram of the variance of the luminanceof 365 Associated Press photos fromMarch 1996

0 2 4 6 8 10 12 14 16

500000

450000

400000

350000

300000

250000

200000

150000

100000

50000

0

–15 –10 –5 0 5 10 15

2.5e+11

2e+11

1.5e+11

1e+11

5e+10

0

0 2000 4000 6000 8000

50

45

40

35

30

25

20

15

10

5

010000

VARIANCE FROM

(5418)EQUATION 3

Page 8: Techniques for Data Hiding

BENDER ET AL. IBM SYSTEMS JOURNAL, VOL 35, NOS 3&4, 1996320

Variance. When searching through a large number ofimages with data embedded using the Patchworkmethod, such as when a robot is looking for copyrightviolations on the Internet World Wide Web (WWW),the use of a generic estimation of variance is desir-able. This avoids the necessity of calculating the vari-ance of every image. Suspect images can then beexamined thoroughly.

When, in the analysis above, the number of pointsneeded in Equation 3 was computed, the variance ofthe luminance was assumed to be 5418. This assump-tion turns out to be higher than the average observedvalues (see Figure 7). The question is, then, whatvalue should be used.

An examination of the variance of 365 AssociatedPress photos from March 1996 yielded an averagevalue of 3877.4 and a distribution that can be seen inFigure 7. While some pictures do have variances ashigh as two-thirds of the maximum, most are clus-tered around the lower variance values. Thus, 5418,the estimate derived from the uniformity assumption,is a conservative but reasonable value to use for ageneric picture.

A minimum value is that for a solid color picture (Fig-ure 8A). This has a variance of 0, a standard deviationof 0, and thus works very well for Patchwork, sinceany modification is evident. The other extreme is thatof a two-color, black and white picture. For these, thevariance is:

(12)

These two values, 0 and 16256, define the extremes ofthe variance to consider when calculating the likeli-hood that a picture is encoded. What is the correctassumption to use for a given picture? The actual vari-ance of the picture being examined is a sensiblechoice, since in most cases Patchwork will increasethe variance only slightly. (This depends on the sizeand depth of the patch, the number of patches, and thehistogram of the original image.) However, if a largenumber of pictures are to examined, a generic value isa practical choice.

Summary. There are several limitations inherent tothe Patchwork technique. The first is the extremelylow embedded data rate it yields, usually a one-bitsignature per image. This limits its usefulness to lowbit-rate applications such as the digital watermark.Second, it is necessary toregister where the pixels inthe image lie. While a number of methods have beeninvestigated, it is still somewhat difficult to decode theimage in the presence of severe affine transforma-tions. These disadvantages aside, without the key forthe pseudorandom number generator, it is extremelydifficult to remove the Patchwork coding withoutdegrading the picture beyond recognition.

The Patchwork method is subject to cryptographicattack if it is used to encode a large number of identi-cally sized images using the same key. If the imagesare averaged together, thepatches will show up aslighter or darker than average regions. This weaknessis a common one in cryptography, and points to thetruism that for a static key, as the amount of trafficincreases, it becomes easier to “crack” the encryption.One solution is to use multiple pseudorandom pat-terns for the patches. Even the use of just two keys,while increasing decoding time, will make Patchworkmuch more robust to attack. Another solution is to usethe same pattern, but to reverse the polarity of thepatches. Both solutions deter cryptographic attack byaveraging.

Texture Block Coding: A visual approach

A second method for low bit-rate data hiding inimages isTexture Block Coding. This method hidesdata within the continuous random texture patterns ofa picture. The Texture Block Coding technique is

0 127–( )2

2------------------------ 255 127.5–( )2

2-----------------------------------+ 16256≈Figure 8 Histograms of pictures with minimum (A) and

maximum (B) variance

MIN MAXANY

(A) (B)

Page 9: Techniques for Data Hiding

IBM SYSTEMS JOURNAL, VOL 35, NOS 3&4, 1996 BENDER ET AL. 321

implemented by copying a region from a random tex-ture pattern found in a picture to an area that has simi-lar texture. This results in a pair of identically texturedregions in the image (see Figure 9).

These regions can be detected as follows:

1. Autocorrelate the image with itself. This will pro-duce peaks at every point in the autocorrelationwhere identical regions of the image overlap. Iflarge enough areas of an image are copied, thiswill produce an additional large autocorrelationpeak at the correct alignment for decoding.

2. Shift the image as indicated by the peaks in Step 1.Now subtract the image from its shifted copy, pad-ding the edges with zeros as needed.

3. Square the result and threshold it to recover onlythose values quite close to zero. The copied regionwill be visible as these values.

Since the two regions are identical, they are modifiedin the same way if the picture is uniformly trans-formed. By making the regions reasonably large, theinner part of the block changes identically under mostnongeometric transformations. In our experiments,coded 16× 16 pixel blocks can be decoded when thepicture is subjected to a combination of filtering, com-pression, and rotation.

Texture Block Coding is not without its disadvan-tages. Currently it requires a human operator tochoose the source and destination regions, and to eval-uate the visual impact of the modifications on theimage. It should be possible to automate this processby allowing a computer to identify possible textureregions in the image to copy from and paste to. How-ever, this technique will not work on images that lackmoderately large areas of continuous texture fromwhich to draw.

Figure 9 Texture Block Coding example (photograph courtesy of Webb Chapel)

Page 10: Techniques for Data Hiding

BENDER ET AL. IBM SYSTEMS JOURNAL, VOL 35, NOS 3&4, 1996322

Future research in this area includes the possibility ofcutting and pasting blocks from only part of the imagefrequency spectrum (this would allow less noticeableblocks to be moved around, and a final encoding thatis considerably more robust to various image com-pression algorithms) along with automatic textureregion selection and analysis of perceivability of thefinal result.

High bit-rate coding

High bit-rate methods can be designed to have mini-mal impact upon the perception of the host signal, butthey do not tend to be immune to image modifica-tions. In return, there is an expectation that a relativelylarge amount of data are able to be encoded. The mostcommon form of high bit-rate encoding is the replace-ment of the least significant luminance bit of imagedata with the embedded data. Other techniquesinclude the introduction of high-frequency, low-amplitude noise and the use of direct sequence spreadspectrum coding. All high bit-rate methods can bemade more robust through the use of error-correctioncoding, at the expense of data rate. High bit-rate codesare only appropriate where it is reasonable to expectthat a great deal of control will be maintained over theimages.

Individually, none of the known techniques for datahiding are resistant to all possible transforms or com-binations of transforms. In combination, often one

technique can supplement another. Supplementarytechniques are particularly important for recoveryfrom geometric modifications such as affine transfor-mations, and maintaining synchronization for spread-spectrum encoding.

Affine coding. Some of the data-hiding techniques,such as Patchwork, are vulnerable to affine trans-forms. It makes sense to develop methods that can beused to facilitate the recovery of embedded data afteraffine application.Affine coding is one such method:A predefined reference pattern is embedded into ahost image using any of the high bit-rate coding tech-niques. Estimation of geometric transformation of theimage is achieved by comparing the original shape,size, and orientation of the reference pattern to thatfound in the transformed image. Since affine trans-forms are linear, the inverse transform can be appliedto recover the original image. Once this is done, theimage is ready for further extraction of embeddeddata.

Applications

Placing data in images is useful in a variety of appli-cations. We highlight below four applications that dif-fer in the quantity of data to be embedded and the typeof transforms to which the data are likely to be sub-jected.

Digital watermark. The objective of a digital water-mark is to place an indelible mark on an image. Usu-ally, this means encoding only a handful of bits,sometimes as few as one. This “signature” could beused as a means of tracing the distribution of imagesfor an on-line news service and for photographerswho are selling their work for digital publication. Onecould build a digital camera that places a watermarkon every photograph it takes. Theoretically, thiswould allow photographers to employ a “web-search-ing agent” to locate sites where their photographsappear.

It can be expected that if information about legal own-ership is to be included in an image, it is likely thatsomeone might want to remove it. A requirement of adigital watermark is that it must be difficult to remove.Both the Patchwork and Texture Block Coding tech-niques show promise as digital watermarks. Patch-work, being the more secure of the two, answers thequestion “Is this my picture?” Texture Block Coding,which can be made readily accessible to the public,answers the question “Whose picture is this?”

Figure 10 Characterizing the difference betweentamper-proofing and other data-hidingtechniques

EF

FO

RT

(A) (B)MODIFICATION

Page 11: Techniques for Data Hiding

IBM SYSTEMS JOURNAL, VOL 35, NOS 3&4, 1996 BENDER ET AL. 323

Tamper-proofing. The objective of tamper-proofingis to answer the question, “Has this image been modi-fied?” Tamper-proofing techniques are related, butdistinct from the other data-hiding technologies. Whatdifferentiates them is the degree to which informationis secured from the host signal. In Figure 10, the dif-ference between tamper-proofing and other data-hid-ing techniques is characterized. Figure 10A illustratesthat data hiding requires a deepinformation well thatis resilient to large displacements. Figure 10B illus-trates that tamper-proofing requires a shallow wellthat is only resilient to small displacements, but istriggered by large displacements. Most data-hidingtechniques attempt to secure data in the face of allmodifications. Tamper-proofing techniques must beresilient to small modifications (e.g., cropping, tone-scale or gamma correction for images or balance orequalization for sounds) but not to large modifications(e.g., removing or inserting people from an image ortaking words out of context in an audio recording).

There are several ways to implement tamper-proofing.The easiest way is to encode a check-sum of theimage within the image. However, this method is trig-gered by small changes in the image. This suggests anapproach involving a pattern overlaid on the image.The key to a successful overlay is to find a patternresilient to simple modifications such as filtering andgamma correction, yet is not easily removed. Thesearch for such patterns and other methods of detect-ing tampering remains an active area of research.

Feature tagging. Another application of data hidingis tagging the location of features within an image.Using data hiding it is possible for an editor (ormachine) to encode descriptive information, such asthe location and identification of features of interest,directly into specific regions of an image. Thisenables retrieval of the descriptive information wher-ever the image goes. Since the embedded informationis spatially located in the image, it is not removedunless the feature of interest is removed. It also trans-lates, scales, and rotates exactly as the feature of inter-est does.

This application does not have the same requirementsfor robustness as the digital watermark. It can beassumed that, since feature location is providing a ser-vice, it is unlikely someone will maliciously try toremove the encoded information.

Embedded captions. Typical news photograph cap-tions contain one kB of data. Thus embedded captions

is a relatively high bit-rate application for data hiding.As with feature tagging, caption data are usually notsubject to malicious removal.

While captions are useful by themselves, they becomeeven more useful when combined with feature loca-tion. It is then possible for portions of the caption todirectly reference items in the picture. Captions canbecome self-editing once this is done. If an item refer-enced in the caption is cropped out of the picture, thenthe reference to that item in the caption can beremoved automatically.

Data hiding in audio

Data hiding in audio signals is especially challenging,because the human auditory system (HAS) operatesover a wide dynamic range. TheHAS perceives over arange of power greater than one billion to one and arange of frequencies greater than one thousand to one.Sensitivity to additive random noise is also acute. Theperturbations in a sound file can be detected as low asone part in ten million (80 dB below ambient level).However, there are some “holes” available. While theHAS has a large dynamic range, it has a fairly smalldifferential range. As a result, loud sounds tend tomask out quiet sounds. Additionally, theHAS isunable to perceive absolute phase, only relative phase.Finally, there are some environmental distortions socommon as to be ignored by the listener in mostcases.

We exploit many of these traits in the methods we dis-cuss next, while being careful to bear in mind theextreme sensitivities of theHAS.

Audio environments

When developing a data-hiding method for audio, oneof the first considerations is the likely environmentsthe sound signal will travel between encoding anddecoding. There are two main areas of modificationwhich we will consider. First, the storage environ-ment, or digital representation of the signal that willbe used, and second the transmission pathway the sig-nal might travel.

Digital representation. There are two critical param-eters to most digital audio representations: samplequantization method and temporal sampling rate.

The most popular format for representing samples ofhigh-quality digital audio is a 16-bit linear quantiza-

Page 12: Techniques for Data Hiding

BENDER ET AL. IBM SYSTEMS JOURNAL, VOL 35, NOS 3&4, 1996324

tion, e.g., Windows Audio-Visual (WAV) and AudioInterchange File Format (AIFF). Another popular for-mat for lower quality audio is the logarithmicallyscaled 8-bitµ-law. These quantization methods intro-duce some signal distortion, somewhat more evidentin the case of 8-bitµ-law.

Popular temporal sampling rates for audio include8 kHz (kilohertz), 9.6 kHz, 10 kHz, 12 kHz, 16 kHz,22.05 kHz, and 44.1 kHz. Sampling rate impacts datahiding in that it puts an upper bound on the usableportion of the frequency spectrum (if a signal is sam-pled at ~8 kHz, you cannot introduce modificationsthat have frequency components above ~4 kHz). Formost data-hiding techniques we have developed,usable data space increases at least linearly withincreased sampling rate.

A last representation to consider is that produced bylossy, perceptual compression algorithms, such as theInternational Standards Organization Motion PicturesExpert Group—Audio (ISO MPEG-AUDIO) perceptual

encoding standard. These representations drasticallychange the statistics of the signal; they preserve onlythe characteristics that a listener perceives (i.e., it willsound similar to the original, even if the signal is com-pletely different in aleast squares sense).

Transmission environment. There are many differ-ent transmission environments that a signal mightexperience on its way from encoder to decoder. Weconsider four general classes for illustrative purposes(see Figure 11). The first is the digital end-to-endenvironment (Figure 11A). This is the environment ofa sound file that is copied from machine to machine,but never modified in any way. As a result, the sam-pling is exactly the same at the encoder and decoder.This class puts the least constraints on data-hidingmethods.

The next consideration is when a signal is resampledto a higher or lower sampling rate, but remains digitalthroughout (Figure 11B). This transform preserves theabsolute magnitude and phase of most of the signal,but changes the temporal characteristics of the signal.

The third case is when a signal is “played” into ananalog state, transmitted on a reasonably clean analogline and resampled (Figure 11C). Absolute signalmagnitude, sample quantization, and temporal sam-pling rate are not preserved. In general, phase will bepreserved.

The last case is when the signal is “played into theair” and “resampled with a microphone” (Figure11D). The signal will be subjected to possiblyunknown nonlinear modifications resulting in phasechanges, amplitude changes, drift of different fre-quency components, echoes, etc.

Signal representation and transmission pathway mustbe considered when choosing a data-hiding method.Data rate is very dependent on the sampling rate andthe type of sound being encoded. A typical value is16 bps, but the number can range from 2 bps to128 bps.

Low-bit coding

Low-bit coding is the simplest way to embed data intoother data structures. By replacing the least significantbit of each sampling point by a coded binary string,we can encode a large amount of data in an audio sig-nal. Ideally, the channel capacity is 1 kb per second(kbps) per 1 kilohertz (kHz), e.g., in a noiseless chan-

Figure 11 Transmission environments

SOURCE

SOURCE

SOURCE

SOURCE

DESTINATION

DIGITAL

RESAMPLED

ANALOG

“OVER THE AIR”

(A)

(B)

(C)

(D)

DESTINATION

DESTINATION

DESTINATION

Page 13: Techniques for Data Hiding

IBM SYSTEMS JOURNAL, VOL 35, NOS 3&4, 1996 BENDER ET AL. 325

nel, the bit rate will be 8 kbps in an 8 kHz sampledsequence and 44 kbps in a 44 kHz sampled sequence.In return for this large channel capacity, audible noiseis introduced. The impact of this noise is a directfunction of the content of the host signal, e.g., crowdnoise during a live sports event would mask low-bitencoding noise that would be audible in a string quar-tet performance. Adaptive data attenuation has beenused to compensate this variation.

The major disadvantage of this method is its poorimmunity to manipulation. Encoded information canbe destroyed by channel noise, resampling, etc.,unless it is encoded using redundancy techniques. Inorder to be robust, these techniques reduce the datarate, often by one to two orders of magnitude. In prac-tice, this method is useful only in closed, digital-to-digital environments.

Phase coding

The phase coding method works by substituting thephase of an initial audio segment with a referencephase that represents the data. The phase of subse-quent segments is adjusted in order to preserve therelative phase between segments.

Phase coding, when it can be used, is one of the mosteffective coding methods in terms of the signal-to-perceived noise ratio. When the phase relationbetween each frequency component is dramaticallychanged, a noticeable phase dispersion will occur.However, as long as the modification of the phase issufficiently small (sufficiently small depends on theobserver; professionals in broadcast radio can detectmodifications that are imperceivable to an averageobserver), aninaudible coding can be achieved.

Procedure. The procedure for phase coding is as fol-lows:

1. Break the sound sequences[i], (0 ≤ i ≤ I − 1), into aseries ofN short segments,sn[i] where (0≤ n ≤ N −1) (Figure 12A, 12B).

2. Apply a K-points discrete Fourier transform(DFT)13 to n-th segment,sn[i], where (K = I/N), andcreate a matrix of the phase,φn(ωk), and magni-tude,An(ωk) for (0 ≤ k ≤ K − 1) (Figure 12C).

3. Store the phase difference between each adjacentsegment for (0≤ n ≤ N − 1) (Figure 12D):

(13)∆φn 1+ ωk( ) φn 1+ ωk( ) φn ωk( )–=

Figure 12 Phase coding schematic

(A) ORIGINAL SOUND SIGNAL

(B) BREAK INTO N SEGMENTS S

(C) TRANSFORM EACH SEGMENT INTO MAGNITUDE

(D) CALCULATE THE PHASE DIFFERENCE BETWEEN

(E) FOR SEGMENT S0 CREATE AN ARTIFICIAL

(F) FOR ALL OTHER SEGMENTS, CREATE NEW

(G) COMBINE NEW PHASE AND ORIGINAL MAGNITUDE

(H) CONCATENATE NEW SEGMENTS TO CREATE OUTPUT

S0 S1 S2 SN –1

An

S′n

φn

φ′0

∆φn

ω

ω

ω

CONSECUTIVE SEGMENTS φn+1 − φn

ω

TO GET NEW SEGMENT S′n

AND PHASE

PHASE FRAMES (φ′0+∆φ1)

φ′1

ABSOLUTE PHASE φ′0

Page 14: Techniques for Data Hiding

BENDER ET AL. IBM SYSTEMS JOURNAL, VOL 35, NOS 3&4, 1996326

4. A binary set of data is represented as aφdata = π/2or − π/2 representing 0 or 1 (Figure 12E):

(14)

5. Re-create phase matrixes forn > 0 by using thephase difference (Figure 12F):

(15)

6. Use the modified phase matrixφ′n(ωk) and the orig-inal magnitude matrixAn(ωk) to reconstruct thesound signal by applying the inverseDFT (Figure12G, 12H).

For the decoding process, the synchronization of thesequence is done before the decoding. The length ofthe segment, theDFT points, and the data interval mustbe known at the receiver. The value of the underlyingphase of the first segment is detected as a 0 or 1,which represents the coded binary string.

Sinceφ′0(ωk) is modified, the absolute phases of thefollowing segments are modified respectively. How-ever, the relative phase difference of each adjacentframe is preserved. It is this relative difference inphase that the ear is most sensitive to.

Evaluation. Phase dispersion is a distortion caused bya break in the relationship of the phases between eachof the frequency components. Minimizing phase dis-

persion constrains the data rate of phase coding. Onecause of phase dispersion is the substitution of phaseφ′0(ωk) with the binary code. The magnitude of thephase modifier needs to be close to the original valuein order to minimize distortion. The differencebetween phase modifier states should be maximizedin order to minimize the susceptibility of the encodingto noise. In our modified phase representation, a 0-bitis − π/2 and a 1-bit is +π/2.

Another source of distortion is the rate of change ofthe phase modifier. If distortion is applied to every binof theDFT it is likely to break the phase relationship ofthe adjacent frequency components, resulting in a beatpattern. By changing the phase more slowly and tran-sitioning between phase changes, the audible distor-tion is greatly reduced. In Figure 13, a sharp versussmooth transition is illustrated. In Figure 13A, theedges of the phase transitions are sharp, causingnoticeable distortion. In Figure 13B, they are smooth,reducing this. Note that in each case, the data pointsappear in the same place. This smooth variation hasthe disadvantage of causing a reduction in bandwidth,as space has to be left between each data point toallow for smooth transition.

Results. In our experiments, the phase coding channelcapacity typically varied from 8 bps to 32 bps,depending on the sound context. A channel capacityof ~8 bps can be achieved by allocating 128 frequencyslots per bit under conditions of little backgroundnoise. Capacities of 16 bps to 32 bps can be achievedby allocating 32 to 64 frequency slots per slot whenthere is a noisy background.

Spread spectrum

In a normal communication channel, it is often desir-able to concentrate the information in as narrow aregion of the frequency spectrum as possible in orderto conserve available bandwidth and to reduce power.The basic spread spectrum technique, on the otherhand, is designed to encode a stream of informationby spreading the encoded data across as much of thefrequency spectrum as possible. This allows the signalreception, even if there is interference on some fre-quencies.

While there are many variations on spread spectrumcommunication, we concentrated on Direct SequenceSpread Spectrum encoding (DSSS). TheDSSS methodspreads the signal by multiplying it by achip, a maxi-mal length pseudorandom sequence modulated at a

φ0′ φdata′=

φ1′ ωk( ) φ0′ ωk( ) ∆φ1 ωk( )+=( )…

φn′ ωk( ) φn 1–′ ωk( ) ∆φn ωk( )+=( )…

φN′ ωk( ) φN 1–′ ωk( ) ∆φN ωk( )+=( )

Figure 13 Sharp verses smooth transition

φ

ω

(A) (B)

φ

ω

Page 15: Techniques for Data Hiding

IBM SYSTEMS JOURNAL, VOL 35, NOS 3&4, 1996 BENDER ET AL. 327

known rate. Since the host signals are in discrete-timeformat, we can use the sampling rate as thechip ratefor coding. The result is that the most difficult prob-lem in DSSS receiving, that of establishing the correctstart and end of the chip quanta for phase locking pur-poses, is taken care of by the discrete nature of thesignal. Consequently, a much higher chip rate, andtherefore a higher associated data rate, is possible.Without this, a variety of signal locking algorithmsmay be used, but these are computationally expensive.

Procedure. In DSSS, a key is needed to encode theinformation and the same key is needed to decode it.The key is pseudorandom noise that ideally has flatfrequency response over the frequency range, i.e.,white noise. The key is applied to the coded informa-tion to modulate the sequence into a spread spectrumsequence.

The DSSSmethod is as follows:8,9 The code is multi-plied by the carrier wave and the pseudorandom noisesequence, which has a wide frequency spectrum. As aconsequence, the spectrum of the data is spread overthe available band. Then, the spread data sequence isattenuated and added to the original file as additiverandom noise (see Figure 14).DSSS employs bi-phaseshift keying since the phase of the signal alternateseach time the modulated code alternates (see Figure15). For decoding, phase valuesφ0 and φ0 + π areinterpreted as a “0” or a “1,” which is a coded binarystring.

In the decoding stage, the following is assumed:

1. The pseudorandom key is maximal (it has as manycombinations as possible and does not repeat for aslong as possible). Consequently it has a relativelyflat frequency spectrum.

2. The key stream for the encoding is known by thereceiver. Signal synchronization is done, and thestart/stop point of the spread data are known.

3. The following parameters are known by thereceiver: chip rate, data rate, and carrier frequency.

Results. Unlike phase coding,DSSS introduces addi-tive random noise to the sound. To keep the noiselevel low and inaudible, the spread code is attenuated(without adaptation) to roughly 0.5 percent of thedynamic range of the host sound file. The combina-tion of simple repetition technique and error correc-tion coding ensure the integrity of the code. A shortsegment of the binary code string is concatenated andadded to the host signal so that transient noise can be

reduced by averaging over the segment in the decod-ing stage. The resulting data rate of theDSSS experi-ments is 4 bps.

Echo data hiding

Echo data hiding embeds data into a host audio signalby introducing anecho. The data are hidden by vary-ing three parameters of the echo: initial amplitude,decay rate, and offset (see Figure 16). As the offset (or

Figure 14 Spread spectrum encoding

Figure 15 Synthesized spread spectrum informationencoded by the direct sequence method

ENCODER

OUTPUTCARRIER

“CHIP” SIGNAL DATA SIGNAL

“CHIP” SIGNAL

INPUT

DECODER

BAND-PASS PHASEFILTER DETECTOR DATA

CARRIER WAVE

PSEUDORANDOMSIGNAL (“CHIP”)

BINARY STRING(CODE)

CODED

1

0

−1

1

0

−1

1

0

−1

1

0

−1WAVEFORM

Page 16: Techniques for Data Hiding

BENDER ET AL. IBM SYSTEMS JOURNAL, VOL 35, NOS 3&4, 1996328

delay) between the original and the echo decreases,the two signals blend. At a certain point, the humanear cannot distinguish between the two signals. Theecho is perceived as added resonance. (This point ishard to determine exactly. It depends on the quality ofthe original recording, the type of sound being ech-oed, and the listener. In general, we find that thisfusion occurs around 1/1000 of a second for mostsounds and most listeners.)

The coder uses two delay times, one to represent abinary one (offset) and another to represent a binaryzero (offset + delta). Both delay times are below thethreshold at which the human ear can resolve theecho. In addition to decreasing the delay time, we canalso ensure that the information is not perceivable bysetting the initial amplitude and the decay rate belowthe audible threshold of the human ear.

Encoding. The encoding process can be representedas a system that has one of two possible system func-tions. In the time domain, the system functions arediscrete time exponentials (see Figure 17) differingonly in the delay between impulses.

For simplicity, we chose an example with only twoimpulses (one to copy the original signal and one tocreate an echo). Increasing the number of impulses iswhat increases the number of echoes.

We let the kernel shown in Figure 18A represent thesystem function for encoding a binary one and we usethe system function defined in Figure 18B to encode azero. Processing a signal through either Figures 18Aor 18B will result in an encoded signal (see Figure19).

The delay (δb) between the original signal and theecho is dependent on which kernel or system functionwe use in Figure 19. The “one” kernel (Figure 18A) iscreated with a delay of (δ1) seconds while the “zero”kernel (Figure 18B) has a (δ0) second delay.

In order to encode more than one bit, the original sig-nal is divided into smaller portions. Each individualportion can then be echoed with the desired bit byconsidering each as an independent signal. The finalencoded signal (containing several bits) is the recom-bination of all independently encoded signal portions.

In Figure 20, the example signal has been divided intoseven equal portions labeled a, b, c, d, e, f, and g. Wewant portions a, c, d, and g to contain a one. There-

Figure 17 Discrete time exponential

Figure 18 Echo kernels

(A) “ONE” KERNEL (B) “ZERO” KERNEL

δ1 δ0

Figure 16 Adjustable parameters

ORIGINAL SIGNAL

“ZERO”

“ONE”

OFFSET DELTA

DECAY RATE

INITIAL AMPLITUDE

Page 17: Techniques for Data Hiding

IBM SYSTEMS JOURNAL, VOL 35, NOS 3&4, 1996 BENDER ET AL. 329

fore, we use the “one” kernel (Figure 18A) as the sys-tem function for each of these portions. Each portionis individually convolved with the system function.The zeros encoded into sections b, e, and f areencoded in a similar manner using the “zero” kernel(Figure 18B). Once each section has been individuallyconvolved with the appropriate system function, theresults are recombined. To achieve a less noticeablemix, we create a “one” echo signal by echoing theoriginal signal using the “one” kernel. The “zero” ker-nel is used to create the “zero” echo signal. Theresulting signals are shown in Figure 21.

The “one” echo signal and the “zero” echo signal con-tain only ones and zeros, respectively. In order tocombine the two signals, two mixer signals (see Fig-ure 22) are created. The mixer signals are either oneor zero depending on the bit we would like to hide inthat portion of the original signal.

The “one” mixer signal is multiplied by the “one”echo signal while the “zero” mixer signal is multi-plied by “zero” echo signal. In other words, the echosignals are scaled by either 1 or 0 throughout the sig-nal depending on what bit any particular portion issupposed to contain. Then the two results are added.Note that the “zero” mixer signal is the complementof the “one” mixer signal and that the transitionswithin each signal are ramps. The sum of the twomixer signals is always one. This gives us a smoothtransition between portions encoded with differentbits and prevents abrupt changes in the resonance ofthe final (mixed) signal.

A block diagram representing the entire encoding pro-cess is illustrated in Figure 23.

Decoding. Information is embedded into a signal byechoing the original signal with one of two delay ker-nels. A binary one is represented by an echo kernelwith a (δ1) second delay. A binary zero is representedby a (δ0) second delay. Extraction of the embeddedinformation involves the detection of spacing betweenthe echoes. In order to do this, we examine the magni-tude (at two locations) of the autocorrelation of theencoded signal’s cepstrum:14

(16)

The following procedure is an example of the decod-ing process. We begin with a sample signal that is aseries of impulses such that the impulses are separatedby a set interval and have exponentially decaying

F 1– lncomplex F x( )( )2( )

Figure 19 Echoing example

Figure 20 Divide the original signal into smallerportions to encode information

Figure 21 The first step in encoding process is tocreate a “one” and a “zero” echo signal(purple line is the echoed signal)

Figure 22 Mixer signals

h(t)

δbδb

ORIGINAL SIGNAL OUTPUT

a b c d e f g

1 0 1 1 0 0 1

a b c d e f g

1 0 1 1 0 0 1

1 0 1 1 0 0 1

δ1

δ0

a b c d e f g

1 0 1 1 0 0 1

0

1

0

1

“ONE” MIXER SIGNAL

“ZERO” MIXER SIGNAL

Page 18: Techniques for Data Hiding

BENDER ET AL. IBM SYSTEMS JOURNAL, VOL 35, NOS 3&4, 1996330

amplitudes. The signal is zero elsewhere (see Figure24).

The next step is to find the cepstrum14 of the echoedversion. The result of taking the cepstrum makes thespacing between the echo and the original signal a lit-tle clearer.

Unfortunately, the result of the cepstrum also dupli-cates the echo every (δ) seconds. In Figure 25, this isillustrated by the impulse train in the output. Further-more, the magnitude of the impulses representing theechoes are small relative to the original signal. Assuch, they are difficult to detect. The solution to thisproblem is to take the autocorrelation of the cepstrum.

We echo the signal once with delay (δ) using the ker-nel depicted in Figure 26. The result is illustrated inFigure 27.

Only the first impulse is significantly amplified as it isreinforced by subsequent impulses. Therefore, we geta spike in the position of the first impulse. Like thefirst impulse, the spike is either (δ1) or (δ0) secondsafter the original signal. The remainder of theimpulses approach zero. Conveniently, random noisesuffers the same fate as all the impulses after the first.

The rule for deciding on a one or a zero is based onthe time delay between the original signal and thedelay (δ) before the spike in the autocorrelation.

Figure 23 Encoding process

Figure 24 Example signal: x [n] = anu [n]; (0 < a < 1)

“ONE”KERNEL

“ZERO”KERNEL

ORIGINALSIGNAL

ENCODEDSIGNAL

“ONE”MIXER

“ZERO”MIXER

+

a

a2

a3

a4

Figure 25 Cepstrum of the echo-encoded signal

δ δ δ δ δ

CEPSTRUM

CEPSTRUM

ECHO KERNEL

ORIGINAL SIGNAL

CEPSTRUM OF ENCODED SIGNAL

Page 19: Techniques for Data Hiding

IBM SYSTEMS JOURNAL, VOL 35, NOS 3&4, 1996 BENDER ET AL. 331

Recall that a one was encoded by placing an echo (δ1)seconds after the original and a zero was placed (δ0)seconds after the original. When decoding, we assigna one if the magnitude of the autocorrelation functionis greater at (δ1) seconds than it is at (δ0) seconds. Azero is assigned if the reverse is true. This is the sameas deciding which kernel we used utilizing the factthat the “one” and “zero” kernel differ only in thedelay before the echo (Figure 18).

Results. Using the methods described, it is indeedpossible to encode and decode information in theform of binary digits into a media stream with mini-mal alteration to the original signal at approximately16 bps (see Figure 28). By minimal alteration, wemean that the output of the encoding process ischanged in such a way so that the average human can-not hear any significant difference between the alteredand the original signal. There is little, if any, degrada-tion of the original signal. Instead, the addition of res-onance simply gives the signal a slightly richer sound.While the addition of resonance may be problematicin some music applications, studio engineers may beable to fine-tune the echo hiding parameters duringthe mastering process, enabling its use.

Supplemental techniques

Three supplemental techniques are discussed next.

Adaptive data attenuation. The optimum attenua-tion factor varies as the noise level of the host soundchanges. By adapting the attenuation to the short-termchanges of the sound or noise level, we can keep thecoded noise extremely low during the silent segmentsand increase the coded noise during the noisy seg-ments. In our experiments, the quantized magnitudeenvelope of the host sound wave is used as a referencevalue for the adaptive attenuation; and the maximumnoise level is set to 2 percent of the dynamic range ofthe host signal.

Redundancy and error correction coding. In orderto compensate for errors due to channel noise and hostsignal modification, it is useful to apply error-correc-tion coding (ECC)1 to the data to be embedded. Whilethere exist some efficient methods ofECC, its applica-tion always results in a trade-off between robustnessand data rate.

Sound context analysis. The detectability of whitenoise inserted into a host audio signal is linearlydependent upon the original noise level of the host

signal. To maximize the quantity of embedded data,while ensuring the data are unnoticed, it is useful toexpress the noise level quantitatively. The noise levelis characterized by computing the magnitude ofchange in adjacent samples of the host signal:

(17)σlocal2 1

Smax

------------ 1N----× s n 1+( ) s n( )–[ ]2

n 1=

N 1–

∑×=

Figure 26 Echo kernel used in example

Figure 27 Echoed version of the example signal

δ

11

δ

Table 2 Audio noise level analysis

σ2local Quality

< 0.005> 0.01

StudioCrowd noise

Page 20: Techniques for Data Hiding

BENDER ET AL. IBM SYSTEMS JOURNAL, VOL 35, NOS 3&4, 1996332

where N is the number of sample points in thesequence andSmax is the maximum magnitude in thesequence. We use this measure to categorize hostaudio signals by noise level (see Table 2).

Data hiding in text

Soft-copy text is in many ways the most difficult placeto hide data. (Hard-copy text can be treated as ahighly structured image and is readily amenable to avariety of techniques such as slight variations in letterforms, kerning, baseline, etc.) This is due largely tothe relative lack of redundant information in a text fileas compared with a picture or a sound bite. While it isoften possible to make imperceptible modifications to

a picture, even an extra letter or period in text may benoticed by a casual reader. Data hiding in text is anexercise in the discovery of modifications that are notnoticed by readers. We considered three major meth-ods of encoding data: open space methods that encodethrough manipulation of white space (unused spaceon the printed page), syntactic methods that utilizepunctuation, and semantic methods that encode usingmanipulation of the words themselves.

Open space methods.There are two reasons why themanipulation of white space in particular yields usefulresults. First, changing the number of trailing spaceshas little chance of changing the meaning of a phraseor sentence. Second, a casual reader is unlikely to take

Figure 28 Result of autocepstrum and autocorrelation for (A) “zero” and (B) “one” bits

AUTOCEPSTRUMAUTOCORRELATIONCEPSTRUM

AM

PLI

TU

DE

TIME (SECONDS)

(A) ZERO (FIRST BIT)

18

16

6

4

2

00 0.0004 0.0006 0.0008 0.00100.0002 0.0012 0.0014 0.0016 0.0018 0.0020

TIME (SECONDS)

(B) ONE (FIRST BIT)

AUTOCEPSTRUMAUTOCORRELATIONCEPSTRUM

AM

PLI

TU

DE

18

16

6

4

2

00 0.0004 0.0006 0.0008 0.00100.0002 0.0012 0.0014 0.0016 0.0018 0.0020

Page 21: Techniques for Data Hiding

IBM SYSTEMS JOURNAL, VOL 35, NOS 3&4, 1996 BENDER ET AL. 333

notice of slight modifications to white space. Wedescribe three methods of using white space to encodedata. The methods exploit inter-sentence spacing,end-of-line spaces, and inter-word spacing in justifiedtext.

The first method encodes a binary message into a textby placing either one or two spaces after each termi-nating character, e.g., a period for English prose, asemicolon for C-code, etc. A single space encodes a“0,” while two spaces encode a “1.” This method has anumber of inherent problems. It is inefficient, requir-ing a great deal of text to encode a very few bits. (Onebit per sentence equates to a data rate of approxi-mately one bit per 160 bytes assuming sentences areon average two 80-character lines of text.) Its abilityto encode depends on the structure of the text. (Sometext, such as free-verse poetry, lacks consistent orwell-defined termination characters.) Many word pro-cessors automatically set the number of spaces afterperiods to one or two characters. Finally, inconsistentuse of white space is not transparent.

A second method of exploiting white space to encodedata is to insert spaces at the end of lines. The data areencoded allowing for a predetermined number ofspaces at the end of each line (see Figure 29). Twospaces encode one bit per line, four encode two, eightencode three, etc., dramatically increasing the amountof information we can encode over the previousmethod. In Figure 29, the text has been selectivelyjustified, and has then had spaces added to the end oflines to encode more data. Rules have been added toreveal the white space at the end of lines. Additionaladvantages of this method are that it can be done withany text, and it will go unnoticed by readers, since thisadditional white space is peripheral to the text. Aswith the previous method, some programs, e.g.,“sendmail,” may inadvertently remove the extra spacecharacters. A problem unique to this method is thatthe hidden data cannot be retrieved from hard copy.

A third method of using white space to encode datainvolves right-justification of text. Data are encodedby controlling where the extra spaces are placed. Onespace between words is interpreted as a “0.” Twospaces are interpreted as a “1.” This method results inseveral bits encoded on each line (see Figure 30).Because of constraints upon justification, not everyinter-word space can be used as data. In order todetermine which of the inter-word spaces representhidden data bits and which are part of the originaltext, we have employed a Manchester-like encodingmethod. Manchester encoding groups bits in sets oftwo, interpreting “01” as a “1” and “10” as a “0.” Thebit strings “00” and “11” are null. For example, theencoded message “1000101101” is reduced to “001,”while “110011” is a null string.

Open space methods are useful as long as the textremains in anASCII (American Standard CharacterInterchange) format. As mentioned above, some datamay be lost when the text is printed. Printed docu-ments present opportunities for data hiding far beyondthe capability of anASCII text file. Data hiding in hardcopy is accomplished by making slight variations inword and letter spacing, changes to the baseline posi-tion of letters or punctuation, changes to the letterforms themselves, etc. Also, image data-hiding tech-niques such as those used by Patchwork can be modi-fied to work with printed text.

Syntactic methods.That white space is consideredarbitrary is both its strength and its weakness wheredata hiding is concerned. While the reader may notnotice its manipulation, a word processor may inad-vertently change the number of spaces, destroying thehidden data. Robustness, in light of document refor-matting, is one reason to look for other methods ofdata hiding in text. In addition, the use of syntacticand semantic methods generally does not interferewith the open space methods. These methods can beapplied in parallel.

Figure 29 Example of data hidden using white space

T h e q u i c k b r o w n f o xj u m p s o v e r t h e l a z yd o g .

T h e q u i c k b r o w n f o xj u m p s o v e r t h e l a z yd o g .

NORMAL TEXT WHITE SPACE ENCODED TEXT

T h e q u i c k b r o w n f o xj u m p s o v e r t h e l a z yd o g .

T h e q u i c k b r o w n f o xj u m p s o v e r t h e l a z yd o g .

Page 22: Techniques for Data Hiding

BENDER ET AL. IBM SYSTEMS JOURNAL, VOL 35, NOS 3&4, 1996334

There are many circumstances where punctuation isambiguous or when mispunctuation has low impacton the meaning of the text. For example, the phrases“bread, butter, and milk” and “bread, butter and milk”are both considered correct usage of commas in a list.We can exploit the fact that the choice of form is arbi-trary. Alternation between forms can represent binarydata, e.g., anytime the first phrase structure (charac-terized by a comma appearing before the “and”)occurs, a “1” is inferred, and anytime the secondphrase structure is found, a “0” is inferred. Otherexamples include the controlled use of contractionsand abbreviations. While written English affordsnumerous cases for the application of syntactic datahiding, these situations occur infrequently in typicalprose. The expected data rate of these methods is onthe order of only several bits per kilobyte of text.

Although many of the rules of punctuation are ambig-uous or redundant, inconsistent use of punctuation isnoticeable to even casual readers. Finally, there arecases where changing the punctuation will impact theclarity, or even meaning, of the text considerably. Thismethod should be used with caution.

Syntactic methods include changing the diction andstructure of text without significantly altering mean-

ing or tone. For example, the sentence “Before thenight is over, I will have finished” could be stated “Iwill have finished before the night is over.” Thesemethods are more transparent than the punctuationmethods, but the opportunity to exploit them is lim-ited.

Semantic methods.A final category of data hiding intext involves changing the words themselves. Seman-tic methods are similar to the syntactic method.Rather than encoding binary data by exploiting ambi-guity of form, these methods assign two synonymsprimary or secondary value. For example, the word“big” could be considered primary and “large” sec-ondary. Whether a word has primary or secondaryvalue bears no relevance to how often it will be used,but, when decoding, primary words will be read asones, secondary words as zeros (see Table 3).

Word webssuch as WordNet can be used to automati-cally generate synonym tables. Where there are manysynonyms, more than one bit can be encoded per sub-stitution. (The choice between “propensity,” “predi-lection,” “penchant,” and “proclivity” represents twobits of data.) Problems occur when the nuances ofmeaning interfere with the desire to encode data. Forexample, there is a problem with choice of the syn-onym pair “cool” and “chilly.” Calling someone“cool” has very different connotations than callingthem “chilly.” The sentence “The students in line forregistration are spaced-out” is also ambiguous.

Applications

Data hidden in text has a variety of applications,including copyright verification, authentication, andannotation. Making copyright information inseparable

Figure 30 Data hidden through justification (text from A Connecticut Yankee in King Arthur’s Court by Mark Twain)

01→0

10→1

This distressed the monks and terrified them. They werenot used to hearing these awful beings called names, andthey did not know what might be the consequence. Therewas a dead silence now; superstitious bodings were inevery mind. The magician began to pull his wits together,and when he presently smiled an easy, nonchalant smile, itspread a mighty relief around; for it indicated that hismood was not destructive.

Table 3 Synonymous pairs

≈≈≈≈≈

largelittlecoolcleverstretched

bigsmallchilly

smartspaced

Page 23: Techniques for Data Hiding

IBM SYSTEMS JOURNAL, VOL 35, NOS 3&4, 1996 BENDER ET AL. 335

from the text is one way for publishers to protect theirproducts in an era of increasing electronic distribu-tion. Annotation can be used for tamper protection.For example, if a cryptographic hash of the paper isencoded into the paper, it is a simple matter to deter-mine whether or not the file has been changed. Verifi-cation is among the tasks that could easily beperformed by a server which, in this case, wouldreturn the judgment “authentic” or “unauthentic” asappropriate.

Other uses of data hiding in text involve embeddinginstructions for an autonomous program in a text. Forexample, a mail server can be programmed to checkfor hidden messages when transmitting an electronicmessage. The message is rejected or approveddepending on whether or not any hidden data arefound. In this way a company running its own mailserver can keep confidential documents from beinginadvertently exported.

Conclusion

In this paper, several techniques are discussed as pos-sible methods for embedding data in host text, image,and audio signals. While we have had some degree ofsuccess, all of the proposed methods have limitations.The goal of achieving protection of large amounts ofembedded data against intentional attempts at removalmay be unobtainable.

Automatic detection of geometric and nongeometricmodifications applied to the host signal after data hid-ing is a key data-hiding technology. The optimumtrade-offs between bit rate, robustness, and perceiv-ability need to be defined experimentally. The interac-tion between various data-hiding technologies needsto be better understood.

While compression of image and audio content con-tinues to reduce the necessary bandwidth associatedwith image and audio content, the need for a bettercontextual description of that content is increasing.Despite its current shortcomings, data-hiding technol-ogy is important as a carrier of these descriptions.

Acknowledgments

This work was supported in part by the News in theFuture research consortium at theMIT Media Labora-tory and International Business Machines Corpora-tion.

Cited references

1. P. Sweene,Error Control Coding (An Introduction), Prentice-Hall International Ltd., Englewood Cliffs, NJ (1991).

2. E. Adelson,Digital Signal Encoding and Decoding Apparatus,U.S. Patent No. 4,939,515 (1990).

3. R. Machado, “Stego,” http://www.nitv.net/~mech/Romana/stego.html (1994).

4. W. Bender, “Data Hiding,” News in the Future, MIT MediaLaboratory, unpublished lecture notes (1994).

5. A. Lippman, Receiver-Compatible Enhanced EDTV System,U.S. Patent No. 5,010,405 (1991).

6. D. L. Hecht, “Embedded Data Glyph Technology for HardcopyDigital Documents,”SPIE2171(1995).

7. K. Matsui and K. Tanaka, “Video-Steganography: How toSecretly Embed a Signature in a Picture,”IMA IntellectualProperty Project Proceedings (1994).

8. R. C. Dixon,Spread Spectrum Systems, John Wiley & Sons,Inc., New York (1976).

9. S. K. Marvin,Spread Spectrum Handbook, McGraw-Hill, Inc.,New York (1985).

10. Digimarc Corporation,Identification/Authentication CodingMethod and Apparatus,U.S. Patent (1995).

11. I. Cox, J. Kilian, T. Leighton, and T. Shamoon, “Secure SpreadSpectrum Watermarking for Multimedia,”NECI TechnicalReport 95-10, NEC Research Institute, Princeton, NJ (1995).

12. A. V. Drake,Fundamentals of Applied Probability, McGraw-Hill, Inc., New York (1967).

13. L. R. Rabiner and R. W. Schaffer,Digital Processing of SpeechSignal, Prentice-Hall, Inc., Englewood Cliffs, NJ (1975).

14. A. V. Oppenheim and R. W. Shaffer,Discrete-Time Signal Pro-cessing, Prentice-Hall, Inc., Englewood Cliffs, NJ (1989).

Accepted for publication February 29, 1996.

Walter Bender MIT Media Laboratory, 20 Ames Street, Cam-bridge, Massachusetts 02139-4307 (electronic mail: [email protected]). Mr. Bender is a principal research scientist at theMIT Media Laboratory and principal investigator of the labora-tory’s News in the Future consortium. He received the B.A. degreefrom Harvard University in 1977 and joined the ArchitectureMachine Group at MIT in 1978. He received the M.S. degree fromMIT in 1980. Mr. Bender is a founding member of the Media Labo-ratory.

Daniel Gruhl MIT Media Laboratory, 20 Ames Street, Cambridge,Massachusetts 02139-4307 (electronic mail: [email protected]). Mr.Gruhl is a doctoral student in the department of electrical engineer-ing and computer science at MIT, where he earned an S.B. in 1994and an MEng in 1995. He is an AT&T fellow and research assistantat the Media Lab, where his research interests include informationhiding in images, sound, and text, as well as user modeling for elec-tronic information systems.

Norishige Morimoto IBM Tokyo Research Laboratory, 1623-14Shimo-Tsuruma, Yamato-shi, Kanagawa-ken, 242 Japan (electronicmail: [email protected]). Mr. Morimoto is currently a researcherat the Tokyo Research Laboratory (TRL) of IBM Japan. He joinedIBM in 1987 and was transfered to the TRL in 1995 after gettinghis master’s degree from MIT. His area of interest is rights manage-ment of digital contents and network applications for industry solu-tions. Mr. Morimoto received his B.S. degree in electricalengineering from Keio University, and his M.S. degree in electricalengineering and computer science from the Massachusetts Instituteof Technology.

Page 24: Techniques for Data Hiding

BENDER ET AL. IBM SYSTEMS JOURNAL, VOL 35, NOS 3&4, 1996336

Anthony Lu MIT Media Laboratory, 20 Ames Street, Cambridge,Massachusetts 02139-4307 (electronic mail: [email protected]). Mr. Lu is an undergraduate student in electrical engineering atMIT, concentrating in communications and signal processing. Hewill be a candidate for the master of engineering degree in 1997.

Reprint Order No. G321-5608.