-
20 IEEE SIGNAL PROCESSING MAGAZINE SEPTEMBER 2000
A State-of-the-Art Overview
In the past decade there has been an explosion in theuse and
distribution of digital multimedia data. PCswith Internet
connections have taken homes bystorm and have made the distribution
of multimediadata and applications much easier andfaster.
Electronic commerce applica-tions and on-line services are
rapidlybeing developed. Even the analog au-dio and video equipment
in the homeis in the process of being replaced bytheir digital
successors. As a result, wecan see the digital mass recording
devices for multimediadata enter the consumer market of today.
The Need for WatermarkingAlthough digital data has many
advantages over analogdata, service providers are reluctant to
offer services in dig-ital form because they fear unrestricted
duplication anddissemination of copyrighted material. Because of
possi-ble copyright issues, the intellectual property of
digitallyrecorded material must be protected [90]. The lack ofsuch
adequate protection systems for copyrighted contentwas the reason
for the delayed introduction of the digitalversatile disk (DVD)
[100]. Several media companies ini-tially refused to provide DVD
material until the copy pro-tection problem had been addressed
[89], [81].Representatives of the consumer electronics industry
andthe motion picture industry have agreed to seek
legislationconcerning digital video recording devices.
Recommen-dations describing ways that would protect both
intellec-tual property and consumers rights have been submittedto
the U.S. Congress [81] and resulted in the Digital Mil-lennium
Copyright Act [25], which was signed by Presi-dent Clinton on
October 28, 1998. The European Unionis also preparing such
intellectual property rights protec-tion for digital multimedia
products including CDs orDVDs [28].
To provide copy protection and copyright protectionfor digital
audio and video data, two complementary tech-niques are being
developed: encryption and watermark-ing [23]. Encryption techniques
can be used to protect
digital data during the transmissionfrom the sender to the
receiver [63].After the receiver has received and de-crypted the
data, however, the data isidentical to the original data and
nolonger protected. Watermarkingtechniques can compliment
encryp-
tion by embedding a secret imperceptible signal, a water-mark,
directly into the original data in such a way that italways remains
present. Such a watermark, for instance,can be used for the
following purposes: Copyright Protection: For the protection of
intellectualproperty, the data owner can embed a watermark
repre-senting copyright information in his data. This watermarkcan
prove his ownership in court when someone has in-fringed on his
copyrights. Fingerprinting: To trace the source of illegal copies,
theowner can use a fingerprinting technique. In this case, theowner
can embed different watermarks in the copies of thedata that are
supplied to different customers. Finger-printing can be compared to
embedding a serial numberthat is related to the customers identity
in the data. It en-ables the intellectual property owner to
identify customerswho have broken their license agreement by
supplying thedata to third parties. Copy Protection: The
information stored in a watermarkcan directly control digital
recording devices for copy pro-tection purposes [62]. In this case,
the watermark repre-sents a copy-prohibit bit and watermark
detectors in therecorder determine whether the data offered to the
re-corder may be stored or not. Broadcast Monitoring: By embedding
watermarks incommercial advertisements, an automated
monitoringsystem can verify whether advertisements are
broadcasted
1053-5888/00/$10.002000IEEE
Gerhard C. Langelaar,Iwan Setyawan, and
Reginald L. Lagendijk
-
as contracted [3]. Not only commercials but also valuableTV
products can be protected by broadcast monitoring[53]. News items
can have a value of over US$100,000per hour, which make them very
vulnerable to intellectualproperty rights violation. A broadcast
surveillance systemcan check all broadcast channels and charge the
TV sta-tions according to their findings. Data Authentication:
Fragile watermarks [108] can beused to check the authenticity of
the data. A fragile wa-termark indicates whether the data has been
altered andsupplies localization information as to where the
datawas altered.
Watermarking techniques are not only used for pro-tection
purposes. Other applications include: Indexing: Indexing of video
mail, where comments canbe embedded in the video content; indexing
of moviesand news items, where markers and comments can be
in-serted that can be used by search engines. Medical Safety:
Embedding the date and the patientsname in medical images could be
a useful safety mea-sure [3]. Data Hiding: Watermarking techniques
can be usedfor the transmission of secret private messages. Since
vari-ous governments restrict the use of encryption services,people
may hide their messages in other data.
Some authors, for example in [11], refer to water-marking
technique only when the application embeds afew bits (as few as one
bit) of data for copyright no-tice/protection applications. Other
applications are con-sidered to fall into the category of data
embedding. Weprefer to use the term watermarking, however, for
allthese applications in this article. In our opinion,watermarking
has nowadays been used for applicationsbeyond the limits of copy
protection/authentication, anexample of which is Digimarcs Smart
Images [1].
Watermarking RequirementsEach watermarking application has its
own specific re-quirements. Therefore, there is no set of
requirements tobe met by all watermarking techniques.
Nevertheless,some general directions can be given for most of the
ap-plications mentioned above: Perceptual Transparency: In most
applications thewatermarking algorithm must embed the watermarksuch
that this does not affect the quality of the underlyinghost data. A
watermark-embedding procedure is trulyimperceptible if humans
cannot distinguish the originaldata from the data with the inserted
watermark [97].Even the smallest modification in the host data may
be-come apparent, however, when the original data is com-pared
directly with the watermarked data. Since users ofwatermarked data
normally do not have access to theoriginal data, they cannot
perform this comparison.Therefore, it may be sufficient that the
modifications inthe watermarked data go unnoticed as long as the
data arenot compared with the original data [103].
Payload of the Watermark: The amount of informationthat can be
stored in a watermark depends on the applica-tion. For copy
protection purposes, a payload of one bit isusually sufficient.
According to a recent proposal for audio watermark-ing
technology from the International Federation forthe Phonographic
Industry (IFPI), the minimum pay-load for an audio watermark should
be 20 bits per sec-ond, independently of the signal level and music
type[46]. According to [75], however, this minimum isvery ambitious
and should be lowered to only a few bitsper second.
For the protection of intellectual property rights, itseems
reasonable to assume that one wants to embed anamount of
information similar to that used for ISBN, In-ternational Standard
Book Numbering (roughly 10 dig-its) or better ISRC, International
Standard RecordingCode (roughly 12 alphanumeric letters). On top of
this,one should also add the year of copyright, the
permissionsgranted on the work, and the rating for it [59].
Thismeans that about 60 bits [31] or 70 bits [59] of informa-tion
should be embedded in the host data, the image, thevideo frame, or
the audio fragment.
Another important concept regarding watermark pay-load for
digital audio and video is watermark granularity.Watermark
granularity represents how much data isneeded to embed one unit of
watermark information.Using the example above, one unit of
watermark infor-mation consists of 60 or 70 bits. This could be
embeddedin a single frame of video or spread, for instance, over
100frames of video (or similarly for audio, the watermarkcould be
embedded in a 1-s fragment or spread for in-stance over 5 s of
audio data). Spreading the watermark inthis way may not be
desirable because when someonetakes just 80 frames from the
watermarked video, the wa-termark information is no longer
retrievable. For digitalvideos, 1 s of video is considered to be
the smallest copy-righted entity. Therefore, the watermark
information hasto be embedded in a less than 1 s fragment of the
videostream (approximately 25 frames). Again using the exam-ple
above, the watermark bit rate should then be morethan 70 bits/s.
Robustness: A fragile watermark that has to prove theauthenticity
of the host data does not have to be robustagainst processing
techniques or intentional alterations ofthe host data, since
failure to detect the watermark provesthat the host data has been
modified and is no longer au-
SEPTEMBER 2000 IEEE SIGNAL PROCESSING MAGAZINE 21
A watermark-embeddingprocedure is imperceptible ifhumans cannot
distinguish theoriginal data from the data withthe inserted
watermark.
-
thentic. If a watermark is used for another application,however,
it is desirable that the watermark always re-mains in the host
data, even if the quality of the host datais degraded,
intentionally or unintentionally. Examples ofunintentional
degradations are applications involvingstorage or transmission of
data, where lossy compressiontechniques are applied to the data to
reduce bit rates andincrease efficiency. Other unintentional
quality-degrad-ing processing techniques include filtering,
re-sampling,digital-analog (D/A) and analog-digital (A/D)
conver-sion. On the other hand, a watermark can also be sub-jected
to processing solely intended to remove thewatermark [23]. In
addition, when many copies of thesame content exist with different
watermarks, as wouldbe the case for fingerprinting, watermark
removal is pos-sible because of collusion between several owners of
cop-ies. In general, there should be no way in which thewatermark
can be removed or altered without sufficientdegradation of the
perceptual quality of the host data soas to render it unusable.
Security: The security of watermarking techniques canbe interpreted
in the same way as the security of encryp-tion techniques.
Kerckhoffs assumption states that oneshould assume that the method
used to encrypt the data isknown to an unauthorized party and that
the securitymust lie in the choice of a key [69]. Hence a
watermark-ing technique is truly secure if knowing the exact
algo-rithms for embedding and extracting the watermark doesnot help
an unauthorized party to detect the presence ofthe watermark or
remove it [97].
Oblivious versus Nonoblivious Watermarking: In someapplications,
like copyright protection and data monitor-ing, watermark
extraction algorithms can use the originalunwatermarked data to
find the watermark. This is callednonoblivious watermarking [59].
In most other applica-tions, e.g., copy protection and indexing,
the water-mark-extraction algorithms do not have access to
theoriginal unwatermarked data. This renders the water-mark
extraction more difficult. Watermarking algorithmsof this kind are
referred to as public, blind, or obliviouswatermarking
algorithms.
The requirements listed above are all related to eachother. For
instance, a very robust watermark can be ob-tained by making many
large modifications to the hostdata for each bit of the watermark.
Large modifications inthe host data will be noticeable, however,
and many modi-fications per watermark bit will limit the
maximumamount of watermark bits that can be stored in a data
ob-ject. Hence, a tradeoff should be considered between
thedifferent requirements so that an optimal watermark foreach
application can be developed. The mutual dependen-cies between the
basic requirements are shown in Fig. 1.
The relation between the basic requirements for awell-designed
secure watermark is represented in Fig. 2.The perceptual impact
axis represents the quality degra-dation of the data due to
watermarking. The higher theperceptual impact, the worse the
quality degradation.The payload axis represents the amount of data
that couldbe embedded in the data. The robustness axis
representsthe ability of the watermarking system to resist
attacks.The security of a watermark influences the
robustnessenormously. If a watermark is not secure, it cannot
bevery robust.
Scope of the ArticleTo embed watermark information in host data,
water-mark embedding techniques apply minor modificationsto the
host data in a perceptually invisible manner, wherethe
modifications are related to the watermark informa-tion. The
watermark information can be retrieved after-wards from the
watermarked data by detecting thepresence of these
modifications.
A wide range of modifications in any domain can beused for
watermarking techniques. Prior to embeddingor extracting a
watermark, the host data can be converted,for instance, to the
spatial, the Fourier, the wavelet, thediscrete cosine transform or
even the fractal domain,where the properties of the specific
transform domainscan be exploited. In these domains modifications
can bemade, like least significant bit (LSB) modification,
noiseaddition, coefficient re-ordering, coefficient removal,warping
or morphing data parts, and block similaritiesenforcing. Further,
the impact of the modifications canbe minimized with the aid of
human visual models,whereas modifications can be adapted to the
anticipated
22 IEEE SIGNAL PROCESSING MAGAZINE SEPTEMBER 2000
Perceptual Transparency
Payload Robustness Security
Oblivious versus Nonoblivious
1. Mutual dependencies between the basic requirements.
Per
cept
ual I
mpa
ct
PayloadRo
bustn
ess
Oblivious
Nonoblivious
2. Illustration of the relation between the basic
requirementsfor a secure watermark.
-
post-processing techniques or to the compression formatof the
host data.
Since the most commonly used techniques use addi-tive noise for
watermark embedding and correlation tech-niques for watermark
detection, we discuss the obliviouscorrelation-based techniques
extensively in this article,together with all its possible
variations. Other oblivioustechniques are explained as well. The
cryptographic secu-rity of the methods described here lies in the
key that isused to generate a pseudorandom watermark pattern orto
pseudorandomly select image regions or coefficients toembed the
watermark. In general, the robustness of thewatermark against
processing techniques depends on theembedding depth and the amount
of information bits ofthe watermark.
The article is organized as follows. First we will
discussdigital watermarking techniques based on correlation inthe
next two sections. And then we will discuss digitalwatermarking
techniques that are not based on correla-tion. The last section
presents some conclusion of the arti-cle including a brief
discussion of recent developments inthe digital watermarking
area.
Correlation-BasedWatermarking TechniquesBasic Technique in the
Spatial DomainThe most straightforward way to add a watermark to
animage in the spatial domain is to add a pseudorandomnoise pattern
to the luminance values of its pixels. Manymethods are based on
this principle [91], [10], [76],[18], [36], [35], [77], [93],
[105], [61], [106], [113],[32], [107], [108], [53]. In general, the
pseudorandomnoise pattern consists of the integers {1,0,1},
however,also floating-point numbers can also be used. The patternis
generated based on a key using, for instance, seeds, lin-ear shift
registers or randomly shuffled binary images.The only constraints
are that the energy in the pattern is
more or less uniformly distributed and that the pattern isnot
correlated with the host image content. To create thewatermarked
image I x yW ( , ) the pseudorandom patternW x y( , ) is multiplied
by a small gain factor k and added tothe host image I x y( , ), as
illustrated in Fig. 3
I x y I x y k W x yW ( , ) ( , ) ( , )= + . (1)
To detect a watermark in a possibly watermarked im-age I x yW (
, ) we calculate the correlation between theimage I x yW ( , ) and
the pseudorandom noise patternW x y( , ). In general,W x y( , ) is
normalized to a zero meanbefore correlation. Pseudorandom patterns
generated us-ing different keys have very low correlation with
eachother. Therefore, during the detection process the corre-lation
value will be very high for a pseudorandom patterngenerated with
the correct key and would be very low
SEPTEMBER 2000 IEEE SIGNAL PROCESSING MAGAZINE 23
Multiply by GainFactor k
I x,y( ) I x,yW( )
k
W x,y( ): Pseudorandom Pattern {1,0,1}
3. Watermark embedding procedure.
Cor
rela
tion
Val
ue
300
250
200
150
100
50
0
500 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Random Number Seed
4. Correlation values for a pseudorandom pattern generatedwith
seed=10 correlated with pseudorandom patterns gener-ated with other
seeds.
To add a watermark to animage in the spatial domain,add a
pseudorandom noisepattern to the luminance valuesof its pixels.
False Positive
False Negative
0 T 2k [ ] [ ] I Iw w+
5. Watermark detection procedure.
-
otherwise. This is shown in Fig. 4. Here we have water-marked
the Lena image by adding a pseudorandom pat-tern generated using
with seed = 10 to the image. Figure4 shows the correlation values
of some pseudorandompatterns generated using seeds varying between
0 and 15to the watermarked image. It can be seen that the
correla-tion when the correct seed (10) is used is very high,while
the correlation when the wrong seeds are used arevery low.
During the detection process, it is common to set athreshold T
to decide whether the watermark is detectedor not. If the
correlation exceeds a certain threshold T, thewatermark detector
determines that image I x yW ( , )con-tains watermark W x y( ,
)
R T W x yT W x y
I x y W x yW>
< ( , ) ( , ) ( , )
( , )detected
No detected.(2)
IfW x y( , )only consists of the integers{ , }1 1 and if
thenumber of 1s equals the number of 1s, we can estimatethe
correlation as
RN
I x y W x y
NI W
I x y W x y W ii
N
W ii
W i
i
=
+
=
=
( , ) ( , ) ( , ) ( , )1
1
1
[ ] [ ]{ }=
=
+
+
=
1
2
1
21
12
N
W ii
N
W W
NI W
I x y I x y
i
/ /
( , ) ( , ) . (3)
Here N is the number of pixels in the image I W , and+,
indicates the set of pixels where the correspondingnoise pattern is
positive or negative, and [ ( , )]I x yW
+
represents the average value of set pixels in I x yW+ ( , ).
From (3) it follows that the watermark detection prob-lem
corresponds to testing the hypothesis whether tworandomly selected
sets of pixels in a watermarked imagehave the same mean.
During the detection process, the watermark detectorcan make two
types of errors. In the first place, it can de-
tect the existence of a watermark, although there is none.This
is called a false positive. In the second place, the detec-tor can
reject the existence of the watermark, even thoughthere is one.
This is called a false negative. The probabilityfunction for the
detection process is presented in Fig. 5.
In [52] the probabilities of these two types of errorsare
derived based on a first-order autoregressive imagemodel:
P T N
PT N
fpW I
fnW
W I
=
=
12 2
12
2
erfc and
erfc
( )
2
12
2 2
=
where erfc( ) ./x e dttx (4)
Here, Pfp represents the probability of false positive,Pfn
represents the probability of false negative, W
2 repre-sents the variance of the watermark pixels and I
2 denotesthe variance of the image pixels. If the watermark
patternW x y( , )only consists of the integers { , }1 1 and the
num-ber of -1s equals the number of 1s, the variance of the
wa-termark W
2 equals k2 . The errors Pfp and Pfn can beminimized by
increasing the gain factor k. Using larger
24 IEEE SIGNAL PROCESSING MAGAZINE SEPTEMBER 2000
I0
b0
I5
b5
I10
b10
I24
b24
I1
b1
I6
b6
I2
b2
I7
b7
I3
b3
I8
b8
I4
b4
I9
b9
k
I x,y( ) I x,yW( )
Random Pattern {1,0,1}
b=0
b=1
: 1
: 1
WM: b b b0 1 1L
6. Watermark bit string embedding procedure.
k P=2, =32 32
Without Prefilter Fedge
With Prefilter Fedge
% B
it E
rror
s
60
50
40
30
20
10
00 10 20 30 40 50 60 70 80 90 100
Qjpeg
7. Watermark detection with and without prefiltering.
% B
it E
rror
s
60
50
40
30
20
10
00 10 20 30 40 50 60 70 80 90 100
Qjpeg
k=2k=3
k=1
P=32 32, Prefilter Applied before DetectionFedge
8. Influence of the gain factor k on the robustness of a
water-mark.
-
values for the gain factor, however, decreases the visualquality
of the watermarked image.
Since the image content can interfere with the water-mark,
especially in the low-frequency components, thereliability of the
detector can be improved by applyingmatched filtering before
correlation [26], [91], [35].This decreases the contribution of the
original image tothe correlation. For instance, a simple
edge-enhancing fi-nite impulse response (FIR) filter Fedge can be
used,where Fedge is given by the following convolution kernel:
Fedge =
1 1 11 10 11 1 1
2/ .
(5)
The experimental results presented in the next sec-tion show
that applying thisfilter before correlation re-duces the error
probabilitysignificantly, even when thevisual quality of the
water-marked image was affected se-riously before correlation[35],
[61]. In [67], the au-thors proposed another wayto improve the
robustness ofthe watermark. The robust-ness improvement is
achievedby performing a spectrumequalization prior to water-mark
embedding.
Extensions to EmbedMultiple Bitsor Logos in One ImageFrom the
watermark detectorspoint of view, an image I can beregarded as
Gaussian noise,which distorts the watermarkinformation W. Further,
the
watermarked image I W can be seen as the output of
acommunication channel subject to Gaussian noise overwhich the
watermark information is transmitted. In thiscase, reliable
transmission of the watermark is theoreti-cally possible if its
information rate does not exceed thechannel capacity, which is
given by [92]
C WbW
I
= +
log 2
2
21
bit/pixel.
(6)
Here, C is given in units of watermark informationbits per image
pixel and the available bandwidth Wb isequal to one cycle per
pixel. For practical systems, how-ever, a tighter empirically lower
bound can be determined[93]
C WbW
I
= +
log 2
2
21
bit/pixel.
(7)
Here, is a small headroom factor, which is largerthan one and
typically around three. Since the sig-nal-to-noise ratio W I
2 2/ is significantly smaller thanone, (7) can be approximated
by
C W
I
1
2
2
2ln
bit/pixel.
(8)
According to this equation, it should be possible tostore much
more information in an image than just 1 bitusing the basic
technique described in the previous sec-tion. For instance, a
watermark consisting of the integers
SEPTEMBER 2000 IEEE SIGNAL PROCESSING MAGAZINE 25
RP0RPRPRPRPRPRP
1
2
3
4
5
6
+RP0+RP
RPRP
+RPRP
+RP
1
2
3
4
5
6
b0bbbbbb
1
2
3
4
5
6W
10. Example of a CDMA watermark generation for 7 bits b b b0 1 7
.
WI
IW
E [ ( E[ ] ) ( E[ ] ) ]RP RP I0 0 " W IWE [ ( E[ ] ) ( E[ ] )
]RP RP I1 1 " W IWE [ ( E[ ] ) ( E[ ] ) ]RP RP I2 2 " W IWE [ ( E[
] ) ( E[ ] ) ]RP RP I3 3 " W IWE [ ( E[ ] ) ( E[ ] ) ]RP RP I4 4 "
W IWE [ ( E[ ] ) ( E[ ] ) ]RP RP I5 5 " W IWE [ ( E[ ] ) ( E[ ] )
]RP RP I6 6 " W IW
b0 = 0
b1 = 0
b2 = 1
b3 = 1
b4 = 0
b5 = 1
b6 = 0
11. Example of CDMA watermark extraction, compare to Fig.
10.
% B
it E
rror
s60
50
40
30
20
10
00 10 20 30 40 50 60 70 80 90 100
Qjpeg
P=8 8
P=16 16
P=32 32P=64 64
P=128 128
k=2, Prefilter Applied before DetectionFedge
9. Influence of the number of pixels per watermark bit P on
therobustness of a watermark.
-
{ , }k k added to the 512 512 Lena image (Fig. 3) cancarry
approximately 50, 200, or 500 bits of informationfor k =1 2, , or 3
respectively and for = 3.
There are several ways to increase the payload of thebasic
watermarking technique. The simplest way to em-bed a string of l
watermark bits b b bl0 1 1 in an image is todivide the image I into
l subimages I I I l0 1 1 and to adda watermark to each subimage,
where each watermarkrepresents one bit of the string [93], [35],
[61]. This pro-cedure is depicted in Fig. 6.
Using (8) we can calculate the number of pixels P re-quired per
subimage for reliable de-tection of a single bit in a subimage
P IW
2
2
2lnpixels.
(9)
The watermark bits can be repre-sented in several ways. A
pseudoran-dom pattern can be added if thewatermark bit equals one,
and thesubimage can be left unaffected if thewatermark bit equals
zero. In thiscase, the detector calculates the corre-lation between
the subimage and thepseudorandom pattern and assignsthe value 1 to
the watermark bit if thecorrelation exceeds a certain thresh-old T;
otherwise the watermark bit isassumed to be zero.
The use of a threshold can be cir-cumvented by adding two
differentpseudorandom patterns RP0 and RP1for watermark bit 0 and
1. The detec-tor now calculates the correlation be-tween the
subimage and the twopatterns. The bit value correspondingwith the
pattern that gives the highestcorrelation is assigned to the
water-mark bit. In [93] the two patterns arechosen in such a way
that they onlydiffer in sign, RP RP0 1= . In this
case, the detector only has to calculate the correlation
be-tween the subimage and one of the patterns; the sign of
thecorrelation determines the watermark bit value.
To investigate the effect on the robustness of the wa-termark of
the prefilter in the detector, the gain factor k,and the number of
pixels P per watermark bit, we performthe following experiments. We
first add a watermark toan image with the method of [93]. Next, we
compress thewatermarked image with the JPEG algorithm [73],where
the quality factor Q jpeg of the compression algo-rithm is made
variable. Finally, the watermark is extractedfrom the decompressed
image and compared bit by bitwith the originally embedded watermark
bits. From thisexperiment, we find the percentages of watermark bit
er-rors due to JPEG compression as a function of the JPEGquality
factor.
The first experiment shows the effect of applying theprefilter
given by (5) before detection of a watermark em-bedded with a gain
factor k =2, and P = 32 32 pixels perwatermark bit. In Fig. 7 the
percentages bit errors causedby JPEG compression are plotted for a
detector that usesthis prefilter and for a plain detector. It can
clearly be seenthat prefiltering significantly increases the
robustness ofthe watermark.
The second experiment shows the effect of increasingthe gain
factor k for a watermark embedded with
26 IEEE SIGNAL PROCESSING MAGAZINE SEPTEMBER 2000
Extracted Logo from ImageCompressed with JPEG =50Q
Extracted Logo from ImageCompressed with JPEG =75Q
Extracted Logo from ImageCompressed with JPEG =90Q
Original EmbeddedWatermark Logo
12. Extracted watermark logos from a JPEG distorted image.
(a) (b)
(c) (d)
13. Fourier amplitude watermark. (a) Original image, (b)
watermarked image, (c) dif-ference W x y I Iw( , ) = scaled for
visibility, and (d) heavily marked image.
-
P = 32 32 pixels per watermark bit and detected using
aprefilter. From Fig. 8 it follows that the robustness of
awatermark can be improved significantly by increasingthe gain
factor.
The third experiment shows the influence of the num-ber of
pixels P per watermark bit on the robustness of awatermark embedded
with a gain factor k =2 and de-tected using a prefilter. From Fig.
9 it follows that de-creasing the payload of the watermark by
increasing Pimproves the robustness significantly.
Another way to increase the payload of the basicwatermarking
technique is the use of direct sequencecode division multiple
access (DS-CDMA) spread spec-trum communications [87], [88]. Here,
for each bit bjout of the watermark bit string b b bl0 1 1 a
differentstochastically independent pseudorandom pattern RPi
isgenerated that has the same size as the image. This patternis
dependent on the bit value bj . Here we use the pattern+RPi if bj
represents a 0 and RPi if bj represents a 1. Thesummation of all l
random patterns RPi forms the wa-termark. Prior to adding the
watermark to an image, wecan scale the watermark by a gain factor
or limit it to a cer-tain small range. An example of the
one-dimensional wa-termark generation is presented in Fig. 10. This
exampleuses seven different pseudorandompatterns to embed the seven
water-mark bits 0011010.
Each bit bj out of the watermarkbit string b b bl0 1 1 can be
extractedby calculating the correlation be-tween the normalized
image I W andthe corresponding pseudorandompattern RPi . If the
correlation is pos-itive, the value 0 is assigned to thewatermark
bit, otherwise the water-mark bit is assumed to be one. Fig-ure 11
shows as an example theextraction of the embedded water-mark bits
in Fig. 10.
The methods to extend the water-mark payload described
above,namely using individual image tilesfor each watermark bit and
usingCDMA, have their advantages anddisadvantages. If each
watermark bithas its own image tile, there is no in-terference
between the bits and only asmall number of multiplications
arerequired to calculate the correlations.If the image is cropped,
however, thewatermark bits located at the borderare lost. If CDMA
techniques areused, the probability that all bits canbe recovered
after cropping the imageis high. The watermark bits may inter-fere
with each other, however, andmany multiplications are required
to
calculate the correlations, since each bit is completelyspread
over the image.
The watermark bits embedded using the methodsmentioned above can
represent anything: copyright mes-sages, serial numbers, plain
text, control signals, etc. Thecontent represented by these bits
can be compressed, en-crypted, and protected by error correcting
codes. In somecases it may be more useful to embed a small logo
insteadof a bit string as a watermark. If the watermarked image
isdistorted, the watermark logo will also be affected. Butnow the
sophisticated pattern-recognition capabilities ofthe human visual
system (HVS) can be exploited to de-tect the logo [15], [45],
[102]. For instance, we can em-bed a binary watermark logo with 128
32 pixels in animage with 512 512 pixels using the techniques
de-scribed in this section. Each logo pixel is embedded in animage
tile of 8 8 pixels by adding the pseudorandompattern +RP or RP to
the image tile for a black or whitelogo pixel respectively. As an
example in Fig. 11 the re-sults are shown of the logos extracted
after the water-marked image has been degraded with the lossy
JPEG[73] compression algorithm using several quality factors.From
Fig. 12 it can be seen that, although it is heavily cor-rupted, the
logo can still be recognized.
SEPTEMBER 2000 IEEE SIGNAL PROCESSING MAGAZINE 27
(a)
(c)
(b)
fv
fh
(d)
14. An 8 8 DCT middle band image content independent watermark.
(a) Water-marked image, (b) a heavily watermarked image, (c)
differenceW x y I x y I x yw( , ) ( , ) ( , )= , and (d) Fourier
spectrum W u( , ) .
-
Techniques for Transform DomainsThe techniques described in the
previous section can alsobe applied on transformed image data. Each
transformdomain has it own advantages and disadvantages. In [85]the
phase of the discrete Fourier transform (DFT) is usedto embed a
watermark, because the phase is more impor-tant than the amplitude
of the DFT values for the intelli-gibility of an image. Putting a
watermark in the mostimportant components of an image improves the
robust-ness of the watermark, since tampering with these impor-tant
image components to remove the watermark willseverely degrade the
quality of the image. The second rea-son to use the phase of the
DFT values is that it is wellknown from communication theory that
phase modula-tion often possesses superior noise immunity in
compari-son with amplitude modulation [85].
Many watermarking techniques use DFT amplitudemodulation because
of its translation or shift invariantproperty [40], [41], [74],
[83], [86]-[88]. Because cy-clic translation of the image in the
spatial domain doesnot affect the DFT amplitude, the watermark
embeddedin this domain will be translation invariant. In case aCDMA
watermark is used, it is even slightly resistant tocropping.
Furthermore, the watermark can be embed-ded directly in the most
important middle band fre-quencies, since modulation of thelowest
frequency coefficients resultsin visible artifacts while the
highestfrequency coefficients are very vul-nerable to noise,
filtering, and lossycompression. Finally the watermarkcan easily be
made image content de-pendent by modulating the DFTamplitude
coefficients |I(u,v)| inthe following way [20]:
| | | | ( )I u v I u v k W u vW ( , ) ( , ) ( , )= + 1 .(10)
H e r e , W u v( , ) r e p r e s e n t s aCDMA watermark, a
two-dimen-sional (2-D) pseudorandom pat-tern, and k denotes the
gain factor.Now, the modification of a DFTcoefficient is not fixed
but propor-tional to the amplitude of the DFTcoefficient. Small DFT
coeffi-cients are hardly affected, whereaslarger DFT coefficients
are af-fected more severely. This com-plies with Webers law [50].
TheHVS does not perceive equalchanges in images equally, but
vi-sual sensitivity is nearly constantwith respect to relative
changes inan image. If I is a just noticeabledifference, then I I/
= constant.Rewriting (10) gives
| | | || | | |
I u v I u v
I u vI u v
I u vk W u vW
( , ) ( , )
( , )( , )( , )
( , )
= =
constant.(11)
Since the watermark here is mainly embedded in thelarger DFT
coefficients, i.e., the perceptually most signif-icant components
of the image, the robustness of the wa-termark improves.
Note that the symmetry of the Fourier coefficientsmust be
preserved to ensure that the image data is stillreal valued after
the inverse transform to the spatial do-main. If the coefficient |
|I u v( , ) in an image with N Mpixels is modified according to
(10), its counterpart
28 IEEE SIGNAL PROCESSING MAGAZINE SEPTEMBER 2000
y
x
Image I
FM
8 8 DCTv
u
15. Definition of the middle band frequencies in a DCT
block.
(a) (b)
fh
fv
(c) (d)
16. An 8 8 block DCT middle band image content dependent
watermark. (a) Water-marked image, (b) a heavily watermarked image,
(c) differenceW x y I x y I x yw( , ) ( , ) ( , )= , and (d)
Fourier spectrum W u( , ) .
-
| ( , )|I N u M v must be modified in the same way. InFig. 13(b)
an example is given of an image in which a wa-termark is embedded
using all DFT amplitude coeffi-cients according to (10) and using a
relatively small gainfactor k. Figure 13(c) presents the strongly
amplified dif-ference between the original image and the
watermarkedimage. Figure 13(d) shows an image watermarked usinga
large value of the gain factor k.
Another commonly used domain for embedding awatermark is the
discrete cosine transform (DCT) do-main [12], [20]-[22], [45],
[78], [79], [99], [84],[110]. Using the DCT an image can easily be
split up inpseudo frequency bands, so that the watermark
canconveniently be embedded in the most important mid-dle band
frequencies. Furthermore, the sensitivity ofthe HVS to the DCT
basis images has been extensivelystudied, which resulted in the
recommended JPEGquantization table [73]. These results can be used
forpredicting and minimizing the vi-sual impact of the distortion
causedby the watermark. Finally, theblock-based DCT is widely used
forimage and video compression. Byembedding a watermark in the
samedomain as the compression schemeused to process the image (in
thiscase in the DCT domain) we can an-ticipate lossy compression
becausewe are able to anticipate whichDCT coefficients will be
discardedby the compression scheme. Fur-thermore, we can exploit
the DCTdecomposition to make real-timewatermark applications.
In Fig. 14(a) an example is given ofan image in which a 2-D CDMA
wa-
termark W is embedded in the 8 8 block DCT middleband
frequencies. The 8 8 DCT coefficients F u v( , ) aremodulated
according to the following:
I u vI u v k W u v u v FI u v uW
x y x y M
x yx y,
( , )( , ) ( , ), ,( , ),
, ,
,
=+
,
, , , ,... .
v F
x yM
=1 8 16 (12)
Here FM denotes the middle band frequencies, k thegain factor, (
, )x y the spatial location of an 8 8 pixelblock in image I, and( ,
)u v the DCT coefficient in the cor-responding 8 8 DCT block (Fig.
15).
In Fig. 14(c) the strongly amplified difference be-tween the
original image and the watermarked image ispresented. Figure 14(d)
shows the Fourier spectrum ofthe watermark. Here, it can clearly be
seen that water-mark only affects the middle band frequencies
(white re-gions) while leaving lower and high frequencycomponents
relatively unaffected (dark regions).
The watermark can be made image dependent bychanging the
modulation function to [c.f. (10)]
I u vI u v k W u v u v FI uW
x y x y M
x yx y,
( , )( , ) ( ( , )), ,( ,
, ,
,
= + 1
v u v F
x yM), ,
, , , ,... .
=1 8 16 (13)
If this modulation function is applied, the results fromFig. 13
change into the results shown in Fig. 16. FromFig. 16(b) and (c) it
appears that most distortion intro-duced by the watermark is
located around the edges andin the textured areas.
Further improvements for DCT-domain correla-tion-based
watermarking systems performance couldbe achieved by using
watermark detectors based ongeneralized Gaussian model, instead of
the widely usedpure Gaussian assumption [42]. By performing a
theo-retical analysis for DCT-domain watermarking meth-ods for
images, the authors in [42] provide analytical
SEPTEMBER 2000 IEEE SIGNAL PROCESSING MAGAZINE 29
LL2 HL2
LH2 HH2
HL1
L 1H HH1
17. DWT two-level decomposition of an image.
(a) (b)
18. DWT image content independent watermark. (a) A heavily
watermarked image and(b) difference W x y I x y I x yw( , ) ( , ) (
, )= .
-
expressions which can be used to measure beforehandthe
performance that can be expected for a certain im-age and to
analyze the influence of the image character-istics and system
parameters (e.g., watermark length)on the final performance.
Furthermore, the result ofthis analysis can help determining the
proper detectionthreshold T to obtain a certain false positive
rate. Theauthors in [42] claim that by abandoning the pureGaussian
noise assumption, some substantial perfor-mance improvements could
be obtained.
If watermarking techniques can exploit the character-istics of
the HVS, it is possible to hide watermarks withmore energy in an
image, which makes watermarks morerobust. From this point of view
the discrete wavelet trans-form (DWT) is a very attractive
transform, because it canbe used as a computationally efficient
version of the fre-quency models for the HVS [7]. For instance, it
appearsthat the human eye is less sensitive to noise in high
resolu-tion DWT bands and in the DWT bands having an orien-tation
of 45 (i.e., HH bands). Furthermore, DWT imageand video coding,
such as embedded zero-tree wavelet(EZW) coding, will be included in
the upcoming imageand video compression standards, such as
JPEG2000[112]. By embedding a watermark in the same domain(DWT
domain) we can anticipate lossy EZW compres-sion because we can
anticipate which DWT bands is go-ing to be affected by the
compression scheme.Furthermore, we can exploit the DWT
decomposition tomake real-time watermark applications. Many
ap-proaches apply the basic techniques described at the be-ginning
of this section to the high resolution DWTbands, LH1 , HH1 , and
HL1 (Fig. 17) [7], [12], [56],[84], [112].
In Fig. 18(a) an example is given of an image in whicha 2-D CDMA
watermark W is embedded in the LH1 ,HH1 , and HL1 DWT bands using a
large gain factor k.The DWT coefficients in each of the three DWT
bandsare modulated as follows:
I u v I u v k W u vW ( , ) ( , ) ( , )= + . (14)
Figure 18(b) shows the stronglyamplified difference between
theoriginal image and the watermarkedimage.
The DWT watermark can be madeimage dependent by modulating
theDWT coefficients in each of the threeDWT bands as follows:
I u v I u v k W u vW ( , ) ( , ) ( ( , )).= + 1 (15)In Fig.
19(a) an example is given
of an image in which the sameCDMA watermark W is embeddedin the
LH1 , HH1 , and HL1 DWTbands using (15) with a large gainfactor k.
Figure 19(b) shows thestrongly amplified difference be-
tween the original image and the watermarked image.
Watermark Energy Adaptation Based on HVSThe robustness of a
watermark can be improved by in-creasing the energy of the
watermark. Increasing the en-ergy, however, degrades the image
quality. Byexploiting the properties of the HVS, the energy can
beincreased locally in places where the human eye will notnotice
it. As a result, by exploiting the HVS, one can em-bed perceptually
invisible watermarks that have higherenergy than if this energy
were to be distributed evenlyover the image.
If a visual signal is to be perceived, it must have a mini-mum
amount of contrast, which depends on its mean lu-minance and
frequency. Furthermore, a signal of a givenfrequency can mask a
disturbing signal of a similar fre-quency [104], [6]. This masking
effect is already used inthe image-dependent DCT watermarking
method de-scribed in the previous section, where the
DCT-coeffi-cients are modulated by means of (13). Here, to
eachsinusoid present in the image (masking signal), anothersinusoid
(watermark) is added, having an amplitude pro-portional to the
masking signal. If the gain factor k isproperly set, frequency
masking occurs.
The HVS is less sensitive to changes in regions of
highluminance. This fact can be exploited by making the wa-termark
gain factor luminance dependent [58]. Further-more, since the human
eye is least sensitive to the bluechannel, a perceptually invisible
watermark embedded inthe blue channel can contain more energy than
a percep-tually invisible watermark embedded in the
luminancechannel of a color image [58].
Around edges and in textured areas of an image, theHVS is less
sensitive to distortions than in smooth areas.This effect is called
spatial masking and can also be ex-ploited for watermarking by
increasing the watermarkenergy locally in these masked image areas
[68]. The basic
30 IEEE SIGNAL PROCESSING MAGAZINE SEPTEMBER 2000
(a) (b)
19. DWT image content dependent watermark. (a) A heavily
watermarked image and(b) difference W x y I x y I x yw( , ) ( , ) (
, )= .
-
SEPTEMBER 2000 IEEE SIGNAL PROCESSING MAGAZINE 31
spatial watermarking techniques described in the first
twosubsections of this section can be extended with spatialmasking
compensation, for instance, by using the follow-ing modulation
function:
I x y I x y Msk x y k W x yW ( , ) ( , ) ( , ) ( , )= + .
(16)
Here W x y( , ) represents the 2-D pseudorandom pat-tern of the
watermark, k denotes the fixed gain factor, andMsk x y( , )
represents a masking image. The values of themasking image range
from 0 to k max and give a measureof insensitivity to distortion
for each corresponding pointin the original image I x y( , ). In
[53] the masking imageMsk is generated by filtering the original
image with aLaplacian high-pass filter and by taking the absolute
val-ues of the resulting filtered image.
In Fig. 20(a) a mask is shown for the Lena image [Fig.13(a)]
which is generated by a simple Prewitt edge detec-tor [71]. Figure
20(b) shows the strongly amplified wa-termark modulated with this
mask.
In [70] the squared sum of the 8 8 DCT AC-coeffi-cients is used
to generate a maskingimage. Figure 21(a) shows a maskgenerated
using this DCT-ac energyfor the Lena image. Figure 21(b)presents
the strongly amplified wa-termark modulated with this mask.
Experiments have shown that aperceptually invisible
watermarkmodulated with a gain factor locallyadapted to such a mask
can containtwice as much energy as a perceptu-ally invisible
watermark modulatedwith a fixed gain factor.
To investigate the effect of thisenergy doubling on the
robustnessof the watermark, we perform thefollowing experiment. We
add a wa-termark W x yfixed ( , ) to the Lena im-age with the tiled
spread spectrumwatermarking method described in[93] using a fixed
gain factor k =2.Increasing this fixed gain factorcauses visible
artifacts in the result-ing watermarked image. Next, weadd a
watermark W x yvar ( , ) to an-other Lena image with the
samemethod, but now we use a variablegain factor locally adapted to
themasking image presented in Fig.19(a). Although the watermarkW x
yvar ( , ) contains about twice asmuch energy as W x yfixed ( , )
the wa-termark is not noticeable in the re-sulting watermarked
image. Thenwe compress both watermarked im-
ages with the JPEG algorithm [73], where the qualityfactor Q
jpeg of the compression algorithm is made vari-able. Finally, the
watermarks are extracted from the de-compressed image and compared
bit by bit with theoriginally embedded watermark bits. From this
experi-ment, we find the percentages of watermark bit errorsdue to
JPEG compression as a function of the JPEGquality factor. In Fig.
22 the error curves are plotted forboth watermarks W x yfixed ( , )
and W x yvar ( , ). It can beseen that the robustness can be
slightly improved by ap-plying a variable gain factor adapted to
the HVS.
Spatial masking can also be applied if the watermark isembedded
in another domain, e.g., DFT, DCT, or DWT.In this case, the
nonspatial watermark is first embedded inan image I, resulting in
the temporary image I Wt . Thewatermarked image I W is now
constructed by mixing theoriginal image I and this temporary image
I Wt by meansof a masking image Msk as described above [6],
[78]:
I x y Msk x y I x y Msk x y I x yW Wt( , ) ( ( , )) ( , ) ( , )
( , )= + 1 .
(17)
(a) (b)
20. Watermarking using masking image based on Prewitt operator.
(a) Masking imageand (b) difference W x y I x y I x yw( , ) ( , ) (
, )= .
(a) (b)
21. Watermarking where a masking image is used based on DCT-AC
energy. (a)Masking image and (b) difference W x y I x y I x yw( , )
( , ) ( , )= .
-
Here the masking image must be scaled to values in therange from
zero to one. Watermarking methods based onmore sophisticated models
for the HVS can be found in[6], [7], [30], [34], [56], [78], [79],
[94], [95], [109],and [110].
Extended Correlation-BasedWatermarking TechniquesAnticipating
Lossy Compression and FilteringWatermarks that have been embedded
in an image bymeans of the spatial watermarking techniques
earliercannot be detected reliably after the watermarked imagehas
been highly compressed with the lossy JPEG com-pression algorithm.
This is due to the fact that such wa-termarks consist essentially
of low-power, high-frequency noise. Since JPEG allocates fewer bits
to thehigher frequency components, such watermarks can eas-ily be
distorted. Furthermore, these watermarks can alsobe affected
severely by low-pass operations like linear ormedian filters.
The robustness to JPEG compression can be im-proved in several
ways. In [93] the pseudorandom pat-tern W is first compressed and
then decompressed usingthe JPEG algorithm. The energy of the
resulting patternW is increased to compensate for the energy
lostthrough the compression. Finally, this pattern is addedto the
image to generate the watermarked image. Theidea here is to use the
compression algorithm to filter
out in advance all the energy that would otherwise belost later
in the course of the compression. It is assumedthat a watermark
formed in this way is invariant to fur-ther JPEG compression that
uses the same quality fac-tor, except for small numerical
artifacts. Otherpredistortion of the watermark pattern, such as
filtering,can be applied to prevent other anticipated degradationof
the watermarked image.
In [72] the energy of the watermark pattern is shiftedto the
lower frequencies by calculating an individual gainfactor kx y, for
each pixel of the watermark pattern insteadof using the same gain
factor k for all pixels. First apseudorandom patternW x y( , ) is
generated consisting ofthe integers 0 and k. Next, the pattern is
divided into 8 8blocks, and the DCT transform W u v( , ) is
calculated foreach 8 8 block. The nonzero elements in the 8 8
blocksare now regarded as gain factors kx y, and are adapted insuch
a way that the energy in the vulnerable high fre-quency DCT bands
FH is minimized (Fig. 23):
{ }= = < <
W u v F u v u vu v F
HH
( , ) , | , .,
2 5 8 5 8
(18)
The energy is minimized under the following con-straints:
W x y k W x y k k kyx
x yyx
x y( , ) ( , ) ,, min , = == ==
1
8
1
8
1
8
1
8
kmax .
(19)
The effect of this high-energy minimization on the wa-termark
pattern is illustrated in Fig. 24. Figure 24(a)shows the watermark
pattern within an 8 8 block, wherea constant gain factor of k =3 is
used. After the high-en-ergy minimization with kmin =0 and kmax =6,
the water-mark pattern fades smoothly to zero [Fig. 24(b)]although
the sum of the nonzero pixels still equals thesum of the nonzero
pixels in the original pattern.
In [35] and [61], JPEG compression immunity is ob-tained by
deriving a different gain factor k for each 32 32 pixel block based
on a lower quality JPEG compressedimage. A 32 32 pseudorandom
pattern representing awatermark bit is added to a 32 32 image tile.
A copy ofthis watermarked image tile is degraded according to
theJPEG standard for which end a relatively low quality fac-tor is
used. If the watermark bit cannot be extracted cor-rectly from this
degraded copy, the watermark pattern isadded to the image by means
of a higher gain factor and anew degraded copy is formed to check
the bit. This proce-dure is repeated iteratively for each bit until
all bits can beextracted reliably from the degraded copies. A
watermarkformed in this way is resistant to JPEG compression usinga
quality factor equal to or greater than the quality factorused to
degrade the copies. In Fig. 25 an example of sucha watermark is
shown, amplified for visibility purposes.
32 IEEE SIGNAL PROCESSING MAGAZINE SEPTEMBER 2000
% B
it E
rror
s
60
50
40
30
20
10
00 10 20 30 40 50 60 70 80 90 100
Qjpeg
W kfixed =2
Wvar
P=32 32, Prefilter Applied before DetectionFedge
22. Influence of a variable gain factor adapted to the HVS onthe
robustness of a watermark.
FH
v
u
8 8 DCT
23. DCT bands FH in which the watermark energy is
mini-mized.
-
Anticipating Geometrical TransformsA watermark should not only
be robust to lossy compres-sion techniques, but also to geometrical
transformationssuch as shifting, scaling, cropping, rotation, etc.
Geomet-rical transforms hardly affect the image quality, but theydo
make most of the watermarks that have been embed-ded by means of
the techniques described in the previoussections undetectable for
the watermark detectors. Sincegeometrical transforms typically
affect the synchroniza-tion between the pseudorandom pattern of the
water-mark and the watermarked image, the synchronizationmust be
retrieved before the detector performs the corre-lation
calculations.
The most obvious way to achieve shift invariance is us-ing the
DFT amplitude modulation technique. If, forsome reason, another
watermarking embedding domainis preferred and shift invariance is
required, a marker canbe added in the spatial domain to determine
the transla-tion. This marker can be a pseudorandom pattern like
thewatermark itself. The detector first determines the
spatialposition of this marker by shifting the marker over all
pos-sible locations in the image and calculating the
correlationbetween the marker and the corresponding image part.The
translation with the highest correlation defines thespatial
position of the marker. Finally, the image is shiftedback to its
original position and the normal watermarkingdetection procedure is
applied.
An exhaustive search for a marker is computationallyquite
demanding. Therefore, in [53] a different approachis proposed:
adding a pseudorandom pattern twice, butat different locations in
the image. The content of the wa-termark, i.e., the watermark bits,
is embedded here in therelative positions of the two watermark
patterns. To de-tect the watermark, the detector computes the phase
cor-relation between the image and the watermark patternusing the
fast Fourier transform (FFT) and it detects the
two correlation peaks of the two patterns. The content ofthe
watermark is derived from relative position of thepeaks. If the
whole image is shifted before detection, theabsolute positions of
the correlation peaks will change,but the relative positions will
remain unchanged, leavingthe watermark bits readable for the
detector.
In [30] a method is proposed to add a grid to an imagethat can
be used to scale, rotate, and shift an image back toits original
size and orientation. The grid is representedby a sum of sinusoidal
signals, which appear as peaks inthe FFT frequency domain. These
peaks are used to deter-mine the geometrical distortion.
In [59] a method is proposed which embeds apseudorandom pattern
multiple times at different loca-
SEPTEMBER 2000 IEEE SIGNAL PROCESSING MAGAZINE 33
1 12 23 34 45 56 67 78 81 1
2 23 3
4 45 5
6 67 7
8 80 0
1 1
2 2
3 3
4 4
5 5
6 6
(a) (b)
24. (a) Original watermark block and (b) low frequency watermark
block.
25. Watermark where the local gain factor per block is basedon a
lower quality image.
-
tions in the spatial domain of an image. The detector esti-mates
the watermark W by applying a high pass filterFHP to the
watermarked image
W I F
F
W HP
HP
=
=
0 0 0 1 0 0 00 0 0 1 0 0 00 0 0 1 0 0 01 1 1 12 1 1 1
0 0 0 1 0 0 00 0 0 1 0 0 00 0 0 1 0 0 0
12
/ .
(20)
Next, the autocorrelation function of the estimatedwatermark W
is calculated. This function will have peakvalues at the center and
the positions of the multiple em-bedded watermarks. If the image
has undergone a geo-metrical transformation, the peaks in the
autocorrelationfunction will reflect the same transformation and
henceprovide a grid that can be used to transform the imageback to
its original size and orientation.
In [40], [41], [86], [74], [87], and [88] a method isproposed
that embeds the watermark in a rotation, scale,and translation
invariant domain using a combination ofDFT and a log polar map
(LPM). Figure 26 presents ascheme of this watermarking method.
First the amplitude of the DFT is calculated to obtain
atranslation invariant domain. Next, for every point ( , )u vof the
DFT amplitude a corresponding point in the LPM( , ) is
determined:
u e v e= = cos( ) sin( ). (21)
This coordinate system of the LPM converts rotationand scaling
into translation along the horizontal and ver-tical axis. By taking
the amplitude of the DFT of thisLPM, we obtain a rotation, scale,
and translation invari-ant domain. In this domain a CDMA watermark
can beadded, for instance by modulating the coefficients
using(10).
Figure 27 demonstrates an example of the propertiesof the LPM.
Part (b) shows the LPM of the Lena image(a). Part (c) depicts a
rotated and scaled version of theLena image, and (d) shows its
corresponding LPM. Itcan clearly be seen that the rotation and
scaling in theoriginal spatial domain are converted into
translations inthe LPM domain.
In practice implementing the watermarking scheme il-lustrated in
Fig. 26 has been proven to be difficult. Theauthors therefore
propose a different approach, where aCDMA watermark is embedded in
the translation invari-ant amplitude DFT domain. To make the
watermarkscale and rotation invariant, they embed a second
water-mark, a template, in this domain. To extract the water-mark,
they first determine the scale and orientation of thewatermarked
image by using the template in the follow-ing way: The DFT of the
watermarked image is calculated. The LPM of the DFT amplitudes and
the template pat-tern is calculated. The horizontal and vertical
offsets between the twoLPMs are calculated using exhaustive search
andcross-correlation techniques, resulting in a scale and rota-tion
factor.
34 IEEE SIGNAL PROCESSING MAGAZINE SEPTEMBER 2000
Rotation, Scale, and Translation Invariant WM
DFT
DFT
IDFT
IDFT
LPM ILPM
Image
Phase
Phase
Amplitude
Amplitude
26. Rotation, scale, and translation invariant
watermarkingscheme.
(a) (b) (c) (d)
27. Example of the properties of the LPM. (a) Original image,
(b) LPM of (a), (c) scaled and rotated, and (d) LPM of (c).
-
Next, the image is transformed back to its original sizeand
orientation, and the information-carrying watermarkis
extracted.
Correlation-Based WatermarkingTechniques for MPEGIn real-time
watermarking applications, robustness is notthe only factor that
plays an important role. The other fac-tor that plays a very
important role is computational com-plexity. In general, image or
video data is transmitted inJPEG or MPEG compressed form. Real-time
watermarkembedding must take into account this compressed
form,because first decompressing the data, adding a watermarkand
then recompressing the data is computationally toodemanding.
Therefore, it is desirable to develop water-marking techniques that
can operate directly on the com-pressed bit stream, the code words,
or the DCT trans-formed coefficients because then it is not
necessary tofully decompress and recompress the data. In this
sectionwe discuss two such methods for MPEG video streams.Other
methods that also operate on code words and DCTcoefficients are
discussed in upcoming sections.
In [111] a method is proposed that adds a DCT trans-formed
pseudorandom pattern directly to the DC-DCTcoefficients of an MPEG
compressed video stream. Thewatermarking process only takes the
luminance values ofthe I-frames into account. To embed a watermark
the fol-lowing procedure is performed: First a pseudorandompattern
consisting of the integers {1,1} is generatedbased on a secret a
key. This pattern has the same dimen-sions as the I-frames. Next,
the pattern is modulated by awatermark bit string and multiplied by
a gain factor.Finally, the 8 8 block DCT transform is applied on
themodulated pattern and the resulting DC-coefficients areadded to
the corresponding DC-values of each I-frame.The watermark can be
detected using correlation tech-niques in the DCT domain or in the
spatial domain as de-scribed earlier.
The authors report that the algorithm decreases the vi-sual
quality of the video stream drastically. Therefore, thegain factor
of the watermark has to be chosen to be verylow (> 100,000)
tomaintain reasonable visual quality for the resulting videostream.
This is mainly due to the fact that thewatermark pattern is
embedded in just one ofthe 64 DCT coefficients, the
DC-component.Furthermore, the pattern consists only of lowfrequency
components to which the human eyeis quite sensitive. For
comparison, the algo-rithm used to embed multiple bits using
thecorrelation technique described earlier uses again factor of two
and about 1000 pixels perwatermark bit.
In [36]-[39] and [115] a more sophisticatedwatermarking
algorithm is proposed that em-beds a watermark not only in the
DC-coeffi-
cients, but also in the AC-coefficients of each I-, P-,
andB-frame. The watermark here is also a pseudorandompattern
consisting of the integers {1,1} generated basedon a secret key.
This pattern has the same dimensions asthe video frames. The
pattern is modulated by a water-mark bit string and multiplied by a
gain factor k.
To embed the watermark, the watermark patternW x y( , ) is
divided into 8 8 blocks. These blocks aretransformed to the DCT
domain and denoted byW u vx y, ( , ), where x y, , , ,...=0 8 16
and u v, ,...,=0 7. Next,the 2-D blocks W u vx y, ( , ) are
reordered in a zig-zag scanfashion and become arrays W ix y, ( ),
where i =0 63,..., .W x y, ( )0 represents the DC-coefficient and W
x y, ( )63 de-notes the highest frequency AC-coefficient of a 8 8
wa-termark block. Since the corresponding MPEG encoded8 8 video
content blocks are encoded in the same way asI ix y, ( ), these
arrays can directly be used to add the water-mark. For each video
block I ix y, ( ) out of an I-, P-, orB-frame the following steps
are performed:
1. The DC-coefficient is modulated as follows:
I I WW x y x yx y, ( ) ( ) ( ), ,0 0 0= + (22)
which means that the average value of the watermarkblock is
added to the average value of the video block.
2. To modulate the AC-coefficients the bit stream ofthe encoded
video block is searched VLC-by-VLC for thenext VLC code word,
representing the next nonzeroDCT coefficient. The run and level of
this code word aredecoded to determine its position i along the
zig-zag scanand its amplitude I ix y, ( ).
A candidate DCT coefficient for the watermarkedvideo block is
generated, which is defined as
I i I i W i iW x y x yx y, ( ) ( ) ( ), ., ,= + 0 (23)
Now the constraint that the video bit rate may not beincreased
comes into play. The size SzI of the VLCneeded to encode I ix y, (
) and the size SzIW of the VLCneeded to encode I iW x y, ( ) are
determined using theVLC-Tables B.14 and B.15 of the MPEG-2
standard[47]. If the size of VLC encoding the candidate DCT
co-efficient is equal or smaller than the size of the existingVLC,
the existing VLC is replaced. Otherwise the VLC is
SEPTEMBER 2000 IEEE SIGNAL PROCESSING MAGAZINE 35
DriftCalculation
Coefficient Domain WatermarkingWatermark Embedding
VLD TD DQ Q TC VLC
DCT
MPEGDecoder
MP
EG
Vid
eo
MP
EG
Vid
eo
28. Increase of complexity due to drift compensation.
-
left unaffected. This means that the DCT coefficientI ix y, ( )
is modulated in the following way:
If thenelse
Sz Sz I i I i W iI i
I I W x y x y
W
W x y
x y
= +,
,
( ) ( ) ( )( )
, ,
= I ix y, ( ). (24)
This procedure is repeated until all AC-coefficients of
theencoded video block are processed.
To extract the watermark information, the MPEG en-coded video
stream is first fully decoded and the water-mark bits are retrieved
by correlating the decoded frameswith the watermark patternW x y( ,
) in the spatial domainusing the standard techniques.
A major problem of directly modifying DCT-coeffi-cients in an
MPEG encoded video stream is drift or erroraccumulation. In an MPEG
encoded video stream predic-tions from previous frames are used to
reconstruct the ac-tual frame, which itself may serve as a
reference for futurepredictions. The degradation caused by the
watermark-ing process may propagate in time and may even
spatiallyspread. Since all video frames are watermarked,
water-marks from previous frames and from the current framemay
accumulate and result in visual artifacts. Therefore, adrift
compensation signal Dr must be added. This signalmust be equal to
the difference of the (motion compen-sated) predictions from the
unwatermarked bit streamand the watermarked bit stream. Equation
(23) changesfor a drift compensated watermarking scheme into
I i I i W i Dr iW x y x y x yx y, ( ) ( ) ( ) ( )., , ,= + +
(25)
A disadvantage of this drift signal is that the complex-ity of
the watermark embedding algorithm increases sub-stantially, since
an additional DCT operation and acomplete MPEG decoding step are
required to calculatethe drift compensation signal. The increase in
complexitycompared to the coefficient domain methods is
illustratedin Fig. 28.
Due to the bit-rate constraint, only around 10-20% ofthe DCT
coefficients are altered by the watermark em-bedding process,
depending on the video content and thecoarseness of the MPEG
quantizer. In some cases, espe-cially for very low bit-rate video,
only the DC-coefficientsare modified. This means that only a
fraction of the water-mark pattern W x y( , ) can be embedded,
typically around0.5 ... 3% [115]. Since only existing (nonzero) DCT
co-efficients of the video stream are watermarked, the em-bedded
watermark is video content dependent. In areaswith only
low-frequency content, the watermark auto-matically consists of
only low frequency components.This complies with the HVS. The
watermark energy is
mainly embedded in areas containing a lot of video con-tent
energy.
The authors in [115] report that the complexity of thewatermark
embedding process is much lower than thecomplexity of a decoding
process followed bywatermarking in the spatial domain and
re-encoding. Thecomplexity is somewhat higher than the complexity
of afull MPEG decoding operation. Typical parameter set-tings for
the embedding are k =1 5,..., for the gain factorof the watermark
and P =500 000 1 000 000, ,..., , , for thenumber of pixels per
watermark bit, yielding watermarklabel bit rates of only a few
bytes per second. The authorsclaim that the watermark is not
visible, except in directcomparison to the unwatermarked video, and
that thewatermark is robust against linear and nonlinear
opera-tions like filtering, noise addition and quantization in
thespatial or frequency domain.
Noncorrelation-BasedWatermarking TechniquesLeast Significant Bit
ModificationThe simplest example of a spatial domain
watermarkingtechnique that is not based on correlation is the
LSBmodification method. If each pixel in a gray level image
isrepresented by an 8-bit value, the image can be sliced upin eight
bit planes. In Fig. 29 these eight bit planes arerepresented for
the Lena image, where the upper left im-age represents the most
significant bit plane and the lowerright image represents the LSB
plane.
Since the least significant bit plane does not contain vi-sually
significant information, it can easily be replaced byan enormous
amount of watermark bits. More sophisti-cated watermarking
algorithms that make use of LSBmodifications can be found in [91],
[4], [5], [43], and[33]. These watermarking techniques are not very
secureand not very robust to processing techniques because theLSB
plane can easily be replaced by random bits, effec-tively removing
the watermark bits.
MPEG Video Watermarkingby Parity Bit ModificationIn a compressed
bit stream we have direct access to thecode words used in the
compression algorithm. Similar tothe LSB technique described above,
we can embed water-mark in the stream by modifying these code
words, yield-ing a computationally efficient watermarking
methodwith a high payload [62], [35].
The technique is described as follows. A watermarkconsisting of
l label bits b j lj ( , , ,..., )= 0 1 2 1 is embeddedin the
MPEG-stream by selecting suitable VLCs and forc-ing the LSB of
their quantized level to the value of bj . Toensure that the change
in the VLC is perceptually invisi-ble after decoding and that the
MPEG-bit stream keeps itsoriginal size, we select only those VLCs
for which an-other VLC exists with: the same run length,
36 IEEE SIGNAL PROCESSING MAGAZINE SEPTEMBER 2000
The robustness of a watermarkcan be improved by increasingthe
energy of the watermark.
-
a quantized level difference of one, the same code word
length.
A VLC that meets this requirement is called a
la-bel-bit-carrying-VLC (lc-VLC). According to TablesB.14 and B.15
of the MPEG-2 standard [47], an abun-dance of such lc-VLCs exists.
Furthermore, allfixed-length-coded DCT-coefficients following an
Es-cape-code meet the requirement. Some examples oflc-VLCs are
listed in Table 1, where the symbol s repre-sents the sign-bit.
This sign-bit represents the sign of theDCT coefficient level.
The VLCs in the intra- and intercoded macro blockscan be used in
the watermarking process. The DC coeffi-cients are not used,
because they are predicted from otherDC coefficients and coded with
a different set of VLCsand Escape-codes. Furthermore, replacing
each DC coef-ficient in intra- and intercoded frames can result in
visibleartifacts due to drift. By only taking the AC
coefficientsinto account the watermark will adapt itself more to
thevideo content and the drift will be limited.
To add the label bit stream L to an MPEG-video bitstream, the
VLCs in each macro block are tested. If anlc-VLC is found and the
LSB of its level is unequal to thelabel bit b j lj ( , , ,..., )= 0
1 2 1 , this VLC is replaced by an-other one, whose LSB-level
represents the label bit. If theLSB of its level equals the label
bit bj the VLC is notchanged. The procedure is repeated until all
label bits areembedded. In Fig. 30 an example is given of
thewatermarking process, where three label bits are embed-ded in
the MPEG video stream.
To extract the label bit stream L the VLCs in eachmacro blocks
are tested. If an lc-VLC is found, the valuerepresented by its LSB
is assigned to the label bit bj . Theprocedure is repeated for j l=
0 1 2 1, , ,..., until nolc-VLCs can be found anymore.
This technique gives a high payload (up to 29 kbit/s)without
significant perceptible quality degradation [65].The watermark
embedded with this method can easily beremoved by decoding and
reencoding the video stream orby relabeling the stream using
another randomly gener-ated watermark pattern. This technique can
be extendedto make it resistant to relabeling [65], as follows. The
wa-termark label bits bi are now not directly stored in theLSBs of
the VLCs, but a one-dimensional pseudorandomwatermark patternW x(
)is generated consisting of the in-tegers {1,1} based on a secret
key, which is modulatedwith the label bits bi . The procedure to
add this modu-lated pattern to the video stream is similar to the
proce-dure described above.
However, we now select only those VLCs for whichtwo other VLCs
exist, with the same run length and thesame codeword length. One
VLC must have a level dif-ference of +and the other VLC must have a
level differ-ence of . Most lc-VLCs meet these requirements for
arelative small (e.g., = 1,2,3). For notational simplicitywe call
these pattern-carrying-VLCs (pc-VLCs).
To embed a watermark in a video stream, we add themodulated
watermark pattern to the levels of thepc-VLCs. To extract the
watermark, we collect thepc-VLCs in an array. The watermark label
bits can now beretrieved by calculating the correlation between
this arrayof pc-VLCs and the secret watermark pattern W x( ).
InFig. 31 an example is given of the watermark embeddingprocess.
About 1,000,...,10,000 pc-VLCs are now re-quired to encode one
watermark label bit bi and thus dras-tically reduce the payload of
the watermark. However,several watermark label bit strings can be
added withoutinterfering with each other, if independent
pseudoran-dom patterns are used to form the basic pattern W x(
).
SEPTEMBER 2000 IEEE SIGNAL PROCESSING MAGAZINE 37
29. Bit planes for the Lena image.
-
DCT Coefficient OrderingIn [55], [114], [54], and [17] a
watermarking method isproposed that adds a watermark bit string in
the 8 8block DCT domain. To watermark an image, the image isdivided
into 8 8 blocks. From these 8 8 blocks theDCT transform is
calculated and two or three DCT coef-ficients are selected in each
block in the middle band fre-quencies FM (Fig. 32). The selected
coefficients arequantized using the default JPEG quantization table
[73]and a relatively low JPEG quality factor. The selected
co-efficients are then adapted in such a way that their magni-tudes
form a certain relationship. The relationshipsamong the selected
coefficients compose eight patterns(combinations), which are
divided into three groups.Two groups are used to represent the
watermark bits 1or 0, and the third group represents invalid
patterns. Ifthe modifications which are needed to hold a desired
pat-tern become too large, the block is marked as invalid.
Forexample, if a watermark bit with value 1 must be embed-ded in a
block, the third coefficient should have a lowervalue than the two
other coefficients. The embeddingprocess and the list of patterns
are represented in Fig. 32.
In Fig. 33 the heavily amplified difference between theoriginal
Lena image and the watermarked version isshown. In [13] and [14] a
similar watermarking methodis proposed, but here the DCT
coefficients are modifiedin such a way that they fulfill a linear
or circular constraintimposed by the watermark code.
We note that the techniques described above are simi-lar to the
DEW method for real-time MPEG videowatermarking described in the
next section.
MPEG Video WatermarkingUsing the DEW AlgorithmThe DEW method is
based on selectively discarding highfrequency DCT coefficients in
the compressed data
stream. The information bits of the data identifier (label)are
encoded in the pattern of DCT blocks in which highfrequency DCT
coefficients are removed, i.e., in a patternof energy differences
between DCT blocks. For this rea-son, the technique is called a
differential energy water-mark (DEW).
The technique is described as follows. The informationthat we
wish to embed into the image or video frame isrepresented by the
label bit string L consisting of label bitsL j lj ( , ,..., )= 0 2
1 . This label bit string is embeddedbit-by-bit in a set of n 8 8
DCT blocks taken from aJPEG compressed still image or from an
I-frame of anMPEG compressed video stream. For the purpose of
sim-plicity of the discussion, we will refer to still images
andMPEG I-frames as image.
To obtain sufficient robustness, typically n takes onvalues
between 16 and 64, which means that a single labelbit is embedded
in a region of the image. Before the labelbits are embedded,
however, the positions of the 8 8DCT blocks in the image are
shuffled randomly as illus-trated in Fig. 34. This shuffling
operation, on the onehand, forms the secret key of the labeling
algorithm,while on the other hand it spatially randomizes the
statis-tics of DCT blocks.
Each bit of the label bit string is embedded in its pri-vate
label bit-carrying-region, or lc-region for short, in ashuffled
image. For instance, in Fig. 33 the first bit is lo-cated in the
top-left-corner of the image in an lc-region ofn =16 DCT blocks.
The value of the label bit is encodedby introducing an energy
difference between the high fre-quency DCT-coefficients of the top
half of the lc-region(denoted by lc-subregion A) containing in this
case n/2 =8 DCT blocks, and the bottom half (denoted by
lc-subre-gion B) also containing n/2 = 8 DCT blocks. If the
lc-sub-region A contains more high frequency energy than
thelc-subregion B, the label bit value 0 has been embedded
into the data, and vice versa.To make the determination of high
fre-
quency energy easy for images or videoframes that are JPEG or
MPEG com-pressed, we compute energies over a subsetof zigzag
scanned DCT-coefficients indi-cated by S c( )
{ }S c i i c( ) { , }|( ) .= >0 63 (26)
The zigzag scanned DCT coefficients arenumbered according to
Fig. 35. The indexi =0 refers to the DC-coefficient of a DCTblock.
The subset of DCT coefficients S c( )over which energies are
computed is de-fined by the cut-off index c. The selection of
asuitable cut-off index c for an lc-region is es-sential for the
robustness and the visibilityof the label bit. The larger the
cut-off indexis chosen, the less degradation the label em-bedding
will introduce. Here we assumethat we have available a suitable
cut-off in-
38 IEEE SIGNAL PROCESSING MAGAZINE SEPTEMBER 2000
Table 1. Example of lc-VLCs in Table B.14 of the MPEG-2
Standard.
Variable Length Code VLC size Run Level LSB of Level
0010 0110 s0010 0001 s
8 + 18 + 1
00
56
10
0000 0001 1101 s0000 0001 1000 s
12 + 112 + 1
00
89
01
0000 0000 1101 0 s0000 0000 1100 1 s
13 + 113 + 1
00
1213
01
0000 0000 0111 11 s0000 0000 0111 10 s
14 + 114 + 1
00
1617
01
0000 0000 0011 101 s0000 0000 0011 100 s
15 + 115 + 1
11
1011
01
0000 0000 0001 0011 s0000 0000 0001 0010 s
16 + 116 + 1
11
1516
10
-
dex c for each lc-region [66]. Note that different lc-re-gions
may have different cut-off indexes depending ontheir spatial
contents.
The (DCT high frequency) energy E A in lc-subregionA is now
defined as follows:
( )E c n QAb
n
i b Qi S c
, , [ ] ./
,( )
jpeg jpeg=
=
0
2 1 2
(27)
Here i b, denotes the non-weighted DCT coefficientwith index i
in the bth DCT block of the lc-subregion Aunder consideration.
Prior to the calculation of E A , thenotation [] Q jpeg indicates
that, the DCT-coefficients arere- or prequantized, in our case
using the standard JPEGquantization procedure [73] with quality
factor Q jpeg .For embedding label bits into MPEG
compressedI-frames a similar approach can be followed, but here,
weconfine ourselves to the JPEG notation without loss ofgenerality.
The prequantization is done only in determin-ing the cut-off
indexes and the calculation of (26), but isnot applied to the
actual image data upon embedding the
label. The energy in lc-subregion B, denoted by E B , isdefined
similarly.
We now define the energy difference D between thelc-subregions A
and B as follows:
( ) ( ) ( )D c n Q E c n Q E c n QA B, , , , , , .jpeg jpeg
jpeg= (28)The value of a label bit is encoded as the sign of the
en-
ergy difference D. Label bit 0 is defined as D >0and labelbit
1 as D
-
Salient-Point ModificationIn [82] a watermarking method is
proposed that is basedon the modification of salient points in an
image. Salientpoints are defined as isolated points in an image
forwhich a given saliency function is maximal. These pointscould be
corners in an image or locations of high energy,for example.
To embed a watermark we extract the set of pixels withhighest
saliency S from the image. Next, a binarypseudorandom pattern W x
y( , ) with the same dimen-sions as the image is generated. This
can be a line or blockpattern as represented in Fig. 36. If this
pattern is suffi-ciently random and covers 50% of all the image
pixels,50% of all salient points in set S will be located on the
pat-tern and 50% off the pattern W x y( , ). Finally, the
salientpoints in set S are adapted in such a way that a
statisticallysignificant high percentage of them lies on the
watermarkpattern (i.e., the black pixels in the pattern). There
aretwo ways to adapt the salient points:
The location of the salient points can be changed bywarping the
points towards the watermark pattern. Inthis case small, local
geometrical changes are introducedin the image. The saliency of the
points can be decreased or increasedby adding well-chosen pixel
patterns to the neighbor-hood of a salient point.
To detect the watermark we extract the set of pixelswith highest
saliency S from the image and compare thepercentages of the salient
points on the watermark pat-tern and off the pattern. If both
percentages are about50% no watermark is detected. If there is a
statisticallysignificant high percentage of salient points on the
pat-tern, the watermark is detected. The payload of this wa-termark
is 1 bit.
Fractal-Based WatermarkingSeveral watermark embedding algorithms
based onfractal compression techniques have been proposed
[24],[80], [8], [9]. They mainly use block-based local
iteratedfunction system coding [49]. We first briefly describe
thebasic principles of this fractal compression algorithmhere. An
image is partitioned at two different resolutionlevels. On the
first level, the image is partitioned in rangeblocks of size n n .
On the second level the image is parti-tioned in domain blocks of
size 2 2n n . For each rangeblock, a transformed domain block is
searched for whichthe mean square error between the two blocks is
minimal.Before the range blocks are matched on the domainblocks,
the following transformations are performed onthe domain
blocks.
First, the domain blocks are subsampled by a factor oftwo to get
the same dimensions as the range blocks. Sub-sequently, the eight
isometries of the domain blocks aredetermined (the original block
and its mirrored versionrotated over 0, 90, 180, and 270). Finally,
the scale fac-tor and the offset for the luminance values is
adapted. Theimage is now completely described by a set of relations
foreach range block, by the index number of the best fittingdomain
block, its orientation, the luminance scaling, andthe luminance
offset. Using this set of relations, an imagedecoder can
reconstruct the image by taking any initialrandom image and
calculating the content of each rangeblock from its associated
domain block using the appro-priate geometric and luminance
transformations. Takingthe resulting image as initial image one
repeats this pro-cess iteratively until the original image content
is approxi-mated closely enough.
In [80] a watermarking technique is proposed whichembeds a
watermark of 32 bits b b bl0 1 1 in an image.The embedding
procedure consists of the full fractal en-coding and decoding
process as described above, wherethe watermark embedding takes
place in the fractal en-coding process. First, the image I x y( , )
is split in two re-gions A x y( , ) and B x y( , ). For each
watermark bit bj Urange blocks are pseudorandomly chosen from I x
y( , ). If
40 IEEE SIGNAL PROCESSING MAGAZINE SEPTEMBER 2000
12345678
FM
v
u1 2 3 4 5 6 7 8
8 8 DCT Block with PossibleLocations for Embedding a Bit
HMH
MLL
HLM
MHH
LML
LHM
LLL
HHH
MMM
Patterns for 1
Patterns for 0
Invalid Patterns
Relationships Among ThreeQuantized DCT Coefficients
H: HighM: MiddleL: Low
32. Watermarking based on adapting relationship betweenthree
coefficients.
33. Watermark W x y I x y IW x y( , ) ( , ) ( , )= created by
adaptingrelationships between DCT coefficients.
-
bj equals one, the domain blocks to code the U rangeblocks are
searched in region A x y( , ). If bj equals zero, thedomain blocks
to code the U range blocks are searched inregion B x y( , ). For
range blocks which are not involved inthe embedding process, domain
blocks are searched in re-gions A x y( , )and B x y( , ). To
extract the watermark infor-mation, we must select and re-encode
the U range blocksfor each bit bj . If most of the best fitting
domain blocksare found in region A x y( , ), the value 1 is
assigned to bitbj , otherwise the bit is assumed to be zero.
In [8] and [9] a watermark is embedded by forcingrange blocks to
map exactly on specific domain blocks.The watermark pattern here
consists of this specific map-ping. This mapping is enforced by
adding artificial localsimilarities to the image. The size of the
range blocks maybe chosen to be equal to the size of the domain
blocks. InFig. 37 an example is given of this process.
The left image illustrates how a fractal encoder wouldmap the
range block Rb18 on domain block Db0 in anunwatermarked image. To
embed the watermark, thismapping Db Rb0 18 must for instance be
changed toDb Rb0 21 . To force the mapping to this form, a blockRb
21 is generated from block Db0 by changing its lumi-nance values.
By adding block Rb to the image, wechange the optimal fractal
mapping to its desired formDb Rb0 21 , because the quadratic error
between Db0 ,corrected for luminance scale and offset and Rb21 is
nowsmaller than the error between Db0 and Rb18 .
To detect the watermark we calculate the optimalfractal mapping
between the range blocks and the domainblocks. If a statistically
significant high percentage of themappings between range blocks and
domain blocksmatch the predefined mappings of the watermark
pat-tern, the watermark is detected.
Concluding RemarksThis article has given a state-of-the-art
overview of com-mon watermarking techniques. New watermarking
tech-niques are invented regularly. Some of the
watermarkingtechniques are designed for specific applications,
whilethe others are not well established yet but have a great
po-tential. For the purpose of completeness we briefly list
theprinciples of these watermarking techniques below: For printed
images dithering patterns can be adaptedto hide watermark
information [98], [19]. Instead of the pixel values, the histogram
of an imagecan be modified to embed a watermark [116]. The method
proposed in [16] embeds a watermark bymodifying the mean value of
the pixels of randomly se-lected blocks in an image. The authors in
[10] proposed the so-called textureblock coding in which the
watermark is embedded bycopying one image texture block to another
area in theimage with a similar texture. Recovering the watermark
isachieved by computing the autocorrelation function.This method
offers high robustness to any kind of distor-
tion because both image areas are distorted in a similarway.
This means that the watermark recovery byautocorrelation will still
work. Quantization can be exploited to hide a watermark. In[85] a
method is proposed in which the pixel values of animage are first
coarsely quantized, before some small ad-aptations are made to the
image. To detect these adapta-tions the watermarked image is
subtracted from itscoarsely quantized version. In [57] selected
wavelet coef-ficients are quantized using different quantizers for
wa-termark bits 0 and 1.
SEPTEMBER 2000 IEEE SIGNAL PROCESSING MAGAZINE 41
Label:
lc-region:16 8 8Blocks
1 8 8Block
lc-Subregion:8 8 8Blocks
(a)
1 0 1 1 0 0 1 0 1 0 1 1 1 0 0 1
(b)
(c)
34. (a) Sample I-frame; (b) block-based randomly shuffledI-frame
showing the label-carrying (lc) regions and lc-subre-gions; (c)
difference between the original and watermarkedimage showing that
the DEW algorithm put the watermark inregions with a lot of spatial
details.
-
Watermarks can also be embedded by using projec-tion-based
techniques [96], [2]. In these techniques,the original data
(divided into blocks) are projectedinto another direction/subspace.
The data here can bethe transform coefficients of the original
image. Theprojection direction could be random or image de-pendent.
The authors in [2] also show that their pro-posed technique could
resist rotation and scaling tosome extent. The concept of
self-embedding [101], that is embed-ding important parts of an
image (for example, the eyes ofa person) onto the image itself, is
important to detect(and if possible recover from) a tampering
attack in whicha portion of the image has been altered. In [101]
the au-thors proposed a high capacity watermarking techniquethat is
capable of detecting tampering and to some extentrecover from
it.
In this article we have discussed the most importantclasses of
watermarking techniques. The first class com-prises the
correlation-based methods. Here a watermarkis embedded by adding
pseudorandom