Multimedia Watermarking Techniques Frank Hartung and Martin Kutter ECE 738 In Young Chung.

Multimedia Multimedia Watermarking Watermarking TechniquesTechniques

Frank Hartung and Martin Kutter

ECE 738In Young Chung

OutlineOutline

1. Terminology2. Requirement3. Basic Watermarking Principles4. Watermarking techniques for

1. Text Document2. Image3. Video4. Audio5.other media

5. Conclusion

TerminologyTerminology

Watermarkingtechniques that allow secret communication, usually by embedding or hiding the secret information

Data hiding and data embeddingapplications where the existence of the embedded data are publicly known, but there is no need to protect it

Fingerprinting and Labelingspecial application of watermarking related to copyright protection

Embedded signaturesstand for watermark in early publication

Visible watermarksvisible pattern, like logos, inserted into images or video

Watermarking RequirementsWatermarking Requirements

1) As much information as possible Capacity

2) Only be accessible by authorized parties by means of cryptographic key security

3) Resist against hostile attacks Robusteness

4) Invisibility Imperceptibility

Robustness

Capacity

Imperceptibility

Supplemental W. RequirementsSupplemental W. Requirements

1) Watermark Recovery may or may not allowed to use the original data.

2) Real time watermarking requirementse.g., video fingerprinting (We will see this later)

Design IssuesDesign Issues

Watermark Security and Keys Robustness Imperceptability Capacity Watermark Detection

Robustness

Capacity

Imperceptibility

Basic Watermarking PrinciplesBasic Watermarking Principles

Three issues in the design of a watermarking system

1) Design of the Watermark signal

2) Design of the embedding method

3) Design of the corresponding extraction method

• Where, W: watermark signal, I: watermark information.• K: key, X: host data, Y: watermarked signal

),,(),( 00 XKIfWorKIfW

),(1 WXfY

),(),,(^^

KYgIorKYXgI

Basic Watermarking PrinciplesBasic Watermarking Principles

The public or secret key is used to enforce security

Many watermarking schemes use spread-spectrum methods they add a PN signal with low amplitude to the host data.

Correlator is used for watermark detection

Watermarking TechniquesWatermarking Techniques1. Text Document 1. Text Document

WatermarkingWatermarking

Text Document WatermarkingText Document Watermarking

Two methods to hide information1) in the semantics; in the meaning and

ordering or the words2) in the format

* In the layout and the appearance

- Example of word shifting coding -

Text Document Watermarking;Text Document Watermarking;Three coding methodsThree coding methods

1) Line shift coding*Assumption; lines are uniformly spaced doesn’t need original for watermark extraction

2) word-shift coding

*Assumption; space between words are usually variable needs original for Watermark extraction

3) feature coding; slightly modifies the features

Goal: making watermark removal more expensive than obtaining the right to copy from the copy right owner

Watermarking TechniquesWatermarking Techniques2. Image Watermarking2. Image Watermarking

Why is Image watermarking so important?1) There is a large demand and

productions2) Most of watermarking research much more than video or audio watermark

Common ideas for Image Common ideas for Image watermarkingwatermarking

A lot of watermarking methods are very similar and differ only in parts

Three topics 1) watermark signal design 2) embedding 3) recovery (Detection)

I. Watermark signal designThe watermark signal; typically a pseudorandom signal with low amplitude

e.g., Gaussian, uniform, or bipolar pdf* The watermark signal: often designed in spatial domain, sometimes in DCT or block-wise DCT

Common ideas of Image Common ideas of Image watermarkingwatermarking

ii. Signal embedding (where?)

* embed watermark signal mostly to the luminance channel alone* sometimes in color channels

* in the spatial domain * or in the DCT,DFT and DWT (full-image DCT or block wise DCT domain) advantage in terms of visibility and security

Common ideas of Image Common ideas of Image watermarkingwatermarking

Argue? about embedding domain Low, medium or high freq.?

For maximum robustness embed watermark signal adaptively where the host data populate (typically the low frequency)

Common ideas of Image Common ideas of Image watermarkingwatermarking

iii. Recovery (Detection)* Usually done by correlation method; a correlation receiver or a matched filter

AcknowledgementsAcknowledgements

The Following slides organized by Author name

The author of this paper put stress on embedding domain/methods So, we will mainly deal with embedding domain/method of numerous watermarking methods

TirkelTirkel

Publication: “Electrical Water Mark” in 1993

Proposal m-sequence PN code embedding in LSB

plane* To gain full access to the LSB plane without much distortion compress original image to 7 bits through histogram manipulation

the decoding process use the unique and optimal autocorrelation of m-sequences

Matsui and TanakaMatsui and Tanaka

Publication: “Video Steganography: How to secretly Embed a Signature in a picture”

Proposal Predictive code schemes using key table

* exploit correlation between adjacent pixels by coding the prediction error instead of coding the individual gray scale value

And embed a watermark in forms of a binary string1 iii XX

SmithSmith

New approach

Digital watermarking and digital modulation (especially, direct sequence spread spectrum modulation) share similar concepts

More in depth analysis of 2-D amplitude modulation was given by Hernandez

BenderBender

Proposal 2 MethodsI. Patchwork

1. randomly selected pairs of pixels Are used to hide 1 bit by increasing the ai’s by one and decreasing the bi’s by one

2. The expected value of the sum of N pixel pairs

),( ii ba

Cont-Cont-

II. Texture Block Coding

1. Watermark is embedded by copying one image texture block to another area in the image with similar texture

2. Recovery- autocorrelation

* Remarkable point- high robustness to any kind of distortion, since both image area distorted in a similar way autocorrelation still works

Pitas and KaskalisPitas and Kaskalis

Proposal

Signature casting on digital images- based on same basic idea as the patchwork

1. The watermark is the same size as the original image (here, # of ones = # of zero)

2. The original image is divided into two sets A and B of equal size

I : Original Image , xmn : Luminance value

}{ ,nmsS

Contn-Contn-3. The watermark is superimposed by changing the elements of the subset A by positive factor k

*k is positive integer

4. The watermarked image is given by the union of A’ and B

},{' AxkxA mnmn

LangelaarLangelaar

Proposal Block base spatial watermarking

- Improved version of previous method

1. The image is tiled into square blocks (8x8)2. Each block is selected pseudorandomly3. To embed “1”, k x P added to the block4. To embed “0”, k x P subtracted to the block 5. Each selected block has a PN pattern P

* k: scaling factor, P : PN pattern

BruyndonckxBruyndonckx

Proposal Watermarking with the use of pixel classification

Purpose- increase the performance of the block base spatial watermarking methods

1. Select blocks (PN) & classify the block based on three types of contrast between zones; hard, progressive and noise contrast2. Each zone subdivided into two categories A and B based on gird defined by the coder3. Each pixel is assigned to one of four zone/category combinations e.g., 1/A,…2/B

Cont-Cont-

4. A bit b is embedded by modifying the zone/category means to satisfy the following constraints

: the modified zone/category mean values S: the watermark embedding strength

5. the modification of the mean values is done by applying equal luminance variations for all pixels belonging to the same zone

7. To increase robustness the authors suggest to perform redundant bit embedding and use error correcting codes

6. Good robustness to JPEG compression is reported

*2

*1 ... BA mm

KutterKutter

ProposalImproved spread-spectrum watermarking in the spatial domain

* Exclusively works with the blue image component, in the RGB color space maximize the watermarking strength and minimize visual artifacts

* preprocess the image prior to watermark decoding increased robustness, applicable to any spread-spectrum spatial domain watermarking

Cont-Cont-

1. A single bit b is embedded at a pseudorandomly selected location (I,j) by either adding or subtracting (Amplitude modulation)

Where describes the blue value at location (I,j):the Luminance at the same location : the embedding strength

jiB ,

jiL ,

Cont-Cont-

2. To recover an embedded bit, an estimate of original value is computed

c : the size of the cross-shaped neighborhood

3. The bit value is determined by looking at the sign of the difference

*To increase robustness, (Author suggest) each signature bit is embedded several times and to extract, the sum of all differences is used

jijiji BB,

^

,,

ji ,

MacqMacq

ProposalWatermarking adapted to the HVS using masking and modulation

1. The watermark in spatial domain is low-pass filtered, frequency modulated, masked and then added to the host image

2. A secret key is used to determine the modulation freq.

3. Masking uses an extension of the masking phenomena for monochromatic signals, called gratings

Recovery- demodulation followed by a correlation function

Voyatzis and PitasVoyatzis and Pitas

ProposalWatermarking by inserting logo like patterns using torus automorphism

* A 2-D torus automorphism: a kind of spatial transformation

* It is defined by

* Iterated action of A on a point form a dynamical system expressed by 0

Cont-Cont-

* This system mixes the point in chaotic way.

* Under certain circumstances, the automorphisms may have periodicity

1-iteration 2-iteration 10-iteration T-iteration

…

Cont-Cont-

How/where to embed?

1. Watermark is mixed using the automophism

2. then, overlaid on a selected block in the original image e.g., LSB

Recovery- extracting the mixed watermark then, reconstruct the watermark using the automorphism

* Where, T is the automorphism period.

NA

NTA

Raymond and WolfgangRaymond and Wolfgang

Proposal Watermarking technique to verify image

authenticity based on an approach similar to the m-sequence approach

1. A random sequence generated is mapped from {0,1} to {-1,1}, arranged into a suitable block and added to image

Recovery – overlays the watermarked block with the watermark block and compute inner product and compares the result with ideal value

Cont-Cont-

The test statistics is defined as

If the watermark is unchanged , =0 When is greater than a defined tolerance,

the block fails the watermark test

Chen and WornellChen and Wornell Proposal Quantized Index Watermark (QIM)

*Please recall even-odd embedding and predictive coding scheme using table*Not based on spread-spectrum modulation but,quantization modulation*Based on a set of N-dimensional quantizers.*Quantizers – satisfy a distortion constraint

- each reconstruction values from one quantizer are ‘’far away’’ from the others

1. The message to be transmitted is used as in index for quantizer section

2. Selected quantizer is used to embed information

3. Spatial or DCT domain used

KochKoch Proposal Efficient watermarking in DCT domain for JPEG (first introduction)

1. The image is divided into square blocks of size 8x8 for which the DCT is computed

2. From a pseudoramdomly selected blocks, a pair of midfrequency coefficients are selected from 12 predetermined pairs

3. To embed a bit- the coefficient are modified such that the difference between them (a pair of coefficient) is positive or negative

* Good robustness – JPEG (Q=50%)

SwansonSwanson Proposal DCT domain watermarking technique, Based on

frequency masking of DCT blocks

1. Input image is split up in to square matrix and DCT is computed

2. Frequency mask is computed based on the knowledge that a masking grating raises the visual threshold for signal gratings around the mask freq.

3. The resulting perceptual mask is scaled and multiplied by PN(DCT)-sequence

4. This watermark is added to the corresponding DCT block

PodilchukPodilchuk Proposal Perceptual watermarking using the just noticeable

difference (JND) to determine an image-dependent watermark modulation mask

; the transform coefficients of the original image; the computed JND based on visual models; the watermark values

Recovery- based on the correlation

* Robust to JPEG compression, cropping, scaling and additive noise

vuI ,

vuJND ,

vuw ,

Boland and CoxBoland and Cox

Proposal Frequency-domain watermarking (First)

perceptual adaptive methods which is based on modulation

1. Generate the watermark with statistical distribution ; e.g., N(0,1)

2. The watermark is inserted into the image

Cont-Cont-

; determine the strength

* The watermark embedded 1000 strongest DCT coefficients

Detection- given by the normalized correlation coefficient

* Boland propose a similar techniques – DCT, DWT, Walsh-Hadamard, FFT

SummarySummary

Several different image watermarking methods Most watermarking methods are based on the same

basic principle- small, pseudorandom changes are applied to selected region; spatial or transform domain

Recovery – correlation-like similarity measures. Usually, the number of modified coefficient is much

larger than the number of bits to be encoded Embedding domain have a influence on the robustness spatial – less robust to noise like attack / E.g.- JPEG

- more robust to cropping, translating. freq. – less robust to cropping which destroy the

embedding water mark (DCT,DFT,DWT)

Watermarking TechniquesWatermarking Techniques3. Video Watermarking3. Video Watermarking

Common idea for video Common idea for video watermarkingwatermarking

Video sequences consists of a series of consecutive and equally time-spaced still images in general, very similar with image watermarking so, image watermark method is applicable to video directly

Important differences* available signal space;

for image; very limitedfor video; much larger signal space (# of pixels)

*video watermark imposes real or near real-time watermarking system complexity issue is much more important

Cont-Cont-

The structure of video as a sequence of images give rise to particular attacks frame averaging, frame dropping and frame swapping (Only in video)

Two competing requirements*A good watermarking scheme1. may recover the full watermark from a short part of the sequence2. distribute watermark information over several consecutive frame to have robustness against frame dropping depends on application

Compressed/ uncompressed video

Hartung and GirodHartung and Girod Proposal Watermarking of compressed video for fingerprint

application

* Used spread spectrum approach and added an additive watermark into video

* The watermark is generated using a PN signal with the same dimension as the video signal

*Each information bit is redundantly embedded into many pixels

1. For each compressed video fame, 8x8 DCT transformed watermark signal is added to DCT coefficient of the video

Cont-Cont-

*This method done for I,P, and B frames

* Rate control- by comparing each encoded watermarked DCT coefficient versus the corresponding encoded unwatermarked coefficient

(because video steam use variable length coding, the watermarked signal may or may not need more bits encoding than the unwatermarked one)

If more bits required, the coefficient is not used for embedding

Recovery – correlation using the same PN sequence used for generation of watermark

JordanJordan Proposal Watermarking of compressed video that embed

information in motion vector of motion -compensated prediction schemes.

1. Motion vectors slightly modified in pseudorandom way

Watermark embedding/detection available as long as the video is in compressed format

After decompression? the watermark still be recovered by recompressing and detecting

Artifacts? Because the blocks pointed to by the original and the modified vector are very similar no visible artifacts

Hsu and WuHsu and Wu Proposal Watermarking for compressed video using middle-freq DCT

* Extension of their image watermarking method

1. modifies middle freq DCT coefficients in relation to spatially (I-frames) or temporally ( for P- and B- blocks) neighboring block

* Prior to embedding, the watermark signal is spatially scrambled to make it robust to cropping

Drawback- for watermark extraction, original video and watermark should be known

LangelaarLangelaar

Proposal 2 methods

1) data hiding method

1. adds the label directly in the MPEG bit stream by replacing variable length codes (VLC) of DCT coefficient

* In MPEG-2 code tables there are pairs of code which represent the same run and levels that deviate only by one from each other

We can choose one as bit “1” the other as “0”

Drawback- the label can be easily removed by decompression and recompression

Cont-Cont-

2) watermarking1. for each bit to be embedded, a set of n 8x8-block is pseudorandomly taken from the video frame(here, n typically 16~64)

2. Seudorandomly divide into two subsets of equal size.

3. For each subset, the energy of the high-freq. DCT coefficient is measured

4. In order to embed the bit, the energy of the high-freq coefficient in one or the other subset is reduced by removing high freq. coefficients

Cont-Cont-

Block diagram of watermark embedding into DCT coefficients of compressed video

Recovery - 1. select the same set of blocks 2. divide it into the same subsets 3. Compare the energy of the high-freq.

coefficient in each of the subsets * Here, We can use the secret key for the selection of

blocks

Cont-Cont-

Drawbacks1. robustness is limited2. Re-encoding increases the error rate of the embedded signal much

SwansonSwanson Proposal Multiscale watermarking method working on uncompressed video

*Same scheme as Image watermarking

1. video sequence is segmented into scenes

2. temporal wavelet transform is applied to each scenes, yielding temporal low-pass and high-pass frames

3. the watermark is embedded into each of the temporal components (here, low-freq.)

4.Inverse transform the watermarked components to get the watermarked video

Cont-Cont-

Interesting properties

The watermark has some components that change over time, since they are embedded in low frequency coefficient this allow robustness against frame averaging, frame dropping

Drawback- very high complexity

LinnartzLinnartz

Proposal Embed information in the GOP structure of

the MPEG-2 compressed video

*There is a maximum distance between two successive I-frames

*The frame type signaled in the frame header and can be switched randomly from frame to frame.

Cont-Cont-

* A typical GOP in display order (N=12)

* If I-frame is fixed we have 2048 variations

*However, most available video codecs never use most of the admissible GOP structure purposely use irregular GOP structure, that are very unlikely, to embed information

Drawback- use this method only during compression - decompress and recompression will remove this information completely

Merit - Low complexity

DarmstaedterDarmstaedter Proposal Embed a spatial-domain low-pass spread spectrum

watermark into 8x8 pixel blocks of video sequences

1. The blocks are classified according to their activity*Low activity are not watermarked

2. Low-pass pseudorandom pattern is added to each block

* Each block conveys one bit watermark information

Cont-Cont-

* The block repeated over several blocks and several frames for robustness

Recovery-done in spatial domain after decompression using correlation concept with threshold

DeguillaumeDeguillaume

Proposal Spread spectrum watermark into 3-D

block of video using 3-D DFT

* Embed a spread spectrum watermark into 3-D blocks of video by employing a 3-D DFT and add to the transform coefficients

BuschBusch

Proposal Apply a still-image watermarking method working on

DCT blocks to video sequences

1. The watermarks are embedded into the luminance component of umcompressed video and retrieved after decompression

*In order to improve the invisibility of the watermarks, especially at edge, blocks are selected depend on activity (high-activity block selected)

Cont-Cont-

Recovery- 0~50% error rate are reported depend on the sequencethe author propose to embed the watermark into several consecutive frames (over 50frame a few percent error rate reported)

KalkerKalker

Proposal Video Watermark for video broadcasting

monitoring application

*Called JAWS (just another watermarking system)

* For low complexity, both watermarking and detection are performed in the spatial domainwatermarking before compressiondetection after decompression

Cont-Cont-

* The watermark size is 128x128 and repeated (tiled) to fill whole video frame

* To avoid visible artifacts, the watermark is, on a pixel-by-pixel basis, scaled with scale factor

* Scale factor is derived from an activity measure

* Activity measure is computed using a Laplacian high-pass filter

Detection- correlation detector is used - in case of presence of spatial shift, a search over all possible shift is performed (128x128 position)

SummarySummary

The proposed methods span a wide complexity range from low complexity to considerable complexity In general, the more complex methods provide higher robustness

Most methods operate on uncompressed video; a few methods embed watermark directly into compressed video e.g., DCT embedding or the motion vector embedding, GOP structure embedding

Watermarking TechniquesWatermarking Techniques3. Audio Watermarking3. Audio Watermarking

Common ideas for audio Common ideas for audio watermarkingwatermarking

Compared to image and video, audio signal has much less samples per time interval amount of information which can be embedded is much lower

HAS (Human Audible System) is much more sensitive than the HVS

BoneyBoney

Propose A spread spectrum approach

* PN sequence is filtered in several stage to exploit masking effects of the HAS

* The watermark is low-pass filtered by using full audio compression/decompression scheme to guarantee that it survive audio compression

Tilki and BeexTilki and Beex Propose interactive television application where they embed information

into the audio components of a television signal

1. The information to be embedded is partitioned in blocks of 35bits

2. Each information bit is modulated using a sinusoidal carrier of a specific frequency with low amplitude and added to audio signal if the sinusoidal carrier for a specific bit is present=> “1” otherwise => “0”

* The frequencies of the sinusoidal carrier are above 2.4 kHz To reduce interference from the audio signal , the audio signal is attenuated at frequencies above 2.4 kHz

BenderBender

Propose phase coding

; use the phase information as a data space

1. For encoding, a Fourier Transform is applied and the phase value of each freq. component are lined up as matrix

2. Binary information can be added into this matrix by modifying the phase component.

* HAS is not very sensitive to the phase distortion of the sound

Watermarking TechniquesWatermarking Techniques4. other Multimedia DATA4. other Multimedia DATA

OhbuchiOhbuchi Propose embed visible and invisible watermarks into 3-D polygonal model

s

* This model comprise primitives like points, lines, ploygons, and ployhedrons

* Modify geometry or topology for watermarking

2 methods1) pseudorandomly selects sets of four adjacent triangles* embed information by displacing the vertices of the four triangles up to 1% of the shortest edge of the rectangular bounding box of the entire 3-D model

Cont-Cont-

2) pseudorandomly selects tetrahedron from the mesh and embeds information in the volume ratio of consecutive tetrahedron by modification of the vertices

HartungHartung Proposal

A spread-spectrum method for watermarking of MPEG-4 FAP’s

*MPEG-4 features model-based animation 3-D head modules using FAP (Facial Animation Parameter)

*There are FAP like “rotate head”, “open mouth” or “raise right corner-lip”

1. In order to embed information, the parameters first have to be estimated from the sequence

2. The watermarks are embedded into the animation parameter

*Adaptive amplitude attenuation prevent visible distortion of animated head level.

Cont-Cont-

* Interesting point is that the watermark is not embedded in pixels but in the semantics( the way the head and face move)

ConclusionConclusion

We reviewed the most important aspects design requirements, system issues, and techniques for digital watermarking

We have elaborated on numerous watermarking techniques for still images, video, audio, text, and other multimedia data.

Majority of techniques are similar and based on modulation with a PN signal.

Hypothesis test using correlation is used in the watermark recovery

Although working systems are available, research has to continue

Thanks for your attention

- Questions? -