Sensor Transforms to Improve Metamerism-Based Watermarking Mark S. Drew School of Computing Science Simon Fraser University [email protected] Raja Bala Xerox.

Sensor Transforms to Sensor Transforms to Improve Metamerism-Based Improve Metamerism-Based

WatermarkingWatermarkingMark S. Drew

School of Computing Science

Simon Fraser [email protected]

Raja Bala Xerox Research Center Webster,

Xerox [email protected]

2

Is it possible to use colour to digitally hide a watermark?

• i.e., embed information into a document in a way that is – invisible, – easily seen under a special environment meant to

reveal it, and – difficult to remove

• Here, we’re concentrating on visible watermarking, as opposed to purely digital watermarking == steganography [i.e., bit-flipping]

3

A brief history: • Bala et al. [CIC 2007 “Substrate Fluorescence: Bane or

Boon?”]– use fluorescent property of paper substrate to

make watermark visible under a portable UV lamp:

under D50 under UV

4

more:

5

… brief history… Can this be done

without UV – using just visible light? •Bala et al. [CIC 2009 “Watermark Encoding and Detection using Narrowband Illumination”]

– use narrowband LED lights – consider a pair of inks, with reflectance spectra that are metameric under D50:

max K

min K

So use an LED illuminant to emphasize difference == break metamerism:

6

Ok, how does this LED-light approach do?

“Watermark example for the strongest watermark signal. Top shows image photographed under daylight illumination. Bottom shows the same image photographed under the LED illumination”

7

So, not bad… Another example:

but could we not do better?

8

So far, we used illuminant metamerism, in order to see watermark.

We could use observer metamerism as well, to further break apart the colours,by interposing a camera+display system.

+

9

… brief history… Combating

cinema piracy in video by using observer metamerism to generate a

watermark if a camcorder illegally shoots a film:

•Doyen et al. [CIC 2009 “Description and Evaluation of the Variability of Human Color Vision in an Anti-Piracy Context”]

– use 3-primary projector for part of film, and 4 primaries for watermark : camcorder sees the difference – but, observer variability is substantial, and would have to be taken into consideration in determining a watermark/disturbance signal that is invisible in the cine theater to all "normal" observers, and yet different enough to show up when captured by a camcorder

10

For this paper [printed materials, not video], since we have a camera and hence digital image processing, we go on to transform into a new colour space:

so as to optimally disambiguate the foreground (the watermark) from the background

3 x 3

11

optimize on M

optimize on LED lighting as well

12

We start with 6 inks == 3 pairsthat are metameric under D50:

Again, these aremax-K, min-K metamer pairs

13

Again, we see max-differences:

Differences between spectral pairs

Start off by using fixed set of LEDs

LEDs chosen

500 580 660

14

These inks are metamers under D50:

1

2

34

56

illuminant metamerism: D65

illuminant metamerism:

LEDs

15

Add a camera:

D50

D65

LEDs

16

Ok, let’s go to a transformed colour space: matrix with M

-positivity of transformed colours-Maximize sum of squared differences between members of metamer pairs-set a normalization, to constrain M

Requirements of an optimization:

normalize imageto max brightness

17

Result of optimization

Camera, LED lighting

transformed

Progress over optimization:

18

Ok, for all pairs, optimizationimproves discriminability.

But, could we not do better by optimizingon the LED lights as well?

Suppose we have available 31 narrowband LED illuminants

maxima

19

Let the (float) weights over the set of LEDs be w

Define 93 x 93 matrix W as

Our optimization now reads:

Also define R = 31 x 6 x 3 set of RGB values under each of the LEDs, split into R1 and R2

20

optimized weights w

21

Optimized weights w

no matrixing

optimized matrixing

22

But, we may have only binary weights available: result is almost as good −

4 nonzero contributions from float weights w>0.2

re-optimized matrix M for these LED lights

23

Compare objective function:

… but, using non-perceptual differences

24

Conclusions:

-matrixing has a dramatic effect

-float weights are best, but optimizing for floats and then binarizing is almost as effective

25

-choice of inks: still metamers, but optimized blends along iso-colour loci

-ensure watermark invisible over normal lighting and over observers

-optimize on perceptual differences (could just use Jacobian of CIELAB transform)

-some metamer pairs don’t separate as much weight some pairs more

-model LEDs better

-could design lights just for this task [ACM Transactions on Graphics 2009]

-include more metamer pairs

Future work:

26

Thanks! Natural Sciences and

Engineering Research Council of Canada

Xerox Corporation