Real Time Digital Watermarking System for Audio Signals Yuval Cassuto and Michael Lustig Supervisor: Shay Mizrachi Technion - Israel Institute of Technology.

Real Time Digital Watermarking System for Audio Signals

Yuval Cassuto and Michael Lustig

Supervisor: Shay Mizrachi

Technion - Israel Institute of Technology Department of Electrical Engineering

Signal and Image Processing Laboratory

World Pirate Music Business US $4.2 Billion

World Recording industry US $37 Billion

Motivation - Music Piracy

Internet music is virtually a 100% pirate medium

According to IFPI report

The Watermarking Concept

WWatermarking

system

owner signature:#1345234

Signature Embedding

Signature Detection

You win

Detection system

Adversary

W

owner signature:#1345234

• Embedded

• Inaudible

• Public Algorithm

• Damaged SignatureDamaged Audio

• Resolve Deadlock Keep Original

Human Auditory System

Signature Requirements

The watermarking problem

1 Generation of a unique, robust and hidden signature

2 Find an appropriate embedding method and location

3 Embedding

Signature Embedding Algroithm

FFTWatermarkColoring

Frequency Masking(Psychoacoustic model)

Pseudo RandomNoise Generation

Local KeyCalculation

Output

OriginalSignalSegment

Owner’s Key

+

W

#1345234

Local KeyCalculation

Pseudo RandomNoise Generation

WatermarkColoring

Frequency Masking(Psychoacoustic model)

FFT

+

OriginalSignalSegment

Owner’s Key

Output

1 1 0 1 0 0 0 1 1 0

+

1 1 0 1 0 0 0 1 1 0

+

1 1 0 1 0 0 0 1 1 0

+

1 0 1 0 0 0 1 1 0 1

+

1 0 1 0 0 0 1 1 0 1

+

1 0 1 0 0 0 1 1 0 1

+

0 1 0 0 0 1 1 0 1 0

+

W

W

Frequency MaskingHearing threshold

Threshold in QuietThreshold in Quiet

Signal Spectrum

Finding local maximum

Find Local Maxima

Threshold in Quiet

Find Local Maxima

Signal Spectrum

Tonal Components

Finding Tonal Components

Threshold in Quiet

Find Local Maxima

Tonal Components

Signal Spectrum

Finding Non-Tonal Components

Atonal Components

Threshold in Quiet

Find Local Maxima

Tonal Components

Atonal Components

Signal Spectrum

Tonal Masking components

Tonal Masking

Threshold in Quiet

Find Local Maxima

Tonal Components

Atonal Components

Tonal Masking

Signal Spectrum

Tonal Masking components

Tonal Masking

Threshold in Quiet

Find Local Maxima

Tonal Components

Atonal Components

Tonal Masking

Signal Spectrum

Non-Tonal Masking components

Atonal Masking

Threshold in Quiet

Find Local Maxima

Tonal Components

Atonal Components

Tonal Masking

Signal Spectrum

Atonal Masking

Total Masking components

Total Masking

Threshold in Quiet

Find Local Maxima

Tonal Components

Atonal Components

Tonal Masking

Signal Spectrum

Atonal Masking

Resulting Masking threshold

Total Masking

Threshold in Quiet

Find Local Maxima

Tonal Components

Atonal Components

Tonal Masking

Signal Spectrum

Atonal Masking

Resulting Masking threshold

Total Masking

Threshold in Quiet

Find Local Maxima

Tonal Components

Atonal Components

Tonal Masking

Atonal Masking

Total Masking

Signal SpectrumSignal Spectrum

0 50 100 150 200 2500

10

20

30

40

50

60

70

80

90

100

Frequency index

Dec

ibel

s

Owner’s Key

FFT Coloring

Pseudo RandomNoise Generation

Local KeyCalculation

+

Frequency Masking(Psychoacoustic model)

FFT

FFT Freq. masking

Masking Threshold

0 10 20 -20

0

20

40

60

80

100

Frequency [KHz]

embeddingFreq. maskingFFT

key Random noise

FFT NoiseFiltering

Frequency Masking(Psychoacoustic model)

Pseudo RandomNoise Generation

Local KeyCalculation

+

WatermarkColoring

Watermark Coloring

Masking threshold

Masking Threshold

White Watermark

White Watermark

Masking Threshold

Colored Watermark

Colored Watermark

Masking Threshold

White WatermarkInterception

Raised Watermark

Masking Threshold

White Watermark

Colored Watermark

White Watermark

Colored Watermark

#1345234

W

Signature Calculation

Correlation & Threshold

Gain Matching

Signature Calculation

OriginalSignal

TestedSignal

Owner'sKey

Decision

W

Signature Detection Algorithm

Gain Matching

Gain Matching

#1345234

W

OriginalSignal

TestedSignal

Owner'sKey

Signature Calculation

Decision

Correlation & Threshold

W

W

Signature Detection Algorithm

#1345234

W

OriginalSignal

TestedSignal

Owner'sKey

Signature Calculation

Decision

Correlation & Threshold

Gain Matching

W

W

Signature Detection Algorithm

Why Real-time?

•Portable Devices

•Cellular phones

Real Time Applications

•Audio Streaming

W

W

•Live Broadcast

Real-Time Implementation

• The Problem: In the Windows application, watermark

embedding time is x8 longer than playing time (@44.1[KHz]).

• The Solution:

• MATLABTM simulation• PC application: Embedding & Detection• TMS320C54x DSP embedding

implementation. • TIGER 5410/PC real-time embedding

application.

Development Phases

DSP Implementation Challenges

• Fixed Point• Speed• Memory• Accuracy• Architecture Utilization• Parallel Execution

• Optimization• Capacity• I/O Synchronization

Algorithm Specific Implementation Challenges

• Adaptive Masking Threshold Log-Scale Spectrum Calculation Identifying Masking Components. Calculating Masking Curves.

• Watermark Embedding Creating The Watermark. Coloring The Watermark.

0 50 100 150 200 2500

10

20

30

40

50

60

70

80

90

100

Frequency index

dB

Adaptive Masking Threshold

115 [frames/sec]!{Non Linear Frequency Metric}

(Bark Units)Logarithmic Scale

Masking Threshold Implementation - masking curves

Challenge:• Masking curves are Not

Linear• Not Shift-Invariant

Implementation

• Static Bark Addressing

• Constructing efficient Look-Up Tables

Result:

No Conditional Operations!

0 50 100 150 200 2500

10

20

30

40

50

60

70

80

90

100

Frequency index

dB

Masking Curves

Conditional Operations CPU time

Masking Threshold Exponential Model

Challenge:• Components and Spectrum are in Log-Scale• Threshold filter is in Linear-Scale.• Wide range of values.• Vast usage of ExpLog Transforms (more than 200,000 per

second).

Masking Threshold Exponent Implementation

Calculate ex , x is within a wide range.

• First Approach: Taylor approximation.

• Second Approach: Look-Up table

• Hybrid Approach: exp(a+b) = exp(a)exp(b) a Integer , small LUT b fraction [0,1) Taylor app.

217 words, too Big , too slow

Implementation ChallengesWatermark Coloring

Challenge:

• Watermark Coloring

0 -20 0

20 40 60 80

100 Threshold

Frequency [KHz]

Log

scal

e

512 Tap Filter

Implementation:

• Time Domain Convolution O(n2)

• Frequency Domain Multiplication O(n)

?

Watermark Coloring In Frequency Domain

Facts:

• No Zero-Padding is done.

• Frequency Domain Cyclic ConvolutionExplanation:

•Watermark is a Pseudo Random Noise

•Watermark requirements still achieved –Ability to regenerate the same colored WM using the source and a public key.–WM spectrum matches Psycho-Acoustic model (Inaudibility not affected) .

Filtering in frequency domain is appropriate!

Real-Time optimization• Speed

Psycho acoustic model tables dB Ln. PN26: Averaging 2x indices, Divisions shift. Optimized SQRT,EXP and LOG assembly implementations.

• Fixed Point Optimal representation to each block. 32 bit operations: normalized and saved using dynamic Q 16bit

representations.

• Memory Optimization of memory only fast internal memory used.

DSP Application software

System Schematics

PC

Host Application

software

DSP boardISA BUS

Data via HPI

Sync. By interrupts and status/control registers

Modes of operation

input output

Live mode: Audio input – Audio output

DAC

interrupt

ADC

interrupt

Modes of operation

input output

Live mode: Audio input – Audio output

DAC

interrupt

ADC

interrupt

sync

Conclusion

• Speed - 10 times faster than the PC application

• Quality - Strong inaudible watermark (-22dB)

• Completeness - Full DSP and HOST apps.

• Portability - Low power , low cost DSP (C54)

• Capacity - Easily upgradeable to a Multi-channel system

• Innovation - Non-Standardized field

• Commercial Value - Huge potential market

• The Signal and Image processing lab staff headed by Prof. Malah and Mr. Peleg - for the technical assistance

• TI - for the equipment, support , encouragement and invitations to ICASSP2000 and the “3rd European DSP Education and Research Conference” .

Acknowledgements

For more information visit the lab’s web site:

http:\\www-sipl.technion.ac.il