Top Banner
Distortion Minimisation by Coding Theory CSM25 Secure Information Hiding Dr Hans Georg Schaathun University of Surrey Spring 2008 Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 1 / 40
65

Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Jun 01, 2020

Download

Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Distortion Minimisation by Coding TheoryCSM25 Secure Information Hiding

Dr Hans Georg Schaathun

University of Surrey

Spring 2008

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 1 / 40

Page 2: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Learning Objectives

Get an overview of basic concepts in coding theoryUnderstand the principle of matrix (aka. wet paper) coding

Suggested Reading

Raymond Hill: A first course in coding theory Oxford AppliedMathematics and Computing Science Series. 1986

Suggested Reading

Fridrich et al: ‘Writing on Wet Paper’ IEEE Trans. Signal Proc. Oct2005, pp. 3923-,

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 2 / 40

Page 3: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Coding Theory Overview

Outline

1 Coding TheoryOverview

2 Cosets and syndromesCosetsReducing distortionSyndromeDesign issuesThe Code Parameters

3 Matrix and wet paper codingMatrix codingWet paper codes

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 3 / 40

Page 4: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Coding Theory Overview

OverviewMatrix Coding and Error-Control Coding

Duality betweenError-Control CodingMatrix Coding (e.g. F5)

First: a brief study of error-controlthis will enable us to look further at matrix coding

A linear [n, k ]q code C islinear set (vector space) of qk codewordseach codeword is a vector (c1, c2, . . . , cn)over an alphabet (field) F of q element

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 4 / 40

Page 5: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Coding Theory Overview

OverviewMatrix Coding and Error-Control Coding

Duality betweenError-Control CodingMatrix Coding (e.g. F5)

First: a brief study of error-controlthis will enable us to look further at matrix coding

A linear [n, k ]q code C islinear set (vector space) of qk codewordseach codeword is a vector (c1, c2, . . . , cn)over an alphabet (field) F of q element

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 4 / 40

Page 6: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Coding Theory Overview

OverviewMatrix Coding and Error-Control Coding

Duality betweenError-Control CodingMatrix Coding (e.g. F5)

First: a brief study of error-controlthis will enable us to look further at matrix coding

A linear [n, k ]q code C islinear set (vector space) of qk codewordseach codeword is a vector (c1, c2, . . . , cn)over an alphabet (field) F of q element

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 4 / 40

Page 7: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Coding Theory Overview

Basic definitions

A field F means (essentially) thatMultiplication/division and addition/subtraction are defined on F∀a, b ∈ F , we have a + b ∈ F and a · b ∈ F .∃0, 1 ∈ F , s.t. a + 0 = 0, a · 1 = a, a · 0 = 0 etc.

The binary field F2:Addition is logical XORMultiplication is logical AND

A vector space (linear set) means that∀a, b ∈ C, we have a + b ∈ C.∀a ∈ F ,∀b ∈ C, we have ab ∈ C.

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 5 / 40

Page 8: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Coding Theory Overview

Basic definitions

A field F means (essentially) thatMultiplication/division and addition/subtraction are defined on F∀a, b ∈ F , we have a + b ∈ F and a · b ∈ F .∃0, 1 ∈ F , s.t. a + 0 = 0, a · 1 = a, a · 0 = 0 etc.

The binary field F2:Addition is logical XORMultiplication is logical AND

A vector space (linear set) means that∀a, b ∈ C, we have a + b ∈ C.∀a ∈ F ,∀b ∈ C, we have ab ∈ C.

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 5 / 40

Page 9: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Coding Theory Overview

Basic definitions

A field F means (essentially) thatMultiplication/division and addition/subtraction are defined on F∀a, b ∈ F , we have a + b ∈ F and a · b ∈ F .∃0, 1 ∈ F , s.t. a + 0 = 0, a · 1 = a, a · 0 = 0 etc.

The binary field F2:Addition is logical XORMultiplication is logical AND

A vector space (linear set) means that∀a, b ∈ C, we have a + b ∈ C.∀a ∈ F ,∀b ∈ C, we have ab ∈ C.

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 5 / 40

Page 10: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Coding Theory Overview

Example

The [7, 4]2 Hamming code.Generator matrix

G =

1 0 0 0 1 1 10 1 0 0 1 1 00 0 1 0 1 0 10 0 0 1 0 1 1

(1)

Take any message m = (m1, m2, m3, m4) ∈ F 42

Encoding: c = m · G.Any such c is a codeword

The code is C = {c : ∃m, c = m · G}.

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 6 / 40

Page 11: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Coding Theory Overview

Encoding a message

message m = (0101)

[0110] ·

1 0 0 0 1 1 10 1 0 0 1 1 00 0 1 0 1 0 10 0 0 1 0 1 1

= [0100110] + [0010101]

= [0110011]

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 7 / 40

Page 12: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Coding Theory Overview

The Hamming CodeA diagrammatic view

m1m2

m3

m4

����

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 8 / 40

Page 13: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Coding Theory Overview

The Hamming CodeA diagrammatic view

p1

m1

p2

m2

p3

m3

m4

����

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 8 / 40

Page 14: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Coding Theory Overview

The Hamming CodeA diagrammatic view

p1

m1

p2

m2

p3

m3

m4

s1 s2

����

s3

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 8 / 40

Page 15: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Coding Theory Overview

The Hamming CodeA diagrammatic view

p1

m1

p2

m2

p3

m3

m4

s1 = 0 s2 = 0

����

s3 = 0

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 8 / 40

Page 16: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Coding Theory Overview

The Hamming CodeEncoding

p1

0

p2

1

p3

1

0

s1 = 0 s2 = 0

����

s3 = 0

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 8 / 40

Page 17: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Coding Theory Overview

The Hamming CodeEncoding

0

0

1

1

1

1

0

s1 = 0 s2 = 0

����

s3 = 0

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 8 / 40

Page 18: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Coding Theory Overview

The Hamming CodeOne error in channel

0

0

1

0

1

1

0

s1 s2

����

s3

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 8 / 40

Page 19: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Coding Theory Overview

The Hamming CodeDecoding

0

0

1

0

1

1

0

1 0

����

1

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 8 / 40

Page 20: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Coding Theory Overview

The Hamming CodeDecoding

0

0

1

0

1

1

0

1 0

����

1

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 8 / 40

Page 21: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Coding Theory Overview

The Hamming CodeDecoding

0

0

1

1

1

1

0

0 0

����

0

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 8 / 40

Page 22: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Coding Theory Overview

Hamming Matrix CodingMessage and host

p1

m1

p2

m2

p3

m3

m4

s1 s2

����

s3

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 8 / 40

Page 23: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Coding Theory Overview

Hamming Matrix CodingThe message

p1

m1

p2

m2

p3

m3

m4

0 0

����

1

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 8 / 40

Page 24: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Coding Theory Overview

Hamming Matrix CodingThe host signal

0

1

0

1

1

0

1

0 0

����

1

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 8 / 40

Page 25: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Coding Theory Overview

Hamming Matrix CodingEmbedded message

0

1

0

1

1

0

1

0 0

����

1

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 8 / 40

Page 26: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Coding Theory Overview

Hamming Matrix Coding

0

1

0

1

0

0

1

0 0

����

1

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 8 / 40

Page 27: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Cosets and syndromes Cosets

Outline

1 Coding TheoryOverview

2 Cosets and syndromesCosetsReducing distortionSyndromeDesign issuesThe Code Parameters

3 Matrix and wet paper codingMatrix codingWet paper codes

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 9 / 40

Page 28: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Cosets and syndromes Cosets

A Sample Code

G =

[1 1 1 0 00 0 1 1 1

]

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 10 / 40

Page 29: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Cosets and syndromes Cosets

The Cosets

00000 00001 00010 00100 01000 10000 01001 0101011100 11101 11110 11000 10100 01100 10101 1011000111 00110 00101 00011 01111 10111 01110 0110111011 11010 11001 11111 10011 01011 10010 10001

4 codewords (blue)7 correctable, non-zero error patterns (red)8 co-sets (columns) r + C4 × 8 = 32 words of length 5Look up received word in the table

Go left to find sent wordGo up to find the error

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 11 / 40

Page 30: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Cosets and syndromes Cosets

The Cosets

00000 00001 00010 00100 01000 10000 01001 0101011100 11101 11110 11000 10100 01100 10101 1011000111 00110 00101 00011 01111 10111 01110 0110111011 11010 11001 11111 10011 01011 10010 10001

4 codewords (blue)7 correctable, non-zero error patterns (red)8 co-sets (columns) r + C4 × 8 = 32 words of length 5Look up received word in the table

Go left to find sent wordGo up to find the error

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 11 / 40

Page 31: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Cosets and syndromes Cosets

The Cosets

00000 00001 00010 00100 01000 10000 01001 0101011100 11101 11110 11000 10100 01100 10101 1011000111 00110 00101 00011 01111 10111 01110 0110111011 11010 11001 11111 10011 01011 10010 10001

4 codewords (blue)7 correctable, non-zero error patterns (red)8 co-sets (columns) r + C4 × 8 = 32 words of length 5Look up received word in the table

Go left to find sent wordGo up to find the error

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 11 / 40

Page 32: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Cosets and syndromes Cosets

The Cosets

00000 00001 00010 00100 01000 10000 01001 0101011100 11101 11110 11000 10100 01100 10101 1011000111 00110 00101 00011 01111 10111 01110 0110111011 11010 11001 11111 10011 01011 10010 10001

4 codewords (blue)7 correctable, non-zero error patterns (red)8 co-sets (columns) r + C4 × 8 = 32 words of length 5Look up received word in the table

Go left to find sent wordGo up to find the error

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 11 / 40

Page 33: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Cosets and syndromes Cosets

The Cosets

00000 00001 00010 00100 01000 10000 01001 0101011100 11101 11110 11000 10100 01100 10101 1011000111 00110 00101 00011 01111 10111 01110 0110111011 11010 11001 11111 10011 01011 10010 10001

4 codewords (blue)7 correctable, non-zero error patterns (red)8 co-sets (columns) r + C4 × 8 = 32 words of length 5Look up received word in the table

Go left to find sent wordGo up to find the error

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 11 / 40

Page 34: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Cosets and syndromes Cosets

Correctable error patterns

00000 00001 00010 00100 01000 10000 01001 0101011100 11101 11110 11000 10100 01100 10101 1011000111 00110 00101 00011 01111 10111 01110 0110111011 11010 11001 11111 10011 01011 10010 10001

Correctable error-patterns are chosen arbitrarilyWe may want to correct 10010, but then we cannot correct 01001Low-weight error-patterns are more likely,

hence chosen for correction

but how choose between 10010 and 01001?

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 12 / 40

Page 35: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Cosets and syndromes Cosets

Correctable error patterns

00000 00001 00010 00100 01000 10000 01001 0101011100 11101 11110 11000 10100 01100 10101 1011000111 00110 00101 00011 01111 10111 01110 0110111011 11010 11001 11111 10011 01011 10010 10001

Correctable error-patterns are chosen arbitrarilyWe may want to correct 10010, but then we cannot correct 01001Low-weight error-patterns are more likely,

hence chosen for correction

but how choose between 10010 and 01001?

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 12 / 40

Page 36: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Cosets and syndromes Cosets

Correctable error patterns

00000 00001 00010 00100 01000 10000 01001 0101011100 11101 11110 11000 10100 01100 10101 1011000111 00110 00101 00011 01111 10111 01110 0110111011 11010 11001 11111 10011 01011 10010 10001

Correctable error-patterns are chosen arbitrarilyWe may want to correct 10010, but then we cannot correct 01001Low-weight error-patterns are more likely,

hence chosen for correction

but how choose between 10010 and 01001?

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 12 / 40

Page 37: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Cosets and syndromes Reducing distortion

Outline

1 Coding TheoryOverview

2 Cosets and syndromesCosetsReducing distortionSyndromeDesign issuesThe Code Parameters

3 Matrix and wet paper codingMatrix codingWet paper codes

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 13 / 40

Page 38: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Cosets and syndromes Reducing distortion

A matrix code

00000 00001 00010 00100 01000 10000 01001 0101011100 11101 11110 11000 10100 01100 10101 1011000111 00110 00101 00011 01111 10111 01110 0110111011 11010 11001 11111 10011 01011 10010 10001

Assign a message to each column8 messages : 3 bits

4 codewords per message

F3/F4 would flip 1 12 bit on average

Use five coefficients, and embed any word in the columnflip as few bits as possible

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 14 / 40

Page 39: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Cosets and syndromes Reducing distortion

A simpler example

message 00 01 10 11codeword 1 000 001 010 100codeword 2 111 110 101 011

Embed 01.Cover-image has 100.

Hiding 001 requires change of two bitsHiding 110 requires change of one bits

We change the cover to 110 to represent 01.

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 15 / 40

Page 40: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Cosets and syndromes Reducing distortion

A simpler example

message 00 01 10 11codeword 1 000 001 010 100codeword 2 111 110 101 011

Embed two bitsF3/F4 would flip one bit on averageSometimes two, sometimes one

You never need to flip more than one bit25% of the time no flip is necessary

Flips 0.75 bits on averageMatrix coding reduces detectability by 25%

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 15 / 40

Page 41: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Cosets and syndromes Syndrome

Outline

1 Coding TheoryOverview

2 Cosets and syndromesCosetsReducing distortionSyndromeDesign issuesThe Code Parameters

3 Matrix and wet paper codingMatrix codingWet paper codes

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 16 / 40

Page 42: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Cosets and syndromes Syndrome

Parity check matrix

[n, k ] code CDescribed by k × n generator matrix GAlso, described by (n − k) × n parity check matrix H

where G · HT = 0Note, both G and H have full rank

that is, no linearly dependent rows

C = {c : HcT = 0}

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 17 / 40

Page 43: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Cosets and syndromes Syndrome

Hamming code

G =

1 0 0 0 1 1 10 1 0 0 1 1 00 0 1 0 1 0 10 0 0 1 0 1 1

, (2)

H =

1 1 1 0 1 0 01 1 0 1 0 1 01 0 1 1 0 0 1

(3)

Systematic form:G = [I|P] H = [PT|I]

where I is an identity matrix

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 18 / 40

Page 44: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Cosets and syndromes Syndrome

The Syndrome

Given a received word r,the syndrome is s = rHT

Each syndrome can be mapped to an error vector e ∈ F n

Syndrome decodingError patterns uniquely assigned to a syndrome can be corrected

s = 0 corresponds to no error

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 19 / 40

Page 45: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Cosets and syndromes Syndrome

Syndromes and Cosets

The syndrome is a property of the coset

H =

110000001110101

000 011 010 001 100 101 111 110

00000 00001 00010 00100 01000 10000 01001 0101011100 11101 11110 11000 10100 01100 10101 1011000111 00110 00101 00011 01111 10111 01110 0110111011 11010 11001 11111 10011 01011 10010 10001

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 20 / 40

Page 46: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Cosets and syndromes Design issues

Outline

1 Coding TheoryOverview

2 Cosets and syndromesCosetsReducing distortionSyndromeDesign issuesThe Code Parameters

3 Matrix and wet paper codingMatrix codingWet paper codes

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 21 / 40

Page 47: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Cosets and syndromes Design issues

The decoding problem

Encoding is simple matrix-vector multiplicationDecoding of arbitrary codes is NP-complete

Small codes can easily be decodedExhaustive searchSyndrome lookup table

Intractible for larger codes.

Some classes of codes are known with fast decoding algorithms

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 22 / 40

Page 48: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Cosets and syndromes Design issues

Block length

Compare two codes[400, 200] code C1 able to correct 60 errors[100, 50] code C2 able to correct 15 errors

We can use one codeword of C1 or four of C2C1 can correct any 60 errorsC2 can correct 60 errors if they are evenly distributed over the fourblocks

Since errors are random, they are rarely evenly distributedIn general, larger block lengths give better robustness.

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 23 / 40

Page 49: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Cosets and syndromes Design issues

Block lengths in F5Andreas Westfeld

k n changes information rate bits per change1 1 50.00% 100.00% 2.002 3 25.00% 66.67% 2.673 7 12.50% 42.86% 3.434 15 6.25% 26.67% 4.273 31 3.12% 16.13% 5.163 63 1.56% 9.52% 6.093 127 0.78% 5.51% 7.068 255 0.39% 3.14% 8.039 511 0.20% 1.76% 9.02

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 24 / 40

Page 50: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Cosets and syndromes The Code Parameters

Outline

1 Coding TheoryOverview

2 Cosets and syndromesCosetsReducing distortionSyndromeDesign issuesThe Code Parameters

3 Matrix and wet paper codingMatrix codingWet paper codes

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 25 / 40

Page 51: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Cosets and syndromes The Code Parameters

Hamming Distance

Two vectors

a = (a1, a2, . . . , an),

b = (b1, b2, . . . , bn).

Hamming distanced(a, b) = #{i : ai 6= bi}Number of positions where the vectors differ

The Hamming distance is a metric (distance) in mathematicalterms

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 26 / 40

Page 52: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Cosets and syndromes The Code Parameters

The Noisy ChannelClosest Neighbour Decoding

Alice sends cBob receives rAssumption: few errors more likely than many errors

i.e. probably d(c, r) is smallif r 6∈ C, look for c

minimising d(c, r).This is called closest neighbour decoding

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 27 / 40

Page 53: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Cosets and syndromes The Code Parameters

Minimum distance

[n, k , d ]q code CMinimum distance d is

smallest distance between two distinct codewordsd := mina,b∈C d(a, b) where a 6= b

Let e =⌊d−1

2

⌋spheres of radius e around each c ∈ Cnon-overlapping spheresclosest neighbour decoding returns centre of sphere

Up to e errors are uniquely decodable.

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 28 / 40

Page 54: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Cosets and syndromes The Code Parameters

Covering radius

DefinitionThe covering radius ρ is the largest distance between a word a 6∈ Cand the closest codeword c ∈ C.

Draw spheres of radius ρ around each codeword c ∈ CThese spheres covers the entire spaceSmaller spheres would not cover the space

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 29 / 40

Page 55: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Cosets and syndromes The Code Parameters

Perfect codes

DefinitionIf ρ = b(d − 1)/2c (= e), we say that the code is perfect.

The decoding spheres of radius e covers the spaceIt is impossible to add another codeword without

reducing the error-correction capacity.

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 30 / 40

Page 56: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Cosets and syndromes The Code Parameters

ExerciseCoding parameters

Let a code be given by generator matrix

G =

111111110000111

(4)

What are the parameters [n, k , d ]q?Find nFind kFind dFind q

Hint: you may want to list all possible codewords first.What is the covering radius ρ?

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 31 / 40

Page 57: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Matrix and wet paper coding Matrix coding

Outline

1 Coding TheoryOverview

2 Cosets and syndromesCosetsReducing distortionSyndromeDesign issuesThe Code Parameters

3 Matrix and wet paper codingMatrix codingWet paper codes

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 32 / 40

Page 58: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Matrix and wet paper coding Matrix coding

Matrix coding

[n, k ] matrix code C with low covering radius.k -bit message transmitted as the syndrome s.n-bit host signal corresponds to codeword/received wordEncoder: change as few bits as possible, such that

the resulting syndrome match the message

Decoder: multiply host by H to get syndrome s.

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 33 / 40

Page 59: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Matrix and wet paper coding Matrix coding

The main problem

Coding theory has focused onmaximising minimum distancecorrect many errors

Matrix coding requiresminimising covering radiusminimise distortion

Hence, less theory available for matrix coding

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 34 / 40

Page 60: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Matrix and wet paper coding Matrix coding

Implementation issues

The hard job is at the Sender.Embedding requires solving the decoding problem

which is NP-completelarge codes with good decoding algorithms

few (if any) known with low covering radius

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 35 / 40

Page 61: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Matrix and wet paper coding Wet paper codes

Outline

1 Coding TheoryOverview

2 Cosets and syndromesCosetsReducing distortionSyndromeDesign issuesThe Code Parameters

3 Matrix and wet paper codingMatrix codingWet paper codes

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 36 / 40

Page 62: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Matrix and wet paper coding Wet paper codes

Matrix code vs. wet paper code

Matrix codingforbids more than P changes in the hostallows changes in arbitrary positions

Wet paper codingallows changes in specified positions in the hostforbids changes in remaining positions

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 37 / 40

Page 63: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Matrix and wet paper coding Wet paper codes

Random codes

Random codes generally perform wellOnly problem is complex decoding

Fridrich et al proposed random codes for wet paper.

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 38 / 40

Page 64: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Matrix and wet paper coding Wet paper codes

The encoding problem

The hard job is at the encoderSlightly different problem from matrix codingHas to solve the equation

Hc = m

where c is the coverm is the message

Some entries of c are locked,remaining entries are unknowns in the equation

Solvable by Gaussian eliminationPolynomial O(n3)

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 39 / 40

Page 65: Distortion Minimisation by Coding Theory€¦ · Coding Theory Overview Outline 1 Coding Theory Overview 2 Cosets and syndromes Cosets Reducing distortion Syndrome Design issues The

Matrix and wet paper coding Wet paper codes

Simplification

Divide host signal into smaller blocksembed independently in each block

Reduce problem size at decoder (O(n3))Faster decoding

Shorter blocksLess optimal, i.e. slightly increased distortion

Dr Hans Georg Schaathun Distortion Minimisation by Coding Theory Spring 2008 40 / 40