Error Correction

Training material of NHK-CTI

Error Detection andError Detection and


Error Detection andError Detection and

Error CorrectionError Correction

2007.7.24

1

2007.7.24

NHK Communications Training Institute

Hideo Tsuji [email protected]



ContentsContents

�Conception of error correction

�Fundamental of error correction

�Various error correction codes

2

�Applications in broadcasting field



Concept of error Concept of error

correctioncorrection

3

correctioncorrection



Program production to transmission Program production to transmission

for broadcastingfor broadcastingRecording

Editing

Transmission

Location

Studio

Editing

Reception

4

Field pick up

live

Reception



Error correction on channelError correction on channel

Convert to suitable signal for

transmissionData reductionDigital

data

Source

codingChannel

coding

Channel

(Satellite,

Terrestrial)

Information

(Video/audio/

data etc.

Noise

(Error)Modulation

5

Source

decoding

Channel

decodingDemodulation

Terrestrial)

Received

information



Example of error in digital signalExample of error in digital signal

Original

signal 0 1 0 0 1 0 1 0

Threshold

level

Small noise

Large noise

0 1 0 0 1 0 1 0

Threshold

level

6

noise

Error correction technology

0 1 0 1 1 0 1 0

Advantage of digitalization

Only the distinction between 0 and 1, Strong against noise → The high quality



Error factor generated on the channelError factor generated on the channel

�Random error

� Irregularly and disorderly generated in the

transmission bit sequencetransmission bit sequence

� Caused by a thermal noise and a wave distortion.

�Burst error

� Concentrate partially and occur continuously

� Caused by loss of synchronization, impulse noise and

persistent attenuation like fading, etc.

7

�Quality

� Evaluated by BER (Bit Error Rate)

� Ex.10-3 , 10-6



Error control method in communication systemError control method in communication system

�Reducing method of noise influence

�Boost up power

�Waste of energy

�Cause probability of interference

�Add redundancy to the transmission data according

to the rule and transmit

�Roughly there are two methods to control error on

channel

8

channel

�ARQ (Automatic Repeat Request)

� Issue the repeat request when the transmission error is detected on

the reception side.

�FEC (Forward Error Correction)

�Correct the error on the reception side while decoding



ARQ & FEC ARQ & FEC

�ARQ (Automatic Repeat Request)

�If a receiver detects an error, the receiver requests

retransmission to the transmitterretransmission to the transmitter

�Difficulty in dealing with a lot of receivers

�Used by the packet type communication including TCP/IP

�FEC: Forward Error Correction

�If a receiver detects an error, the receiver corrects using

redundant data

9

redundant data

�Used by broadcasting, mobile communication and storage

media (DVD/CD-ROM)



Sequence of ARQ & FECSequence of ARQ & FEC

data-1 data-2 data-2 data-3TX

ARQ

data-1

data-1

data-2 data-2 data-3


TX

RX

ACK

FEC

NAK

No error Error detect

Time

No error No error

ACK

10



TX

RX Time

Error

correction

Error

correction

Error

correction



Feature of ARQ & FECFeature of ARQ & FEC

ARQ FEC

Return circuit is not

Advantage

High reliabilityReturn circuit is not

needed

Strong against unexpected

errorEasy control as the system

Simple decoder Small delay

Return circuit is neededDifficult to deal with a

11

Disadvantage

Return circuit is neededDifficult to deal with a

very long burst error

Sometime large delay Large redundancy

Necessary to repeat sending

under the bad channel Complex decoder



Error correction schemeError correction scheme

�Principle of error correction

�Error correction can be performed adding �Error correction can be performed adding

redundancy to the information transmitted

�Encoder

�Add redundancy (parity check bits) under some rule

�Decoder

�Detect and correct errors using parity check bits

12

�Detect and correct errors using parity check bits



What is error detection and error What is error detection and error

correction?correction?

�Error may occur during transmission

�impossible decoding the data�impossible decoding the data

?Decoding

External Noise

A 0100

Coding

010?

�Add redundant data before transmission to

13

Error corrective coding

0100 010

DecodingA0100

Error Correction

A 0100 010?010

�Add redundant data before transmission to

detect/correct error when decoding



Parity checkParity check

�Add a parity bit to information data according to the following rule�if No. of “1” in the data is odd, add a parity bit “1”�if No. of “1” in the data is odd, add a parity bit “1”

�if No. of “1” in the data is even, add a parity bit “0”

�Able to detect 1 bit error

Transmitted sequenceInformation sequence Received sequence

14

10010001 100110011 !

00101110 000101110 101011000 ?

Transmitted sequenceInformation sequence Received sequence

110010001



Example of parity check code (ASCII)Example of parity check code (ASCII)

�ASCII stands for American Standard Code for

Information InterchangeInformation Interchange

�Comprised of 7 digits (0 to 127)d

�Add 1 bit for parity check comprising 8 bits

�Character “A” is expressed by (100 0001) assii

�for even parity (0100 0001)

�for odd parity (1100 0001)

15

�for odd parity (1100 0001)

�Receiver side check the number of “1” every 8

bit



ASCII codeASCII code

Character Binary Decimal Hexa-decimal

A 100 0001 65 41

B 100 0010 66 42B 100 0010 66 42

C 100 0011 67 43

D 100 0100 68 44

E 100 0101 69 45

F 100 0110 70 46

Character Binary Decimal Hexa-decimal

a 110 0001 97 61

16

a 110 0001 97 61

b 110 0010 98 62

c 110 0011 99 63

d 110 0100 100 64

e 110 0101 101 65

f 110 0110 102 66



Example of checksum (ISBN)Example of checksum (ISBN)

�ISBN (International Standard Book Number)

�The last digit is a check digit

�Calculation�(modulus 11 with weights 10-2)

� 4-87966-931-?

� 4x10+8x9+7x8+9x7+6x6+6x5+9x4+3x3+1x2

= 40 +72 +56 +63 +36 +30 +36 +9 +2

�The last digit is a check digit

17

= 40 +72 +56 +63 +36 +30 +36 +9 +2

=344

� 344 ⁄⁄⁄⁄ 11 = 31 … 3

� 11-3 = 8 � check digit is 8



Example of error detective codeExample of error detective code

� Is “0-440-21861-5” a correct ISBN code?

� 0x10+4x9+4x8+0x7+2x6+1x5+8x4+6x3+1x2� 0x10+4x9+4x8+0x7+2x6+1x5+8x4+6x3+1x2

= 0 +36 +32 +0 +12 +5 +32 +18 +2

=137

� 137 ⁄⁄⁄⁄ 11 = 12 … 5

� 11-5 = 6 � calculated check digit is 6

� this code is wrong!

18

� this code is wrong!



Example of errors (VTR)Example of errors (VTR)

Expand a portion of image

Error

correction

19

�Black and white spots are errors

(In case of pixel-wise coding )



Example of errorsExample of errors (broadcasting)(broadcasting)

Expand a portion of image

Error

correction

20

�Error is observed like a rectangular mass (block)

(In case of MPEG-2 TS Video coding)



Example of errorsExample of errors

Apart Spatially / Temporally

Transmission/

Recoding

Received message

“This is a pen” “This is b pen”≠

Original message

21

“This is a pen” ≠

Error1001101011 1011101011

In digital, 0→1,1→0In digital, 0→1,1→0In digital, 0→1,1→0In digital, 0→1,1→0In digital, 0→1,1→0In digital, 0→1,1→0In digital, 0→1,1→0In digital, 0→1,1→0



Error detection (example of sentence)Error detection (example of sentence)

Original

messageReceived message

Error

Recognized

“This is a pen” “This is a pbn”

Error detect

No word

messageReceived message Recognized

message

Can not correct

????

22

Error detect

Candidate of pbn is pen, pan, pun,pin etc.Candidate of pbn is pen, pan, pun,pin etc.

Impossible to correct, because there are several Impossible to correct, because there are several

candidates similar to pbn.candidates similar to pbn.

Can not correct



Error correction (example of sentence)Error correction (example of sentence)

OriginalReceived message

Error

Recognized Original


“This is a pen” “This is b pen”

Error detect

Grammar error

Recognized

message

Correct

“This is a pen”

23

Error detect

We can correct the sentence, because if you know We can correct the sentence, because if you know

the grammar.the grammar.

Correct



Error in digital transmissionError in digital transmission

OriginalReceived sequence

Error

Recognized Original

sequenceReceived sequence

1001101011 1011101011

Can not detect error

Recognized

sequence

Can not correct

??

24

It is unknown whether this is right or not, It is unknown whether this is right or not,

because we have no rule to check it.because we have no rule to check it.



Example of creating Example of creating codewordscodewords

(digital transmission)(digital transmission)

Rule: Add one bit to the original message, and Rule: Add one bit to the original message, and

Original

sequence［Number of “1”］

Add

bit Codeword

sequence

［number of “1”］

Rule: Add one bit to the original message, and Rule: Add one bit to the original message, and

make the number of “1” even number.make the number of “1” even number.

25

sequence

0101 ［2(even)］ 0

bit

+

sequence01010

=［2(even)］

1011 ［3(odd)］ 1 10111 ［4（even）］



Error detection (digital transmission)Error detection (digital transmission)

Original Codeword ReceivedError

Recognized

?

?

Original

sequence

0101

Codeword

sequence

01010

1011 10111

Received

sequence

Recognized

sequence

01110

10101


［3(odd)］

［3(odd)］

26

even number of “1” Error detect Can not correct



Example of creating Example of creating codewordscodewords

(digital transmission)(digital transmission)

Rule: Add the same two bits to the original Rule: Add the same two bits to the original

Original bit

0

Codewords

sequence

00000

Add bits

Rule: Add the same two bits to the original Rule: Add the same two bits to the original

one bitone bit

27

0 00000+ =1 11 111



Error correction (digital transmission)Error correction (digital transmission)

Original

bit

Codeword

sequence

Received

sequence

ErrorRecognized

sequencebit

0

sequence

000

1 111

sequence

001

1000

1

1

000

111

111

110

101

011

000

111

000

111

111

0

1

0

1

1

28

000 or 111 Error detect Correct

1 111 011 111 1

Invalid

codeword

find the

nearest

codeword



X11 X12 X13

X21 X22 X23

H1

H2

1.Decide all H, as the sum of row will

be even number

2.Decide all V, as the sum of column

Example of error correctionExample of error correction

X21 X22 X23

X31 X32 X33

H2

H3

V1 V2 V3 P

2.Decide all V, as the sum of column

will be even number

3. Decide P, as the sum of H, V, and P

will be even number

0 1 0

1 1 0

1

0

← Odd number0 1 1

1 1 0

1

0

29

1 1 0

1 0 1

0

0

0 0 1 0

↑

Odd number

1 1 0

1 0 1

0

0

0 0 1 0



Key pointKey point

For error correction or detection,

Create codeword from information bits

(original message) under some rule.

30



Is error correcting Is error correcting

code almighty?code almighty?

31

code almighty?code almighty?



Misscorrection (example of sentence)Misscorrection (example of sentence)

Original


Error

Recognized message

Received message

“This is a pen” “This is a pbn”

Error detect

No word

Recognized

message

Try to correct

“This is a pun”

32

Error detect

Candidate of pbn is pen, pan, pun,pin etc.Candidate of pbn is pen, pan, pun,pin etc.

Force to correct the nearest wordForce to correct the nearest word

Try to correct



Error detection (digital transmission)Error detection (digital transmission)

Original

bit

Codeword

sequence

Received

sequence

Recognized

sequencebit

0

sequence

000

1 111

sequence

101

1000

1

1

000

111

111

110

101

011

111

111

000

111

111

1

1

0

1

1

2 bits

1 bit

1 bit

1 bit

1 bit

Correct

properly

Misscorrection

33

000 or 111 Error detect

1 111 011 111 1

Invalid codeword

find the nearest codeword

1 bit



Undetected error (example of sentence)Undetected error (example of sentence)

OriginalReceived message

Error

Recognized Original


“This is a pen” “This is a pun”

The grammar is right

Recognized

message

“This is a pun”

34

The grammar is right



Undetected error (digital transmission)Undetected error (digital transmission)

Original Codeword Received

Error

Recognized Original

sequence

0101

Codeword

sequence

01010

1011 10111

Received

sequence

Recognized

sequence

11110

10100


［4(even)］

［2(even)］

1111

1010

35

even number of “1”

Undetected error

number of “1” is even.

These are codewords.



Key pointKey point

Number of correctable errors is finite.Number of correctable errors is finite.

If the number of errors exceeds the error

correction capability,

� Misscorrection

� Undetected error might occur

36

� Undetected error might occur



Fundamental of Fundamental of

error correctionerror correction

37

error correctionerror correction



Example of Example of codewordscodewords and Hamming distanceand Hamming distance

InformationCodeword

［Number of “1”］Add

bit ［number of “1”］

�Codewords by even parity

Information

00

01

10

11

［Number of “1”］

［0(even)］［1(odd)］［1(odd)］［2(even)］

000

011

101

110

0

1

1

0

bit

+ ＝


［0(even)］［2(even)］［2 (even) ］［2 (even) ］

�Hamming distance: dH

38

Number of different bits between the two corresponding

codewords.

Codeword 1 0 1

Codeword 1 1 0

dH=2

�Hamming distance: dH



Example of Example of codewordscodewords and Hamming distanceand Hamming distance

Minimum value of Hamming distance of all codewords.

�Minimum Hamming distance

Codewords

000

011

101

110

Minimum value of Hamming distance of all codewords.

All the codewords are 2bit or more different.

Minimum Hamming distance =2

39

110

It doesn't become a codeword by one bit error.

One bit error can be detected.



Geometrical allocation of codewordsGeometrical allocation of codewords

Z

(001) (011)Codeword

Y(010)

(101)

(000)

(111)000 011

101 110

d :Hamming distance

Invalid codeword

Codeword

40

X(100)

(110)dH:Hamming distance

dH=1

In case of 1 bad bit →Code of one Hamming distance =Invalid codeword

In case of 2 bad bit →Code of two Hamming distance =Another codeword



Image of code spaceImage of code space

Code space (finite)

�In case of Hamming code (7,4)�In case of Hamming code (7,4)

all number of word:128=(27),

codeword:16(=24),

Invalid codeword: 112=(128-16)

41

CodewordInvalid codeword

�Minimum Hamming distance dmin:

�Smallest Hamming distance between any pairs of code vectors in the code

�Determine error detecting or error correcting capability



Error detection and minimum Hamming distanceError detection and minimum Hamming distance

Hamming code (7,4)

0000 000

0001 011

0010 110

0011 101

0100 111

0101 100

Can not become codeword within

2bit errors

1000 101

1001 110

1010 011

1011 000

1100 010

1101 001

minimum Hamming distance=3

42

0101 100

0110 001

0111 010 Possible 2bit error detection

1101 001

1110 100

1111 111

Information, Parity



Error detection and minimum Hamming distanceError detection and minimum Hamming distance

dmin≧≧≧≧s+1

Cn=(x1,x2,・・,xN）

C3 dmin≧≧≧≧s+1

Error detection area

Possible to detect less than

or equal s bitss 1

C2C1

s: Number of error detecting

capability

43

Codeword

Invalid codeword

dmin

C2C1



Error correction and minimum Hamming distanceError correction and minimum Hamming distance

The nearest codeword becomes

the codeword as an original

codeword in single bit error

Hamming code (7,4)

0000 000

0001 011

0010 110

0011 101

0100 111

0101 100

1000 101

1001 110

1010 011

1011 000

1100 010

1101 001

minimum Hamming distance=3

44

0101 100

0110 001

0111 010Possible single error correction

1101 001

1110 100

1111 111

Information, Parity



Error correction and minimum Hamming distanceError correction and minimum Hamming distance

C3 dmin≧≧≧≧2t+1

Cn=(x1,x2,・・,xN）

C3

Error correction area

Possible to correct less

than or equal t bits

dmin≧≧≧≧2t+1

t: Number of error

correcting capability

45

t t1

dmin

C2C1

Codeword

Invalid codeword



Various error Various error

correction codescorrection codes

46

correction codescorrection codes



Classification of error correction code Classification of error correction code Hamming code

BCH code

Golay code

Random error

correction code

Block code

Reed-Solomn code

Fire code

Wyner-Ash code

Punctured code /

Viterbi decode

Difference-set cyclic code

Burst error

correction code

Random error

correction code

117 47

Concatenated code

Hangelberger code

Iwatare code

Sequential decode

Turbo code/

Soft decision decode

Convolutional code

Burst error

correction code

Product code


Information

Type of error correction codeType of error correction code

Convolutional codeBlock code

Codeword

�Coding is done independently from

other blocks

�Information message is divided into

�The information coded in the

past influences on the current

coding

48

�Information message is divided into

the constant length

�Able to be a system based on the

algebra

�When t is assumed to be a capable

number of error corrections, the block

code is often written as (n,k,t).

coding

�The error correction capability

is higher than the block coding

�Viterbi decoding is used for

decoding


Block code & Block code & ConvolutionalConvolutional codecode

Information bit Check bit

n-kkn�Block codeword c(n,k)

�Key word; code rate: k/n

Xoutput

Modulo-2 addition

Information bit Check bit

Function

D ＋Delay

�Convolutinal coder

�Key word; code rate:1/2, constraint length:7

49

＋

＋

D D D D D D

Xoutput

Youtput

G1=171oct

G2=133oct

Input

113



Expression of polynomialExpression of polynomial

��The coefficient of nThe coefficient of n--thth power of x in polynomial power of x in polynomial

expression shows the corresponding bit position expression shows the corresponding bit position

in the sequencein the sequencein the sequencein the sequence

�Code comprising of n bit sequence c=(c1, c2, c3, ･････, cn)

is expressed by following polynomial

c(x) = c x + c x + + c x + c1

n-1

2

n-2

n-1 n⋅⋅⋅⋅⋅

50

�Example c=(1,1,0,0,0,1)

1+x+ x= c(x) 45



Operation for code processingOperation for code processing

�Operation 0,1,,,,Xn（ n=0,1,2,3，….）�Calculation of 0 and 1

�(Addition) 0+0=0, 0+1=1, 1+0=1,1+1=0

modulo2,Exclusive ORmodulo2,Exclusive OR�(Multiplication) 0×0=0, 0×1=0, 1×0=0, 1×1=1

�Operation including X

51

�Operation including X�(Addition ) 0+X=X, 1+X=1+X, X+X=0

�(Multiplication) 0×X=0, 1×X =X, X×X = X2



Encoding single error detectionEncoding single error detection

i = (i , i , , i )1 2 k⋅ ⋅ ⋅ ⋅ ⋅

c = (c , c , , c , c )⋅ ⋅ ⋅ ⋅ ⋅

Information bit sequence

c = (c , c , , c , c ) 1 2 k k + 1⋅ ⋅ ⋅ ⋅ ⋅

c = ic = i c = ic = c + c + + c

1 1

2 2

k k

k + 1 1 2 k

⋅

⋅ ⋅ ⋅ ⋅ ⋅

Codeword sequence

Coding

Parity check bit

Redundant bit

52

c = c + c + + ck + 1 1 2 k⋅ ⋅ ⋅ ⋅ ⋅Parity check bit

c + c + + c + c = 01 2 k k + 1⋅ ⋅ ⋅ ⋅ ⋅Parity check equation

Parity check equation is necessary and sufficient Parity check equation is necessary and sufficient

condition to be a codewordcondition to be a codeword



Hamming codeHamming code

53



Hamming code (7,4)Hamming code (7,4)

i = (i , i , i , i )1 2 3 4

c = (c , c , c , c , c , c , c )1 2 3 4 5 6 7

Information bit sequence

Codeword sequence c = (c , c , c , c , c , c , c )1 2 3 4 5 6 7

c = ic = ic = ic = ic = c + c + c

1 1

2 2

3 3

4 4

Codeword sequence

Coding

54

c = c + c + cc = c + c + cc = c + c + c

5 1 2 3

6 2 3 4

7 1 2 4

Parity check bit



How to find parity check bits?How to find parity check bits?

C1 C2 C3 C4 C5 C6 C7

C1 0 C1 C1

1011

C1 C2 C1+C3 C1+C2+C4

C2 C1＋C3 C１＋C4 C5

C2 0 C2 C2

C１＋C3 C1＋C2＋C4 C2＋C5 C6

C１＋C3 0 C１＋C3 C1＋C3

C1＋C2＋C4 C1＋C2＋C3＋C5 C1＋C3＋C6 C7

C1＋C2＋C4 0 C1＋C2＋C4 C1＋C2＋C4

C ＋C ＋C ＋C C ＋C ＋C +C C ＋C ＋C ＋C

55

C1＋C2＋C3＋C5 C2＋C3＋C4+C6 C1＋C2＋C4＋C7

C1＋C2＋C3＋C5 ＝0 C2＋C3＋C4+C6 ＝0 C1＋C2 ＋C4＋C7＝0

C5 ＝ C1＋C2＋C3 C6 ＝ C2＋C3＋C4 C7＝C1＋C2 ＋C4

The codeword is divisible by the generator bit sequence, therefore

Therefore parity check bit becomes as follows



Parity check equationParity check equation

�Transform the parity check equation

�The necessary and sufficient condition to be

0 = c+c+c+c 5 321

�The necessary and sufficient condition to be

a codeword is “parity check equation =0”

�Hamming (7,4) parity check equation

56

0 = c+c+c+c

0 = c+c+c+c

0 = c+c+c+c

7 4 21

6 432



Channel modelChannel model

e(0,0,1,0,0)

Noise source Error pattern

e + c =r Received sequence

e(0,0,1,0,0)

Received sequence

r=c+e

Transmitted sequence

c(1,0,1,1,1)c(1,0,0,1,1)

57

)r , ,r ,(r =

)e+c , ,e+c ,e+(c =

e + c =r

1+k21

1+k1+k2211

⋅⋅⋅⋅⋅

⋅⋅⋅⋅⋅

Received sequence



Syndrome of Hamming code (7,4)Syndrome of Hamming code (7,4)

�Syndrome s：

�Syndrome (s) is the substitution of the element in

received sequence (r) for the parity check equationreceived sequence (r) for the parity check equation

s = r + r + r + r = c + e + c + e + c + e + c + e = e + e + e + e

s = r + r + r + r = c + e + c + e + c + e + c + e

1 1 2 3 5

1 1 2 2 3 3 5 5

1 2 3 5

2 2 3 4 6

58

= c + e + c + e + c + e + c + e = e + e + e + e

s = r + r

2 2 3 3 4 4 6 6

2 3 4 6

3 1 2 4 7

1 1 2 2 4 4 7 7

1 2 4 7

+ r + r = c + e + c + e + c + e + c + e = e + e + e + e



Syndrome for single errorSyndrome for single error

e1 e2 e3 e4 e5 e6 e7 s1 s2 s3

Error pattern Syndrome

1 0 0 0 0 0 0 1 0 1

0 1 0 0 0 0 0 1 1 1

0 0 1 0 0 0 0 1 1 0

0 0 0 1 0 0 0 0 1 1

0 0 0 0 1 0 0 1 0 0

0 0 0 0 0 1 0 0 1 0

59

0 0 0 0 0 1 0 0 1 0

0 0 0 0 0 0 1 0 0 1

0 0 0 0 0 0 0 0 0 0

Error patterns are substituted into the previous page syndrome S1, S2, S3,

then obtain Syndrome pattern


Example of Hamming code (7,4)Example of Hamming code (7,4)

a1 a2 a3 a4 p1 p2 p3

Parity codeInformation code

4322

3211

aaap

aaap

aaap

⊕⊕=

⊕⊕=

⊕⊕=

1 0 1 1 0 0

1 0 0 1 0 0 0

a1 a2 a3 a4 p1 p2 p3

1 1 0

s1 s2 s3

Syndrome

Transmission code

Reception code

Degradation

through channel

Errordetect

Errorcorrect

4213 aaap ⊕⊕=0

S1 S2 S3 Error position

1 0 1 a1

1 1 1 a2

1 1 0 a3

0 1 1 a4

1 0 0 P1

0 1 0 P2

0 0 1 P3

60

1 0 0 1 0 0 0

1 1

q1 q2 q3

1 1 0

+

+

+Detection of bit error

position by the

Syndrome value

correct

(0 1)

0

4213

4322

3211

aaaq

aaaq

aaaq

⊕⊕=

⊕⊕=

⊕⊕=

0 0 0 None



Example of Hamming coderExample of Hamming coder

cC4

c3

c2

c1

c7

c

c4

c3

c2

c1

61

c6

c5



Example of Hamming decoderExample of Hamming decoder

c4

c3

c4

c3c3

c2

c1

c7

c3

c2

c1

62

c6

c5



As an example of the rule, if the information

polynomial is divided by some signal

Important point (CRC)Important point (CRC)

polynomial is divided by some signal

(generator polynomial) and the remainder is

added to the information, information can be

divisible. CRC (Cyclic Redundancy Check)

� divisible → No error

63

� divisible → No error

� indivisible → Error



Example of CRCExample of CRC

�Principal

�Information desired to transmit data is 5,678

�As a rule, decide divisor as 98 beforehand

�2 digits redundant data is added 5,678 567,800

�Divide 567,800 by 98, remainder becomes 86

�Add (98-86=12) to the 567,800

�567,800+12=567,812 is divisible by 98

�Transmitting information is 567,812

64

�Transmitting information is 567,812

�In receiver side, divide the received data by 98, if there is

no remainder, no error, it is true data, discard 2bit from

LSB,

...........otherwise erroneous data



Coding and decoding by generator polynomialCoding and decoding by generator polynomial

EncodingEncode codeword polynomial c(x) which is divisible by

generator polynomial g(x)

)()()()()( xgxqxpxixxc kn =+= −

Decoding

generator polynomial g(x)

{ } )(mod)()()(mod)()( +== xgxexcxgxrxs

Divide received polynomial r(x) by g(x), which contains error

polynomial e(x),

65

From syndrome polynomial s(x), find the syndrome pattern and

correct the reception message and get the right codeword

{ }

g(x)mode(x)

)(mod)()()(mod)()(

=

+== xgxexcxgxrxs



How to create CRC How to create CRC

�i(x): Information polynomial i (i1,12,,,,ik:kbits)

�f(x): Shift i(x) to the left by (n-k) bits for CRC (n-

k)bits, f(x)=xn-ki(x)k)bits, f(x)=xn-ki(x)

�If f(x) is divide by generator polynomial g(x), then

the quotient is q(x) and remainder is p(x), therefore

f(x) is expressed f(x)=q(x)g(x)+p(x)

�Add both side p(x) to examine the codeword

polynomial c(x)=f(x)+p(x)

66

polynomial c(x)=f(x)+p(x)� Left side f(x)+p(x)= xn-ki(x)+p(x)

� Right side q(x)g(x)+p(x)+p(x)=q(x)g(x)

� Therefore codeword polynomial c(x)=xn-ki(x)+p(x) is divisible by g(x)

� Codeword c (i1,12,,,,ik,,p1p2,,,pn-k:nbits)



Example of CRCExample of CRC

�Information code i=(1,0,1,0) �Polynomial expression of information code

� i(x)=x3+x� i(x)=x3+x

�Shift i(x) by (n-k:3) bit to the left � f(x)=x3i(x)=x6+x4

�Divide f(x) by generator polynomial g(x)=x3+x+1�Find the remainder p(x):CRC polynomial

1)( 3

46

+=+

= xxxxf

remainder p(x)=x+1

67

�Therefore c(x)=f(x)+p(x)=x6+x4+x+1� Convert to information code c=(1,0,1,0,0,1,1)

11)(

)( 3

3+=

++

+= x

xx

xx

xg

xfremainder p(x)=x+1



CRC generation circuit CRC generation circuit

g

x1 x2 Xn-1 Xn

D0 D1

g1g0gn-1

Dn

g2

Output: q(x)Input: f(x)

gi coefficient Di shift register modulo2

68

Input:f(x), Output:q(x), Remainder:R(x) D0,D1,D2,,,,Dn

Generator polynomial g(x)=g0+g1x+g2x2+ +gn-1x

n-1+xn

124



Review of polynomialReview of polynomial

�The coefficient of n-th power of x in polynomial

expression shows the corresponding bit position of the

code sequence

c ( x ) = c x + c x + + c x + c1

n -1

2

n - 2

n -1 n⋅ ⋅ ⋅ ⋅ ⋅

Code sequence c ＝ (c1, c2, c3, ･････, cn) is expressed by

polynomial

Information sequence i ＝ (i1, i2, i3, ･････, ik) is expressed by

polynomial

code sequence

69

i (x ) = i x + i x + + i x + i1

k -1

2

k - 2

k -1 k⋅ ⋅ ⋅ ⋅ ⋅

polynomial

Generator polynomial

Codeword)x(p)x(g)x(ix)x(c kn += − p(x): remainder



Hamming codeHamming code

Code length n

�Capable of single error correction code

ik-1 ・・

Information bit length: k

� Code length n = 2r - 1（bit）� Information bit length k = n - m（bit）

Parity bit length: m

・・ i1 i0 pm-1 ・ p0p1

70

� Information bit length k = n - m（bit）� Parity bit length m = rt （bit）� Minimum length dmin = 2t + 1 = 3 （bit） ( t = 1)

r：Integer of three or more

t：Number of error correcting capability



Hamming code (7,4)Hamming code (7,4)

「Hamming code (7,4)」

Code length n = 7，information bit length k = 4，parity bit

length m = 3length m = 3

Generator polynomial: X3+X+1

i3 i2 i1 i0

Information code Codeword

i3 i2 i1 i0 p2 p1 p0

71

i3 ii2 i1 i0 000

1011remainder p=（p2 p1 p0）

Expression of generator polynomial



How to find parity check bit How to find parity check bit

Information bit sequence (i3 i2 i1 i0)) is shifted to the left by

three bits, it becomes (i3 i2 i1 i0 000), then divide it by

generator polynomial, and find remainder

1 0 1 0 0 0 0

1 0 1 1

1 0 1 1

1 0 0 0

1 0 1 1

1 0 0 1

(i3) (i2) (i1) (i0) (p2) (p1) (p0)

generator polynomial, and find remainder

72

Parity check bit

1 0 1 1

0 1 1Information bit

Codewords is 1010011



Hamming codeword (7,4)Hamming codeword (7,4)

Information biti3 i2 i1 i0

Codewordi3 i2 i1 i0 p2 p1 p0

0000000011

0010111100

0011001100

0101010101

0000000011

0010111100

0011001100

0101010101

0011110011

0110100101

0101101010

73

1111111

0001111

0110011

1010101

1111111

0001111

0110011

1010101

1000011

1101001

0100101



Error example in Hamming code (7,4)Error example in Hamming code (7,4)

If error is generated to the first bit

Information bit sequence (1010)→codeword sequence (1010011)

If error is generated to the first bit

Codeword (1010011) →(0010011)

If error is generated to the second bit

Divide by generator polynomial:

(1011)→ remainder (101)

0010011

- 1011

101

74

1110011

- 1011

101011

- 1011

111

If error is generated to the second bit

Codeword (1010011) →(1110011)

Divide by generator polynomial:

(1011) → remainder (111)



Single bit error in Hamming code (7,4) Single bit error in Hamming code (7,4)

i error

Remainder divided by g(x) when one bit error occurs

i error i error i error p error p error p errori i i i p p p

Codeword

i3 error

101101101101101101101101101101

111111111111111111111111111111

110110110110110110110110110 110

011011011011011011011011011011

100100100100100100100100100 100

010010010010010010010010 010010

001001001001001001001001001001

i2 error i1error i0error p2 error p1 error p0 errori3 i2 i1 i0 p2 p１ p0

000000001

001011110

001100110

010101010

001111001

011010010

010110101

75

101101101101101101101101

111111111111111 111111111

110110110110 110110110110

011011011011011 011011011

100100100100 100100100100

010010010010 010010010010 010

001001 001001001001001 001001

11111111

00001111

00110011

01010101

11000011

01101001

10100101



Error correction by Hamming codeError correction by Hamming code

� Divide received word by generator polynomial

�� if divisible → No error

� if indivisible → Error detected

�An erroneous bit position is judged by the form of the

remainder

�The error is corrected by reversing (0→1,1→0) the bit of

the erroneous bit position

76

the erroneous bit position

�More than or equal to two bits error

→ misscorrection or undetection

Possible to correct single errorPossible to correct single error



Example of Hamming codeExample of Hamming code

�Single error correction code (dminminminmin=3)

�Hamming (7,4)

Generator bit: 「1011」 x3+x+1

�Hamming (15,11)

Generator bit: 「10011」 x4+x+1

dmin=3

Generator

polynomial

77

�Hamming (31,26)

Generator bit: 「100101」 x5+ x2 +1



EEbb/N/Noo vs BER (Hamming code)vs BER (Hamming code)

10 -1

10 0

-5

10 -4

10 -3

10 -2

10

Bit e

rror ra

te

No coding

H(7,4)

H(63,57)

78

0 1 2 3 4 5 6 7 8 9 10 10

-6

10 -5

Eb/No (dB)

(7,4) (63,57)

C/N(dB)

2.4dB

0.4dB



BER before BER before vsvs after error correctionafter error correction

(Hamming code)(Hamming code)

-1

100

Bit e

rror ra

te a

fter

err

or co

rrec

tion

10-5

10-4

10-3

10-2

10-1

Bit e

rror ra

te a

fter

err

or co

rrec

tion

(7,4)

(63,57)

79

10-3

10-2

10-1

10-6

10-5

Bit error rate before error correction

Bit e

rror ra

te a

fter

err

or co

rrec

tion



BCH codeBCH code

(Bose Chaudhuri (Bose Chaudhuri

80

(Bose Chaudhuri (Bose Chaudhuri

Hochquenghem )Hochquenghem )



BCH codeBCH code

�Capable of more than two or equal bits error correction code

�The degree of freedom in encoding is high

ik-1 ・・

Information bit length: k Parity bit length: m

Code length: n

・・ i1 i0 pm-1 ・ p0p1

81

�Code length n≦ 2r-１（bit）

�Information bit length k = n - m （bit）�Parity bit length m≦rt （bit）�Minimum distance dmin≧2t + 1 （bit）

r: Integer of more than equal three

t: Number of error correction capability



「BCH（15, 7）codeword」 (r=4,t=2)

Code length: n = 15, Information bit length: k = 7, Parity bit

length: m = 8

BCH code (15,7)BCH code (15,7)

length: m = 8

Generator polynomial：g(x)=g1(x)g3(x)=x8+x7+x6+x4+1

i3 i2 i1 i0

Information bits

Codeword

i6 i5 i4

82

i6i5i4i3i2i1io00000000

111010001Remainder: R＝p7p6p5p4p3p2p1p0

Codeword

p5 p4 p3 p2 p1 p0p6p7i3 i2 i1 i0i6 i5 i4



BCH code (15,7)BCH code (15,7)

Bit expression of Generator polynomial:111010001

Generator polynomial: g(x)=x8+x7+x6+x4+1

=(x4+x+1)(x4+x3+x2+x+1)=(x4+x+1)(x4+x3+x2+x+1)

Where let g1(x), g3(x) be as follows;

g1(x)= x4+x+1

g3(x)=x4+x3+x2+x+1

g(x) becomes as follows;

g(x)=g1(x)g3(x)

83

1 3

「If codeword polynomial is divisible by generator polynomial

g(x) 」 means

「it is also divisible by polynomial g1(x) or g3(x) 」



Single bit error in BCH code (15,7)Single bit error in BCH code (15,7)

Position of Remainder of


Remainder of

i6i5i4i3i2i1i0p

Position of

error

10011101111111100111101001011011

11111000010000100001111110000100

Remainder of

g1(x)

g1(x)=x4+x+1

「10011」

Bit error position

(single bit error)

Remainder of

g3(x)

84

0p7p6p5p4p3p2p1p0

10111100011000111000010000100001

01000010000111111000010000100001

g3(x)=x4+x3+x2+x+1

「11111」

Remainder divided by

g1(x), g3(x)



2 bits error in BCH code (15,7)2 bits error in BCH code (15,7)

010001100111

R of g1(x) R of g3(x)

i6, i5i6, i4i6, i3

Position

011110111101

i4, p3i4, p3i4, p1

011110111101

110000000110

i0, p6i0, p5i0, p4

100100110110

101010010111

R of g1(x) R of g3(x)Position R of g1(x) R of g3(x)Position


011111100011110000100101111110100001110110111000001000111010011110000110

i6, i3i6, i2i6, i1i6, i0i6, p7i6, p6i6, p5i6, p4i6, p3i6, p2i6, p1i6, p0i5, i4i5, i3i5, i2i5, i1i5, i0i , p

110111100000011110111100111000000111101111011110110010101001011100001100

i4, p1i4, p0i3, i2i3, i1i3, i0i3, p7i3, p6i3, p5i3, p4i3, p3i3, p2i3, p1i3, p0i2, i1i2, i0i2, p7i2, p6i , p

110111101001010010110101001010001101011010101100111111010010110010110001

011001011011110110100101000000111101101001100000001111101001010100110000

i0, p4i0, p3i0, p2i0, p1i0, p0p7, p6p7, p5p7, p4p7, p3p7, p2p7, p1p7, p0p6, p5p6, p4p6, p3p6, p2p6, p1p , p

011011010001011101000111110110000011111110011010101011110100100011101101

011100001100101010010110010110111100000001100101001111011010011000000011

85

0110000110111110010110011111110000011000010110100100001110011100

5 0i5, p7i5, p6i5, p5i5, p4i5, p3i5, p2i5, p1i5, p0i4, i3i4, i2i4, i1i4, i0i4, p7i4, p6i4, p5i4, p4

1100101010010111000011001010100101100101101111000000011001011011

2 6i2, p5i2, p4i2, p3i2, p2i2, p1i2, p0i1, i0i1, p7i1, p6i1, p5i1, p4i1, p3i1, p2i1, p1i1, p0i0, p7

0001010011110011010101101111000101101100100100101110100010111110

0000111010010001010100000111101111011110000001111011110111101100

6 1p6, p0p5, p4p5, p3p5, p2p5, p1p5, p0p4, p3p4, p2p4, p1p4, p0p3, p2p3, p1p3, p0p2, p1p2, p0p1, p0

1101010111100010010001111011011100010010110010101001011001010011

0011111010010101001100000111101111011110110010101001011001010011



Remainder and bit error position in BCH (15,7)Remainder and bit error position in BCH (15,7)

Remainder of g3(x)


i5,i0i4,p7

i3,p1

i3,p6

i6,p4

i2,p0

p6,p1

i2,p5

i1,p4

i5,p3

p3

i0

p2

p3,p2

i5,i4i4,i0i5,p7

i0,p7

i0,p2

i5,p2

p6

p1

i3

p6,p3

p3,p1

i3,p3

i5,p6

i0,p6

p6,p2

i3,p2

p2,p1

i4,i3p7,p1

i3,p7

i6,p0

i2,p4

i1,p5

p4,p0

i6,i2

p5,p4

p5

p0

i0,p0

i5,p0

i2,i0i5,i2

p5,p3

p3,p0

i4,i2

i2,p7

p5,p2

p7,p0

i4,p0

i4,p5

p2,p0

i１,p１i３,i１

i1,p6

p4,p1

i6,p6

i3,p5

p5,p1

p6,p5

i3,i2i2,p1

p7,p4

i4,p4

i6,p7

i6,i4i1,p2

i1,p7

i4,i1

i6,i5i6,i0i1,p3

i0,p4

i5,p4

i6,p3

i1

i6

0000

1000

0100

1100

0010

1010

0110

1110

0001

1001

0101

0000 00100100 11001000 1010 0110 1110 0001 1001 0101 1101 0011 1011 0111 1111

Remai

nder of

g1(x)

86

i5,p3

i0,p3

i6,i1i4,p2

p5,p0

p7,p2

i0i5

p7

i4

p7,p3

i4,p3

i5,i3i3,i0i0,p1

i5,p1

i3,p7

i4,p1

i4,p6

p7,p6

p5,p4

p2,i1

i1,p0

i6,p5

i2

i0,p5

i5,p5

i2,p3

p2,p0

p7,p5

i2,p2

i6,p6

i3,p4

i6,p1

i6,i3p6,p4

i2,p1

p6,p0

p1,p0

i2,p6

i3,p0

i4,i1i6,p2

p4,p2

p4,p3

i5,i1i1,i0

p4

0101

1101

0011

1011

0111

1111

Ex. single bit error: i3 ←→ remainder of g1(x) :1110, remainder of g3(x): 0010

Ex. 2 bits error :i5, i3←→ remainder of g1(x) :0011, remainder of g3(x): 1010



Example of BCH codeExample of BCH code

�t bit error correction capability (dmin≧≧≧≧2t+1)

�BCH (7,4) :t=1�BCH (7,4) :t=1

Generator bit :「1011」g(x)=x3+x+1

�BCH (15,7) :t=2

Generator bit：「111010001」g(x)=x8+x7+x6+x4+1

dmin=3

dmin=5

87

g(x)=x +x +x +x +1

�BCH (31,16) : t=3

Generator bit：「1000111110101111」g(x)=x15+x11+x10+x9+x8+x7+x5+x3+x2+x+1

dmin=7



ReedReed--Solomon codeSolomon code

88



ReedReed--Solomon codeSolomon code

�Symbol error correction code (in general byte)

�The degree of freedom in encoding is high

Code length: n

ik-1 ・・

Information symbol length: k

�Code length n≦ 2r - 1 （Symbol）

Parity symbol length: m

Code length: n

・・ i1 i0 pm-1 ・ p0p1

89

�Code length n≦ 2r - 1 （Symbol）�Information symbol length k = n - m （Symbol）�Parity symbol length m = 2t （Symbol）�Minimum distance dmin≧2t+1 （Symbol）

r：Number of bits comprising symbol

t： Number of symbol capable of error correction



Feature of roots of generator polynomialFeature of roots of generator polynomial

Let α be a primitive element in GF(24) which is constructed based

on the primitive polynomial ; g(x)

where g(x)=x3+x+1where g(x)=x +x+1

α satisfies the following equation

α3+α+1=0 α3=1+α

α4=α(1+α)==α+α

2

α5=α(α+α

2)==α

2+α

3

　 =1+α+α2

α6=α(1+α+α

2)

Power formPolynomial

form

3-Tuple

formSymbol

0 0 0 0 0 0

α0 =1 1 1 0 0 4

Field of 8 elements generated by x3+x+1

90

α =α(1+α+α )

=α+α2+α

3

　=α+α2+(1+α)

=1+α2

α7=α(1+α

2)=α+α

3

　 =α+(1+α)=1

α1 =α α 0 1 0 2

α2

α2 0 0 1 1

α3 1 +α 1 1 0 6

α4

α +α2 0 1 1 3

α5

1 +α +α2 1 1 1 7

α6

1 + α2 1 0 1 5

α7=α

0 1 1 0 0 1



Addition of roots for generator polynomialAddition of roots for generator polynomial

000 100 010 001 110 011 111 101

3-Tuple Power 0 1 α α2

α3

α4

α5

α6

000 0 0 1 α α2

α3

α4

α5

α6

100 1 1 0 α3

α6

α α5

α4

α2

Vector expression

100 1 1 0 α α α α α α

010 α α α3

0 α4

1 α2

α6

α5

001 α2

α2

α6

α4

0 α5

α α3

1

110 α3

α3

α 1 α5

0 α6

α2

α4

011 α4

α4

α5

α2

α α6

0 1 α3

111 α5

α5

α4

α6

α3

α2

1 0 α

101 α6

α6

α2

α5

1 α4

α3

α 0

000 100 010 001 110 011 111 101


α3

α4

α5

α6

000 0 100 001 110 011

Vector expression

91

000 0 000 100 010 001 110 011 111 101

100 1 100 000 110 101 010 111 011 001

010 α 010 110 000 011 100 001 101 111

001 α2

001 101 011 000 111 010 110 100

110 α3

110 010 100 111 000 101 001 011

011 α4

011 111 001 010 101 000 100 110

111 α5 111 011 101 110 001 100 000 010

101 α6 101 001 111 100 011 110 010 000



Multiplication of roots for generator polynomialMultiplication of roots for generator polynomial

000 100 010 001 110 011 111 101


α3

α4

α5

α6

000 0 0 0 0 0 0 0 0 0

100 1 0 1 α α2

α3

α4

α5

α6

2 3 4 5 6

Vector expression

010 α 0 α α2

α3

α4

α5

α6 1

001 α2 0 α

2α

3α

4α

5α

6 1 α

110 α3 0 α

3α

4α

5α

6 1 α α2

011 α4 0 α

4α

5α

6 1 α α2

α3

111 α5 0 α

5α

6 1 α α2

α3

α4

101 α6 0 α

6 1 α α2

α3

α4

α5

000 100 010 001 110 011 111 101


α3

α4

α5

α6

Vector expression

92

000 0 000 000 000 000 000 000 000 000

100 1 000 100 010 001 110 011 111 101

010 α 000 010 001 110 011 111 101 100

001 α2 000 001 110 011 111 101 100 010

110 α3 000 110 011 111 101 100 010 001

011 α4 000 011 111 101 100 010 001 110

111 α5 000 111 101 100 010 001 110 011

101 α6 000 101 100 010 001 110 011 111



RS code (5,3)RS code (5,3)

「RS code(5,3)」(r=3, 1symbol=3 bits)

Code length: n=5, information symbol length: k=3, Parity bit

symbol length: m=2symbol length: m=2

A B C

Information bits

α: a root of g(x)=x3+x+1=0

A+B+C+P+Q=0

Codeword

A B C P Q

93

A+B+C+P+Q=0

αA+α2B+α3C+α4P+α5Q=0

From above equations, P and Q are as follows:

PP= α6A+α3B+α2C

QQ==α2A+αB+α6C

Parity check equation



Characteristics of error in RS code (5,3)Characteristics of error in RS code (5,3)Set syndrome S as follows;

S1=A+B+C+P+Q

S2=αA+α2B+α3C+α4P+α5Q Codeword

A B C P QNo error

S1=0 and S2=0

�One symbol error�If symbol A is error, and it becomes A+A’

S1=(A+A’)+B+C+P+Q=A’

A B C P Q

94

S1=(A+A’)+B+C+P+Q=A’

S2=α(A+A’)+α2B+α3C+α4P+α5Q=αA’

∴S2＝＝＝＝αS1

�If symbol B is error, and it becomes B+B’

S1=A+(B+B’)+C+P+Q=B’

S2=αA+α2(B+B’)+α3C+α4P+α5Q= α2B’

∴S2＝＝＝＝α2S1



Characteristics of error in RS code (5,3)Characteristics of error in RS code (5,3)

�One symbol error�If symbol C is error, and it becomes C+C’

S1=A+B+(C+C’)+P+Q=C’1

S2=αA+α2B+α3(C+C’)+α4P+α5Q=α3C’

∴S2＝＝＝＝α3S1

Symbol A → S2＝＝＝＝αS1

Symbol B → S2＝＝＝＝α2S1

Symbol C → S2＝＝＝＝α3S1

95

Possible to correct one symbolPossible to correct one symbol

Detection of error location → from above equations

and S1is error pattern

Error correction → Add S1 to the error detected symbol



Example of RS code (5,3) encodingExample of RS code (5,3) encoding

Let A,B,C set as follows; (1 symbol is 3 bits)

A=100, B=010, C=001

A=1, B=α, C=α2A=1, B=α, C=α2

P=α6A+α3B+α2C

=α6・1+α3・α+α2・α2

=α6+α4+α4

=α6

=101

Q=α2A+αB+α6C

=α2・1+α・α+α6・α2

=α2+α2+α8

=α8

=α

96

=101 =α

=010

100 101 010010 001

A B C P Q



Error correction example of RS code (5,3)Error correction example of RS code (5,3)

If symbol A is error

A=100 → 011 (3 bits error)

S1=A+B+C+P+Q

＝α4＋α＋α2＋α6＋α

＝α2＋α4＋α6

＝α＋α6

＝ α5

＝111

S2＝αA＋α2B＋α3C＋α4P＋α5Q

＝α・α4＋α2・α＋α3・α2＋α4・α6＋α5・α

＝α5＋α3＋α5＋α10＋α5

＝α3＋α3＋α6

＝α6

＝αS１

97

S2 =αS1, ;error location is symbol A

A= 011+S1=011+111=100



Examples of RS codeExamples of RS code

� t symbol error correction code (dmin=2t+1)

�RS(32,30) code, etc. :t=1

�RS(64,60) code, etc. :t=2

�RS(128,120) code, etc. :t=4

Suitable for burst error correctionSuitable for burst error correction

98

�RS(128,120) code, etc. :t=4

�RS(204,188) code, etc. :t=8

�RS(207,187) code, etc. :t=10



ConvolutionalConvolutional codecode

99



Trellis diagram of Trellis diagram of convolutionalconvolutional codecode

:1bit delay

＋：Adder

a a

c1＋

1101011

10

00 10/0

01/011/0

00/1a

a2a1 a3

Output c1c2

a1 a3

c2＋

Input

01

1100

11/1

00/0 10/1

01/1

10/0 00/1

State A:(0 0)

State (a3 a2)

00/0

00/1

c1c2/a1

c1=a1+a2 +a3

c2=a1 +a3

a2

100

State B:(0 1)

State C:(1 0)

State D:(1 1)

00/1

11/0

11/1

10/0

10/1

01/0

01/1133



Principal of Principal of ViterbiViterbi decodingdecoding

�Examine cheaper route, whenever the railroad joins

�Select the route which a fare is cheaper

�Examine the cheaper route one after another, finally

Kasumigaseki

�Examine the cheaper route one after another, finally

find the cheapest route

Shinjyuku

Harajyuku

Uehara Simokitazawa Seijyo160

160

150/310

150/310

160/470

130/440 180/620

Kasumigaseki Shinjyuku Seijyou

101

Omotesandou

Shibuya

160

160/320

150/310

160/320 120/430210/520

Shibuya



Error correction by Viterbi decoding Error correction by Viterbi decoding

0011 01 0001

0 1

10 01

10 11 11

00 11 00

1 10

0001 11

Received message

Transmitted message

Information message 0 0 0

State A:(0 0)

State B:(0 1)

State C:(1 0)

102 (3)1 (1)

1 (1) 0 (1)

1 (2)

1 (2)

(5)

(2)

(3)

(4)(2)

(3)

(2)

102

State D:(1 1)

00/0

00/1

11/0

11/1

10/0

10/1

01/0

01/1

c1c2/a1

1 (2) (2)

(3)

Path metricBlanch metric

1 (2)

145



Key pointKey point

�Block code�Block code

�Hamming code

�Single error correction

�BCH code

�t bit error correction

�Reed solomon code

�t symbol error correction

103

�t symbol error correction

�Convolutional coding +Viterbi decode



Application in Application in

broadcasting fieldbroadcasting field

104

broadcasting fieldbroadcasting field



Applications of error correcting codeApplications of error correcting code

Device Type of error Feature Ex. 0f Code

Digital Random & Convolutional

Usage

TransmissionDigital

broadcastingReceiver

Random &

BurstLow error rate

Convolutional

+ RS

Semiconductor

memoryRandom

Hight speed

& low error rateHamming

Magnetic memory,

Optical memory

(HDD,Tape,CD-ROM)

(Random)

BurstLow error rate

CRC

RS product

Video

Audio

Magnetic memory,

Optical memory

(VTR,CD,DVD)

(Random)

Burst

Relatively high

error rate than RS product

Conputor

Data

Record

105

RS: Reed Solomon

Audio (VTR,CD,DVD) Bursterror rate than

data

RS product



Applications for broadcasting VTR Applications for broadcasting VTR

VTR Width of tape

System (inch) Inner code Outer code

Error correction codeVideo

System (inch) Inner code Outer code

D1 SD Component 3/4 RS (64,60) RS (32,30)

D2 SD Composite 3/4 RS (93,85) RS (68,64)

D3 SD Composite 1/2 RS (95,87) RS (136,128)

D5 SD,HD Component 1/2 RS (95,87) RS (128,120)

106

β CAM SD Component 1/2 RS (178,164) RS (106,96)

D6 HD Component 3/4 RS (227,211) RS (254,240)

HD1 HD Component 1 RS (110,104) RS (65,60)



Applications for broadcastingApplications for broadcasting

Inner code Outer code

Error correction codeBroadcasting

Inner code Outer code

Digital Terrestrial Television Broadcasting Convolutional code

DVB-T, ISDB-T (r=1/2,2/3,3/4,5/6,7/8)

Trellis code (r=2/3)

Convolutional code(1/2)

Digital Satellite Television Broadcasting Trellis code (r=2/3)

Convolutional code

RS (204,188)

RS (204,188)

RS (207,187) ATSC

107

Convolutional code

(r=1/2,2/3,3/4,5/6,7/8)

Convolutional code

(r=1/2,2/3,3/4,5/6,7/8)

RS (204,188)

RS (204,188) Digital Communication Satellite

Television Broadcasting

ISDB-S



Application of error correction techniqueApplication of error correction technique

�Magnetic tape (RS code)

�Magnetic disk (RS code)

�Compact disk (RS code)�Compact disk (RS code)

�Digital audio tape (RS code)

�Mobile phone (RS & Convolutional coding)

�Satellite communication (RS & Convolutional coding)

�Satellite broadcasting (RS & Convolutional coding)

Teletext broadcasting (Cyclic code )

108

�Teletext broadcasting (Cyclic code )

�Digital broadcasting (RS & Convolutional coding)

�Intercontinental optical communications (RS code)



FinallyFinally

�The error correction is indispensable

for digital technique.

�Importance of the error correction

technology is increasing further because

109

technology is increasing further because

of high compression of source coding.



Informative referenceInformative reference

110



ReedReed--Solomon generatorSolomon generator

Gate

Connect for first (k) Bytes

Open for last (N-k) Bytes

X1 X2 X3X19 X20

α=174 α=165 α=121 α=121 α=181 α=152

N-k=20 Parity Bytes

BN=207 Encoded Data

Mod(256) add two field elements (Bytes)

Mod(256) multiply a field element with a fixed element α

111

A

N=207 Encoded Data

Connect A for first (k) Bytes

Open B for last (N-k) Bytes

Source one element (Byte)X

Primitive Field Generator Polynomial (Galois):P(X)

P(X)=X8+X4+X3+X2+1

k=187 Data Bytes

Code Generator Polynomial: G(X)

G(X)=X20+152X19+185X18+240X17+5X16+111X15+99X14+6X13+220X12+112X11+150X10+69X9+36X8+187X7+22X6

+228X5+198X4+121X3+121X2+165X1+174



Punctured codePunctured code

Convolutional code r=1/1Convolutional

coderB0

X

PuncturedP1

P

r=6/8=3/4

P1=X1,Y2,Y3

P0=Y1,X3,Y4

X:X1,X2,X3,X4

Y:Y1,Y2,Y3,Y4

Improvement of coding rate

+

X171octal

B0 PuncturedY

P0

112

r=4/6=2/3

P1=X1,Y2

P0=Y1,X3

Y:Y1,Y2,Y3,Y4

X:X1,X2,X3

Y:Y1,Y2,Y3

D ● D ● D ● D D ● D ●●

+

Y133octal

B0

D : 1bit delay + :Exclusive OR



Linear code and cyclic code Linear code and cyclic code

Block code

Linear code

None linear code

Cyclic code

None cyclic code

Linear code

If C1 and C2 is two arbitrary codewords, added (C1+C2 ) is also codeword

Cyclic code

If message (n-bits in length) is expressed by polynomial, cyclic code is the all

None linear code None cyclic code

113

If message (n-bits in length) is expressed by polynomial, cyclic code is the all

codewords (code polynomial) that are divisible by generator polynomial G(X)

If Cn=(x1,x2,x3,・・,xN） is codeword,

each element shifted Cm=(x2,x3,・・ xN,x1）is also codeword



ExampleExample

Linear code

0001 011 (C1)Hamming code (7,4)

0000 000

0001 011

0010 110

0011 101

0100 111

0101 100

1000 101

1001 110

1010 011

1011 000

1100 010

1101 001

0001 011 (C1)

+ 0010 110 (C2)

0011 101 (C1+C2=C3)

Cyclic code

114

0101 100

0110 001

0111 010

1101 001

1110 100

1111 111 0111 010 (C7)

1bit shift to left

1110 100 (C7→C14)



Coding rateCoding rate

Number of bits for information (k)Number of bits for information (k)Coding rate =Coding rate =

Number of bits for codeword (n)Number of bits for codeword (n)

Block code : (n,k)

coding rate r=k/n

Example : (6,4) code

Cap

ability o

f er

ror

corr

ection

hig

hlo

w

Code B

115

Example : (6,4) code

coding rate: 2/3

Coding rateC

apab

ility o

f er

ror

corr

ection

high low

hig

h

Code A1011 01



Appendix Appendix

Concatenated codeConcatenated code

116

Concatenated codeConcatenated code



Encoder

MPEG-2 Source Coding &

Multiplexing

1

nEncoder

Mux

Adaptation

Energy

Dispersal

Outer

coder

Outer

interleaver

Inner

coder

Example of concatenated code Example of concatenated code

n

Inner

interleaverMapper

Frame

AdaptationOFDM

Guard

Interval D/A Front

End

To Antenna

Encoder

Encoder

Encoder

1

n

Dispersal

Mux

Adaptation

Energy

Dispersal

Outer

coder

Outer

interleaver

Inner

coder

117

interleaverMapper

AdaptationOFDM Interval

Insertion LPFEnd

Pilots &

TPS

SignalsTerrestrial Channel Adapter

DVB-T functional block diagram



Concept of interleaveConcept of interleave

118



InterleavingInterleaving

�Interleaving is a way to rearrange data in order to increase performance to cope with burst error.

Direction of write

Direc

tion o

f rea

d

119

Direc

tion o

f rea

d

Conceptional diagram of interleaving



Example of concatenated code (Turbo code)Example of concatenated code (Turbo code)

Additive

White

Gaussian

Noise

Channel

(AWGN)RSC

DEC1 Interleaver

RSC

DEC2

Deinterleaver

Deinterleaver

Interleaver

120

Turbo decoderTurbo encoder

RSC: Recursive systematic convolutional encoder

DEC: Soft-in / soft-out decoder



Example of concatenated code (Example of concatenated code (Product code)Product code)

n1

The random & burst error is effectively corrected

Information

n1

n2k2

k1

C1:(n１,k1) code

Par

ity o

f C

1121

Parity of C2

C2(n2,k2) code

Par

ity o

f C



Example of product code by RS codeExample of product code by RS code

B15 P11 P12 (1) Can not correct

Error

B14B11 B13B12 B15 P11

B22 B13 B24 B25 P21

B31 B32 B33 B34 B35 P31

B41 B42 B43 B44 B45 P41

P12

P22

P32

P42

Q11 Q12 Q13 Q14 Q15 P51 P52

BiiInformation

symbol

QnjPik Parity symbol

(1) Can not correct

(2) B21 correct

B14B11

B21

B13B12

122

Q11 Q12 Q13 Q14 Q15 P51 P52

(6) B14 correct

QnjPik Parity symbol

Q21 Q22 Q23 Q24 Q25 P61 P62

(3) B11 correct

(4) B12 correct

(5) B13 correct



Appendix Appendix

Cyclic coderCyclic coder

123

Cyclic coderCyclic coder



Cyclic code by shift register (1)Cyclic code by shift register (1)

0 0 0x6 +x5 +x3 +x

1 1 0 1 0 1 0

0 0Output: q(x)

x6+x5 +x3 +x

x6 +x4+x3

x5+x4

+x3+x2+x+1

x +x +x +x

x3 +x +1

x3+x+1

0 0

(x6+x5+x3+x)+x+1= x6+x5+x3+1

Cyclic code:

Input: f(x)

Output: q(x)

124

x5+x4

x5 +x3+x2

x4+x3+x2+x

x4 +x2+x

x+1

x3

x3 +x+1

(x +x +x +x)+x+1= x +x +x +1

Remainder: R(x)



0 0 1x5 +x3 +x

1 0 1 0 1 0

0 0


x +x +x

x3 +x +1

0 0

125



0 1 1+x3 +x

0 1 0 1 0

0 0


+x +x

x3 +x +1

0 0

126



1 1 0+x3 +x

1 0 1 0

1 1


+x +x

x3 +x +1

1 1

127



1 1 0+x

0 1 0

1 1

1


x6+x5 +x3 +x

x6 +x4+x3

x5+x4

x3

1101010

+x

1011

1100

x3 +x +1

x3+x+11011

1 1

1

128

x5+x41100



1 1 1+x

1 0

1 1

11


x6+x5 +x3 +x

x6 +x4+x3

x5+x4

+x3+x2

1101010

+x

1011

1100

x3 +x +1

x3+x+11011

1 1

11

129

x5+x4

x5 +x3+x2

x4+x3+x2+x

1100

1111

1011



1 0 00

1 1

111


x6+x5 +x3 +x

x6 +x4+x3

x5+x4

+x3+x2+x

1101010

1011

1100

x3 +x +1

x3+x+11011

1 1

111

130

x5+x4

x5 +x3+x2

x4+x3+x2+x

x4 +x2+x

x3

1100

1111

1000

1011

1011



0 1 11111


Output: q(x)

x6+x5 +x3 +x

x6 +x4+x3

x5+x4

+x3+x2+x+1

1101010

1011

1100

x3 +x +1

x3+x+11011

1111

Output: q(x)

Output: q(x)

Remainder (contents of resisters) : R(x)

131

x5+x4

x5 +x3+x2

x4+x3+x2+x

x4 +x2+x

x+1

x3

x3 +x+1

1100

1111

011

1000

1011

1011

1011

Remainder: R(x)



ConvolutionalConvolutional

coder & decodercoder & decoder

132

coder & decodercoder & decoder


:1bit delay

＋：Adder

a a

c1＋

1101011

10

00 10/0

01/011/0

00/1

Trellis diagramTrellis diagram

a

a2a1 a3

Output c1c2

a1 a3

c2＋

Input

01

1100

11/1

00/0 10/1

01/1

10/0 00/1

State A:(0 0)

State (a3 a2)

00/0

00/1

c1c2/a1

c1=a1+a2 +a3

c2=a1 +a3

a2

133

State B:(0 1)

State C:(1 0)

State D:(1 1)

00/1

11/0

11/1

10/0

10/1

01/0

01/1


:1bit delay

＋：Adder

0 0

c1＋

11010

10

00/0 10/110/0

01/011/0

00/1

Trellis diagram (1)Trellis diagram (1)

a2a1 a3

Output c1c2

0 0

c2＋

Input

01

1100

11/1

00/0 10/1

01/1

10/0 00/1

State A:(0 0)

State (a3 a2)

00/0

00/1

c1c2/a1

c1=a1+a2 +a3

c2=a1 +a3

State B:(0 1)

State C:(1 0)

State D:(1 1)

00/1

11/0

11/1

10/0

10/1

01/0

01/1

134



:1bit delay

＋：Adder

0 0 0

0＋

1101

10

00/0 10/110/0

01/011/0

00/1

a1 a2 a3

Output c1c2

0 0 0

0＋

Input

01

1100

11/1

00/0 10/1

01/1

10/0 00/1

State A:(0 0)

State (a3 a2)

00/0

00/1

c1c2/a1

0

0

00

c1=a1+a2 +a3

c2=a1 +a30

State B:(0 1)

State C:(1 0)

State D:(1 1)

00/1

11/0

11/1

10/0

10/1

01/0

01/1

135


:1bit delay

＋：Adder

1 0 0

＋

110

10

00/0 10/110/0

01/011/0

00/1


c1

a1 a2 a3

Output c1c2

1 0 0

＋

Input

01

1100

11/1

00/0 10/1

01/1

10/0 00/1

State A:(0 0)

State (a3 a2)

00/0

00/1

c1c2/a100

c1=a1+a2 +a3

c2=a1 +a30

c2

State B:(0 1)

State C:(1 0)

State D:(1 1)

00/1

11/0

11/1

10/0

10/1

01/0

01/1

136


:1bit delay

＋：Adder

1 0 0

1＋

11011

10

00 10/0

01/011/0

00/1


a1 a2 a3

Output c1c2

1 0 0

1＋

Input

01

1100

11/1

00/0 10/1

01/1

10/0 00/1

State A:(0 0)

State (a3 a2)

00/0

00/1

c1c2/a1

1

1

00 11

c1=a1+a2 +a3

c2=a1 +a3

0 1

137

State B:(0 1)

State C:(1 0)

State D:(1 1)

00/1

11/0

11/1

10/0

10/1

01/0

01/1


:1bit delay

＋：Adder

0 1 0

＋

1111

10

00 10/0

01/011/0

00/1


c1

a1 a2 a3

Output c1c2

0 1 0

＋

Input

01

1100

11/1

00/0 10/1

01/1

10/0 00/1

State A:(0 0)

State (a3 a2)

00/0

00/1

c1c2/a100 11

c1=a1+a2 +a3

c2=a1 +a3

0 1

c2

138

State B:(0 1)

State C:(1 0)

State D:(1 1)

00/1

11/0

11/1

10/0

10/1

01/0

01/1


:1bit delay

＋：Adder

0 1 0

1＋

1111

10

00 10/0

01/011/0

00/1


a1 a2 a3

Output c1c2

0 1 0

0＋

Input

01

1100

11/1

00/0 10/1

01/1

10/0 00/1

State A:(0 0)

State (a3 a2)

00/0

00/1

c1c2/a100 11 10

c1=a1+a2 +a3

c2=a1 +a3

1

0

0 1 0

139

State B:(0 1)

State C:(1 0)

State D:(1 1)

00/1

11/0

11/1

10/0

10/1

01/0

01/1


:1bit delay

＋：Adder

1 0 1

＋

111

10

00 10/0

01/011/0

00/1


c1

a1 a2 a3

Output c1c2

1 0 1

＋

Input

01

1100

11/1

00/0 10/1

01/1

10/0 00/1

State A:(0 0)

State (a3 a2)

00/0

00/1

c1c2/a100 11 10

c1=a1+a2 +a3

c2=a1 +a3

0 1 0

c2

140

State B:(0 1)

State C:(1 0)

State D:(1 1)

00/1

11/0

11/1

10/0

10/1

01/0

01/1


:1bit delay

＋：Adder

1 0 1

0＋

111

10

00 10/0

01/011/0

00/1


a1 a2 a3

Output c1c2

1 0 1

0＋

Input

01

1100

11/1

00/0 10/1

01/1

10/0 00/1

State A:(0 0)

State (a3 a2)

00/0

00/1

c1c2/a1

0

0

00 11 01

c1=a1+a2 +a3

c2=a1 +a3

000 1 0 1

141

State B:(0 1)

State C:(1 0)

State D:(1 1)

00/1

11/0

11/1

10/0

10/1

01/0

01/1


:1bit delay

＋：Adder

1 1 0

＋

11

10

00 10/0

01/011/0

00/1


c1

a1 a2 a3

Output c1c2

1 1 0

＋

Input

01

1100

11/1

00/0 10/1

01/1

10/0 00/1

State A:(0 0)

State (a3 a2)

00/0

00/1

c1c2/a100 11 01

c1=a1+a2 +a3

c2=a1 +a3

000 1 0 1

c2

142

State B:(0 1)

State C:(1 0)

State D:(1 1)

00/1

11/0

11/1

10/0

10/1

01/0

01/1


:1bit delay

＋：Adder

1 1 0

0＋

11

10

00 10/0

01/011/0

00/1


a1 a2 a3

Output c1c2

1 1 0

1＋

Input

01

1100

11/1

00/0 10/1

01/1

10/0 00/1

State A:(0 0)

State (a3 a2)

00/0

00/1

c1c2/a1

0

1

00 11 01

c1=a1+a2 +a3

c2=a1 +a3

00 010 1 0 1 1

143

State B:(0 1)

State C:(1 0)

State D:(1 1)

00/1

11/0

11/1

10/0

10/1

01/0

01/1


:1bit delay

＋：Adder

1 1

＋

11

10

00 10/0

01/011/0

00/1


c1

a1 a2 a3

Output c1c2

1 1

＋

Input

01

1100

11/1

00/0 10/1

01/1

10/0 00/1

State A:(0 0)

State (a3 a2)

00/0

00/1

c1c2/a100 11 01

c1=a1+a2 +a3

c2=a1 +a3

00 010 1 0 1 1

c2

144

State B:(0 1)

State C:(1 0)

State D:(1 1)

00/1

11/0

11/1

10/0

10/1

01/0

01/1


Error correction by Viterbi decodingError correction by Viterbi decoding

0011 01 0001

0 1

10 01

10 11 11

00 11 00

1 10

0001 11

Received message

Transmitted message


State A:(0 0)

State B:(0 1)

State C:(1 0)

2 (3)1 (1)

1

(1) 0 (1)

1 (2)

(5)

(2)

(3)

(4)

(2)

(3)

(2)

0011 01 000110 11 11Received message

145

State D:(1 1)

00/0

00/1

11/0

11/1

10/0

10/1

01/0

01/1

c1c2/a1

1 (2)(2)

(3)

Path metricBlanch metric

1 (2)



0011 01 0001

0 1

10 01

10 11 11

00 11 00

1 10

0001 11

0 0 0

Received message

Transmitted message

Information message

State A:(0 0)

State B:(0 1)

State C:(1 0)

(3)(1)

(1) (1)

(2)

(2)

(2)

(3)

(2)

(2)

(4)

(2)

(4)

(4)

(2)

(3)

0011 01 000110 11 11Received message

146

State D:(1 1)

00/0

00/1

11/0

11/1

10/0

10/1

01/0

01/1

c1c2/a1

(2)(2) (4)

(3)


0011 01 0001

0 1

10 01

10 11 11


00 11 00

1 10

0001 11

Received message

Transmitted message


State A:(0 0)

State B:(0 1)

State C:(1 0)

0011 01 000110 11 11(1)

(1) (1)

(2)

(2)

(2)

(2)

(2)

(2)

(3)

(3)

(4)

(4)

(3)

(4)

(2)

(3)

(3)

(3)

Received message

147

State D:(1 1)

00/0

00/1

11/0

11/1

10/0

10/1

01/0

01/1

c1c2/a1

(2) (3)

(3)

(2)

(5)


0011 01 0001

0 1

10 01

10 11 11


00 11 00

1 10

0001 11

0 0 0

Received message

Transmitted message

Information message

State A:(0 0)

State B:(0 1)

State C:(1 0)

0011 01 000110 11 11(1)

(1) (1)

(2)

(2)

(2)

(2)

(2)

(2)

(3)

(3)

(3)

(4)

(4)

(4)

(4)

(5)

(2)(3)

Received message

148

State D:(1 1)

00/0

00/1

11/0

11/1

10/0

10/1

01/0

01/1

c1c2/a1

(2)

(3)

(2) (3)

(4)


0011 01 0001

0 1

10 01

10 11 11


00 11 00

1 10

0001 11

Received message

Transmitted message


State A:(0 0)

State B:(0 1)

State C:(1 0)

0011 01 000110 11 11(1)

(1) (1)

(2)

(2)

(2)

(2)

(2)

(2)

(3)

(3)

(4)

(4)

(2)

(6)

(2)

(4)

(4)

(5)

(4)

Received message

149

State D:(1 1)

00/0

00/1

11/0

11/1

10/0

10/1

01/0

01/1

c1c2/a1

(2)

(3)

(2) (3) (5)

(4)


0011 01 0001

0 1

10 01

10 11 11


00 11 00

1 10

0001 11

Received message

Transmitted message


State A:(0 0)

State B:(0 1)

State C:(1 0)

0011 01 000110 11 11(1)

(1) (1)

(2)

(2)

(2)

(2)

(2)

(3)

(2)

(2)

(4)

(4)

(2)

(4)

(4)

(5)

(5)

(6)

Received message

150

State D:(1 1)

00/0

00/1

11/0

11/1

10/0

10/1

01/0

01/1

c1c2/a1

(2) (3)

(4)

(5)

(5)


0011 01 0001

0 1

10 01

10 11 11

00 11 00

1 10

0001 11

Received message

Transmitted message



State A:(0 0)

State B:(0 1)

State C:(1 0)

0011 01 000110 11 11(1)

(1) (1)

(2)

(2)

(2)

(2)

(2)

(3)

(2)

(2)

(4)

(4)

(2)

(4)

(5)

(6)

Received message

151

State D:(1 1)

00/0

00/1

11/0

11/1

10/0

10/1

01/0

01/1

c1c2/a1

(2) (3)

(4)

(5)


0011 01 0001

0 1

10 01

10 11 11

00 11 00

1 10

0001 11

Received message

Transmitted message



State A:(0 0)

State B:(0 1)

0011 01 000110 11 11

(1)

(1)

(2)

(2)

(2)

10 0100 11 00 0001 11Corrected message

Received message

Error

152

State B:(0 1)

State C:(1 0)

State D:(1 1)

(2)

(2)

(2)

(2)



Appendix Appendix

Concept of error correctionConcept of error correction

153


Relationship between error correction, detection and Relationship between error correction, detection and

minimum Hamming distanceminimum Hamming distance

Cn=(x1,x2,・・,xn）C3Corrected area

s* 1

dmin≧2t*+s*+1

Possible to correct less and equal t*bits

Possible to detect less and equal than

(t*+s*) bits

t* t*

154

s* 1

dmin

C2

Detected area

C1 Codeword

Invalid codeword

t* t*


Example error correction and detectionExample error correction and detection

dC2C1

Possible to correct and detect less and

equal than 2bits errors

t*dmin=5

t*=2、s*=0t*

s*

dmin

Error

detected

equal than 2bits errors

Error

corrected

area

C2C1

dmin

t* t*Possible to correct 1bit error and detect

less and equal than 3bits errors

t*=1、s*=2

155

detected

area

C2C1dmin

s*Impossible to correct.

Possible to detect less and equal than

4bits errors

t*=0、s*=4

Error Correction

Documents

error data

cti error correction

error correction data

transmission error

cti example of error

cti error detection

cti random error

cti error factor