Unit 4 –Error-Control Coding

Dept. of Electrical and Computer Eng., NCTU 1

Unit 4 – Error-Control Coding

Instructor: Sau-Hsuan Wu

Comm. Systems Unit 4 – Error-Control Coding Sau-Hsuan Wu

What is error-control coding? Adding redundant bits to the message bits according to a

prescribed rule to minimize the effect of channel noise

Error-control codes are classified into block codes and convolutional codes A block code encoder accepts information in successive k-bit

blocks, and adds n-k redundant bits for each block The n-bit block is called a code word whose code rate is r = k/n



In a convolutional code, the encoding operation may be viewed as the discrete-time convolution of the input sequence with the impulse response of the encoder

A convolutional code encoder accepts message bits as a continuous sequence and thereby generates a continuous sequence of encoded bits

A waveform channel is said to be memoryless if the detector output in a given interval depends on the signal sent in that interval, and not on any previous transmission Under a memoryless channel, the combination of the

modulator, the waveform channel, and the detector may be modeled as a discrete memoryless channel (DMC)

A DMC is completely described by the transition probability p(j|i) of receiving symbol j, given symbol i was transmitted



If the modulator has only the binary symbols 0 and 1 as inputs, and the demodulator outputs binary quantized data only, the DMC becomes a binary symmetric channel (BSC)

A BSC is completely described by the transmission probability p

In the sequel, we discuss error-control coding in BSC



Linear block codes Consider an (n, k) linear block code in which k bits of the n

coded bits are always identical to the message sequence

A code is said to be linear if any two code words c1 and c2in the code book Bn can be added in modulo-2 arithmetic to produce a third code word c = c1 + c2 in the code book Bn

Block codes in which the message bits are transmitted in unaltered form are called systematic codes



The code word is defined as , 0, … , 1, , … , 1

The k right-most bits are message bits mi

The (n-k) left-most bits, bi, are parity bits, which are defined as linear sums of mi : bi = m0 p0,i + m1 p1,i +…+ mk-1 pk-1,i pj,i = 1 if bi depends on mj , otherwise pj,i = 0

Define m = [m0 , m1 , …, mk-1], b = [b0 , b1 , …, bn-k-1]. We have b = mP

, ⋯ ,⋮ ⋱ ⋮, ⋯ ,

And c = [c0 , c1 , …, cn-1] = [b : m] = m[P : Ik]



We thus define the k-by-n generator matrix as G = [P : Ik] G is said to be canonical if its k rows are linearly independent Therefore, we may rewrite c = m[P : Ik] = mG

Some properties of a systematic linear block codes Closure: ci + cj = miG + mjG = (mi + mj)G = ck Bn

Let H = [In-k : PT] H GT = PT + PT = 0 cHT = mG = mGHT = 0

Therefore, H is called the parity-check matrix of the code



Ex. 1: Repetition codes The simplest type of linear block codes with c = mG =

m[P : I1] = m[1n-1T : 1], and H = [In-1 : PT] = [In-1 : 1n-1]

Syndrome Let r = c + e, with e = {ei}, ei =1 or 0, being the error vector The syndrome is defined as s = rHT of dimension 1 by n-k Properties of the syndrome s

Property 1: s = (c + e)HT = cHT + eHT = eHT

(syndrome depends only on the error pattern, and not on c) Property 2: ei HT = ci HT + eHT = eHT , for i = 0,1,…,2k-1

For an error vector e, the set of vectors {ei |i = 0,1,…,2k-1} is called a coset of the code

Each coset of the code is characterized by a unique syndromeDept. of Electrical and Computer Eng., NCTU 8


The set of the n-k linear equations of sT = HeT is given bys0 = e0 + p0,0 en-k + p1,0 en-k+1 +…+ pk-1,0 en-1s1 = e1 + p0,1 en-k + p1,1 en-k+1 +…+ pk-1,1 en-1

⋮sn-k-1 = en-k-1 + p0, n-k-1 en-k + p1, n-k-1 en-k+1 +…+ pk-1, n-k-1 en-1 This set of equations is underdetermined There are 2k error patterns satisfy the syndrome equation

The minimum distance decoding principle The Hamming distance d(c1, c2) between two code words is

defined as the number of locations in which they differ The Hamming weight w(c) of a code word is defined as the

number of nonzero elements in the code word The Hamming distance d(c1, c2) = w(c1+c2)



The minimum distance dmin of a linear block code is defined as the dmin = min{d(ci, cj)} = min{w(ci+cj)}, i, j, ij

From the closure property of linear block codes, the sum (or the difference) of two code words is another code word. Accordingly, we have dmin = min{w(ci)}, i with ci 0

Suppose a code vector ci is transmitted, and r = ci + e The best decoding strategy is to pick ci for which d(ci, r) is

the smallest out of the 2k possible code vectors Based on this decoding method, we have whenever

w(e) t , provided that dmin = min{d(ci, cj)} 2t + 1



Ex. 2: (3, 1) codes A (3, 1) repetition code has a code work [0 0 0] for bit ‘0’

and another code word [1 1 1] for bit ‘1’ dmin = 3 Another (3, 1) code with a code work [0 1 0] for bit ‘0’ and

another code word [1 0 1] for bit ‘1’ dmin = 3 For a (3, 1) code with [0 0 0] for bit ‘0’ and [0 1 0] for bit ‘1’ dmin = 1 , and is bad code !!

Geometrical view of a (3, 1) repetition code Any two points that have a

Hamming distance of 3 forma good (3, 1) code



An (n, k) linear block code of minimum distance dmin can correct up to t errors if, and only if

12 1

Syndrome decoding Let , , … , denote the 2k code words of an (n, k) code The receiver has the task of partitioning the 2n possible

received vector r into 2k disjoint subsets , , … , in such a way that the subset corresponds to ci, for 1 i 2k

The 2k subsets { } constitute a standard array of the linear block, and can be constructed in a way described below Place the 2k code words in a row with the all-zero code word

c1 as the left-most element



Choose an error pattern e2, which has not previously appeared in the standard array, and place it under c1

The second row is formed by adding e2 to each of the remaining code words in the first row

Repeat the previous two steps for all possible error patterns ej

The 2k columns represent the disjoint subsets , , … ,



The 2n-k rows of the array represent the cosets of the code, and their first elements , , … , are called coset leaders

The received vector r is decoded into ci if it is in How do we decide to which the vector r belong? In the case of a BSC, the smaller the Hamming weight of an

e, the more likely it is to occur, with a probability of pw(e)(1-p)[n-w(e)] provided that the bit error rate p 0.5

The standard array should be constructed with each cosetleader having the minimum Hamming weight in its coset

We now have a syndrome decoding procedure as follows For the received vector r , compute s = rHT

Within the coset characterized by s, identify the coset leader e0

Compute the code word with c = r + e0



Ex. 3: Hamming codes A family of (n, k) linear block codes with the properties of

Block length: n = 2m -1 Number of message bits: k = 2m – m – 1 Number of parity bits: n – k = m, with m 3

Consider a (n, k) Hamming code with n = 7, k = 4 and m = 3

The generator matrix is 1 1 00 1 11 1 1

1 0 00 1 00 0 1

000

1 0 1 0 0 0 1

= [P : Ik]

The parity check matrix is 1 0 00 1 00 0 1

1 0 11 1 10 1 1

101

= [In-k : PT]



Given that k = 4, there are 2k = 16 code words shown below

The min distance of the code word is dmin = 3 The smallest number of columns in H that sum to zero is 3

Single-error correcting perfect code ∵ t 1 1



The 7 coset leaders {ei} are shown in the right table

The syndrome of ei is equal to the i-th row vector of HT

And ei corresponds to an error at the i-th bit of the code word

Ex: Suppose c =[1110010] is sent And r = [1100010] Then, s = rHT = [0 0 1], which

corresponds to e3 =[0010000] Flip the 3rd bit of r, yielding

[1110010] = c


Coset Leaders


Note that the sum of the 4th and 6th row of HT equals to [0 0 1] as well, which implies an e =[0001010] also satisfies s =[0 0 1]

Nevertheless, e =[0001010] is less likely to happen than e3 does The encoding and decoding procedure of Hamming code

Dual code For the G and H of a (n, k) code, we may define a dual

(n, n - k) code which has H as its generator matrix and G as its parity check matrix, with H and G satisfying GHT = 0


ĉ


Cyclic codes A binary code is said to be a cyclic code if it possesses:

Linearity property: c1 and c2 Bn c3 = c1 + c2 Bn

Cyclic property: Any cyclic shift of c Bn is still a code in Bn

Let c = [c0 , c1 , …, cn-1] denote a code word of an (n, k) code Bn

Then all [cn-1 , c0 , c1, …, cn-2], [cn-2 , cn-1, c0 ,…, cn-3],….[c1, c2,…, cn-1, c0] Bn

Cyclic codes are easy to encode, and posses a well-defined algebraic structure, which leads to efficient decoding

The algebraic properties of cyclic codes are developed based on the code polynomial defined on [c0 , c1 , …, cn-1]

⋯ The addition is operated on mod(Xn +1)



Xi c(X) mod(Xn +1) represents i-bit shifts in time

⋯⋯

⋯ mod 1⋯ ⋯

⋯ ⋯

Generator polynomial of degree (n k) ≜ 1 ⋯ A cyclic code is uniquely determined by in that

c(X) = a(X)g(X), where a(X) is a polynomial of degree k 1 Any factor of 1with degree (n k) can be used as



Encoding procedure of an (n, k) systematic cyclic code Given the message bits m0, m1,…, mk-1, define a message

polynomial ≜ ⋯ For parity bits b0 , b1 ,…, bn-k-1 , define

≜ ⋯ Let ≜

is the remainder after dividing by The entire encoding procedure is stated as follows

Compute Divide by to obtain the remainder Compute



Parity-Check polynomial A cyclic code is also uniquely specified by the parity-check

polynomial: ≜ 1 ⋯ Both and need to be the factors of 1, i.e.

mod 1 0 Any factor of 1with degree (n k) can be used as For large n, 1 may have many factors of degree (n k)

some are good cyclic codes, whereas some are bad cyclic codes The generator matrix G of an (n, k) cyclic code

The code c is spanned by , ,…, , namely



The parity-check matrix of an (n, k) cyclic code = 1

The degree of n, and the degree of k1 The coefficients of Xk, Xk+1,…, Xn-1 of must be zero

∑ 0, , … , 1



Recall that cHT, the parity-check matrix given by≜ 1 ⋯ is

The generator matrix G, and the parity-check matrix Hconstructed here are not in their systematic forms



Encoder for an (n, k) cyclic code with≜ 1 ⋯

Switch on the gate, and shift k message bits m0, m1,…,mk-1into the channel, with the first bit mk-1, and the last bit m0

Switch off the gate, and continue to shift out the content, which are b0, b1,…,bn-k-1, with bn-k-1 coming out the first, then bn-k-2, …, b1, and b0 the last bit, sequentially


m0, m1,…,mk-1

mk-1only


Calculation of the syndrome Let the received code word r = [r0, r1,…,rn-1] whose error

polynomial is ≜ ⋯

The remainder s(X) of degree nk1 is the syndrome polynomial of r(X)

When all ri are shifted into the register, its content defines



The syndrome of is Given that

⇒

Therefore, s(X) is also the syndrome of e(X) If the degree of nk1, then



Ex. 3: Hamming codes revisited Consider a (7, 4) Hamming code with n = 2m 1, m = 3 Factorize X7+1 into irreducible polynomials

X7 + 1=(1 + X)(1 + X2 + X3) (1 + X + X3) A polynomial is irreducible if it cannot be factored using only

polynomials with coefficients from binary field An irreducible polynomial of degree m is primitive if the

smallest n of Xn + 1 which the polynomial divides is n = 2m 1 Define the generator polynomial as g(X) = (1 + X + X3) This implies the parity-check polynomial is equal to

h(X) =(1 + X)(1 + X2 + X3) = 1 + X + X2 + X4

Assume we want to send a message sequence of 1001 whose polynomial is m(X) = 1 + X3



Step1: Multiplying m(X) by Xn-k yields m(X)Xn-k = X3 + X6

Step2: Divide m(X)Xn-k by g(X) = (1 + X + X3), leading to

The quotient a(X) = X + X3

The remainder b(X) = X + X2

Step3: The code polynomial is c(X) = b(X) + m(X)Xn-k = X + X2 + X3 + X6 011 1001


Message bitsParity bits


Step4: Show that g(X) and h(X) uniquely specify the generator matrix G and the parity-check matrix H Recall that code c is spanned by , ,…, Given g(X) = (1 + X + X3), we have

coeffs. ofcoeffs. ofcoeffs. ofcoeffs. of

1 1 00 1 10 0 1

1 0 00 1 01 0 1

000

0 0 0 1 1 0 1

Note that G’ is not in systematic form, nevertheless, we may add row 1 to row 3, and the sum of 1st and 2nd row to row 4 to

form

1 1 00 1 11 1 1

1 0 00 1 00 0 1

000

1 0 1 0 0 0 1

= [P : Ik]



For the construction of the parity check matrix H, first we need the reciprocal of h(X) = X4 h(X-1) = 1 + X2 + X3 + X4

leading to

coeffs. of coeffs. of coeffs. of

1 0 10 1 0

1 1 01 1 1

00

0 0 1 0 1 1 1

Again this is not in systematic form, while we may add the 3rd row to the 1st row to obtain

1 0 00 1 00 0 1

1 0 11 1 10 1 1

101

= [In-k : PT]



Encoder for the (7, 4) cyclic code of g(X) = (1 + X + X3)

Suppose we input message sequence (1001) The content of the

registers is shown to the right

The content after fourshifts is the parity bits



Syndrome calculator for g(X) = (1 + X + X3)

Let the transmitted bits be (0111001), and the received word be (0110001) The content of the

register is shown to the right

After 7 shifts, s = (110) From Table 2, we have

e = (0001000)


Unit 4 –Error-Control Coding

Documents