ECEN 5682 Theory and Practice of Error Control …ecee.colorado.edu/~mathys/ecen5682/slides/block99.pdfPeter Mathys ECEN 5682 Theory and Practice of Error Control Codes. Introduction

Introduction to Block Codes

ECEN 5682 Theory and Practice of Error ControlCodes

Introduction to Block Codes

Peter Mathys

University of Colorado

Spring 2007

Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes

Introduction to Block CodesBasic DefinitionsLinear Block Codes

Basic Definitions

Definition: A block code of length n and size M over an alphabetwith q symbols is a set of M q-ary n-tuples called codewords.

Example: Code #1. Binary code of length n = 5 with M = 4codewords given by

C = {00000, 01011, 10101, 11110} .

Definition: The rate R of a q-ary block code of length n with Mcodewords is given by

R =logq M

n.



Definition: The redundancy r of a q-ary block code of length nwith M codewords is given by

r = n − logq M .

Example: Code #1 has rate R = log2 4/5 = 2/5 = 0.4 andredundancy r = 5− log2 4 = 3 bits.

Example: Code #2. 5-ary code of length n = 4 with M = 5codewords given by

C = {0000, 1342, 2134, 3421, 4213} .

This code has rate R = log5 5/4 = 1/4 = 0.25 and redundancyr = 4− log5 5 = 3 symbols.



The goal when using error control codes is to detect and/or correcttransmission errors. Suppose code #1 is used and the (corrupted)codeword v = (00101) is received. Comparing v with all “legal”codewords and marking the discrepancies with * yields:

00000 01011 10101 1111000101 00101 00101 00101----- ----- ----- -----..*.* .***. *.... **.*.

The discrepancies are the result of transmission errors. If all errorpositions are marked with one and all other positions with zero,then the received codeword v = (00101) corresponds to the set ofpossible errors

E = {00101, 01110, 10000, 11010} ,

when code #1 is used. But which of these 4 errors is the “right”one?



To decide which error out of a set of errors is the “right” one, oneneeds to make additional assumptions about the likelihood oferrors and error patterns. The two most common models for theoccurrence of errors are:

(i) Independent and identically distributed (iid) errors withprobablity ε. This requires a memoryless transmission channelmodel.

(ii) Burst errors of length L. If an error occurs, it is very likelythat it is followed by L− 1 more errors. Burst errors occur forinstance in mobile communications due to fading and inmagnetic recording due to media defects. Burst errors can beconverted to iid errors by the use of an interleaver.



More generally, and especially for non-binary codes, one also needsa model for the error amplitudes. Two possibilities are

(i) Uniformly distributed non-zero error amplitudes. This is agood model for orthogonal signaling.

(ii) Non-uniformly distributed non-zero error amplitudes withsmaller error magnitudes more likely than larger ones. This isa good model for QAM signaling.



In addition to the models that describe which error pattern e ismost likely, a transmission channel model is also needed thatspecifies how codewords c and error patterns e are combined toform the received codeword v = f (c, e). The most prevalent modelis the additive model shown in the following figure. Note thataddition is often assumed to be modulo q addition for a q-ary code.

+

error e

c

codeword

v = c + e

receivedcodewordvector addition

(often modulo q)

A concise graphical way to describe simple error models is in theform of a discrete channel model that shows all possible transitionsfrom the channel input X to the channel output Y , together withthe associated transition probabilities pY |X (y |x).



Example: The simplest discrete channel model is the memorylessbinary symmetric channel (BSC) shown in the following figure.

• •

••

0 0

1 1

InputX

OutputY

1− ε

ε

ε

1− ε

This channel is completely described by the set of four transitionprobabilities:

pY |X (0|0) = 1− ε , pY |X (1|0) = ε ,

pY |X (0|1) = ε , pY |X (1|1) = 1− ε .

Clearly P{Y 6= X} = ε and thus the (uncoded) probability of a biterror is Pb(E) = ε.



Thus, if ε < 0.5 on a memoryless BSC, fewer errors are more likelyand the “right” error pattern is the one with the fewest number of1’s in it. Note that, since all symbols are binary here, only errors ofamplitude 1 are possible and no specification for the distribution oferror amplitudes is needed.

Example: Suppose q = 5, errors occur iid with Pr{Y 6= X} < 0.5,and uniformly distributed amplitudes. The corresponding channelmodel is a memoryless 5-ary symmetric channel (5SC) withtransition probabilities

pY |X (y |x) =

{ε/4 , if y 6= x , x , y ∈ {0, 1, 2, 3, 4} ,1− ε , if y = x , x , y ∈ {0, 1, 2, 3, 4} .

In this case the decoding rule assumes again that the “right” errorpattern is the one with the fewest nonzero symbols in it.



Example: Suppose again q = 5 and errors occur iid withP{Y 6= X} < 0.5. But now assume that only errors of magnitude1 occur, with +1 and −1 being equally likely. This leads toanother memoryless 5SC with transition probabilities

pY |X (y |x) =

ε/2 , if y = x ± 1 (mod 5) , x , y ∈ {0, 1, 2, 3, 4} ,1− ε , if y = x , x , y ∈ {0, 1, 2, 3, 4} ,0 , otherwise .

Now the decoder decides on the error pattern with the fewestnumber of ±1 (mod 5) symbols as the “right” error.



Once an error and a channel model are defined, a distancemeasure between codewords can be defined. Then one candetermine the minimum distance between any two distinctcodewords and this in turn determines how many errors a code candetect and/or correct under the given error and channel models.

For the iid error model with (discrete) uniform error amplitudedistribution the most appropriate measure is Hamming distancewhich is defined as follows.

Definition: The Hamming distance d (H)(x, y) (or simply d(x, y))between two q-ary n-tuples x and y is the number of places inwhich they differ.

Example: d(10221, 20122) = 3.



The Hamming distance is probably the most popular distancemeasure for error control codes. Another measure that is moresuitable in cases where smaller error magnitudes are more likelythan larger ones is the Lee distance which is defined next.

Definition: The Lee distance d (L)(x, y) between two q-ary n-tuplesx and y is defined as

d (L)(x, y) = |x0 − y0|+ |x1 − y1|+ . . . + |xn−1 − yn−1| ,where the magnitude |v | of a q-ary symbol v is computed moduloq as

|v | = min{v , q − v} .

Definition: The minimum distance dmin of a codeC = {ci , i = 0, 1, . . . M − 1} is the smallest distance between anytwo distinct codewords of the code, i.e.,

dmin = mini 6=j

d(ci , cj) , ci , cj ∈ C .



Example: Code #2 has has the following Hamming distancesbetween pairs of distinct codewords:

d (H)(x, y) 0000 1342 2134 3421 4213

0000 − 4 4 4 41342 4 − 4 4 42134 4 4 − 4 43421 4 4 4 − 44213 4 4 4 4 −

The Lee distances between pairs of distinct codewords are:

d (L)(x, y) 0000 1342 2134 3421 4213

0000 − 6 6 6 61342 6 − 6 6 62134 6 6 − 6 63421 6 6 6 − 64213 6 6 6 6 −

Thus, for code #2, d(H)

min = 4 and d(L)

min = 6.



Example: Code #3. Binary code with n = 10 and the followingset of M = 4 codewords

C = {0010010111, 0100101110, 1001011100, 1110001001} .

For this code the Hamming distances between pairs of codewordsare

d (H)(x, y) 0010010111 0100101110 1001011100 1110001001

0010010111 − 6 6 60100101110 6 − 6 61001011100 6 6 − 61110001001 6 6 6 −

That is, code #3 has minimum Hamming distance 6.



Theorem: A code with minimum Hamming distance dmin candetect all error patterns with dmin − 1 or fewer nonzerocomponents.

Proof: The only error patterns that cannot be detected are thosethat make the transmitted codeword look like another codeword.But because the smallest Hamming distance between any twodistinct codewords is dmin, this can only happen if the error patternaffects dmin or more coordinates of the transmitted codeword.QED

Definition: The sphere of radius t about codeword c is the set

St(c) = {v | d(c, v) ≤ t} ,

where d(., .) is the distance measure used (e.g., Hammingdistance).



Example: Consider the codeword c = 01011 from binary code #1.Then, using Hamming distance as the distance measure,

S0(01011) = {01011}S1(01011) = {01011, 11011, 00011, 01111, 01001, 01010}S2(01011) = {01011, 11011, 00011, 01111, 01001, 01010,

10011, 11111, 11001, 11010, 00111, 00001, 00010, 01101, 01110, 01000} .

Theorem: The Hamming distance satisfies the triangle inequality,i.e., for any 3 n-tuples x, y, z

d(x, y) + d(y, z) ≥ d(x, z) .



d(x, y) + d(y, z) ≥ d(x, z)

Proof: First note that for any u = (u0, u1, . . . , un−1) andv = (v0, v1, . . . , vn−1), the Hamming distance satisfies

d(u, v) = d(u0, v0)+d(u1, v1)+. . .+d(un−1, vn−1), d(ui , vi ) ∈ {0, 1},

where the addition is over the reals. Consider now a coordinate,say j , where x and z differ, i.e., xj 6= zj . Then there are threepossible cases for yj :

(i) yj = xj , which implies d(yj , zj) = 1.

(ii) yj = zj , which implies d(yj , xj) = 1.

(iii) yj 6= xj and yj 6= zj , which implies d(yj , xj) = 1 andd(yj , zj) = 1.

Thus, in all three cases d(x, y) + d(y, z) increases by at least onewhile d(x, z) increases by exactly one. QED



Theorem: A code with minimum Hamming distance dmin cancorrect all error patterns with t or fewer nonzero components aslong as 2t < dmin.

Proof: Because of the triangle inequality, the spheres St(ci ) andSt(cj) of any two distinct codewords ci and cj contain no commonelements as long as 2t < dmin.

Theorem: The Hamming or sphere-packing bound for q-ary codeswith dmin = 2t + 1 states that the redundancy r must satisfy

r ≥ logq

t∑j=0

(n

j

)(q − 1)j

.

Proof: Left as an exercise.



Definition: A code which satisfies the Hamming bound withequality is called a perfect code.

Example: Code #4. The binary code with blocklength n = 7 andthe following set of M = 16 codewords

C = {0000000, 0001011, 0010101, 0011110, 0100110, 0101101, 0110011, 0111000,

1000111, 1001100, 1010010, 1011001, 1100001, 1101010, 1110100, 1111111} ,

is a perfect code. By inspection dmin = 3 is found and thus allpatterns of t = 1 errors are correctable. Therefore

r ≥ log2

[(7

0

)+

(7

1

)]= log2(1 + 7) = 3 .

But r = n − log2 M = 7− 4 = 3, i.e., the code satisfies theHamming bound with equality. This code is known as binary(7, 4, 3) Hamming code.



Example: Probably the most prominent and celebrated perfectcode is the binary Golay code with blocklength n = 23, M = 212

codewords and minimum Hamming distance dmin = 7. It cancorrect all error patterns with up to t = 3 errors and thus theHamming bound requires that

r ≥ log2

»“23

0

”+

“23

1

”+

“23

2

”+

“23

3

”–= log2(1+23+253+1771) = log2(2048) = 11.

The actual redundancy of the code isr = 23− log2 212 = 23− 12 = 11 and therefore it is perfect.

Note: Don’t take the name “perfect code” too literally. Perfectjust simply means that the spheres of radius t = (dmin − 1)/2around all codewords fill out the whole codewordspace perfectly. Itdoes not necessarily mean that perfect codes are the best errordetecting and/or correcting codes.



How easy is it to find a block code by trial and error?

Example: Binary rate R = 0.8 code of length n = 10. There are(210

28

)ways to choose 256 codwords out of 1024 possibilities. Here

is the actual number:



Linear Block Codes

Definition: A q-ary linear (n, k) blockcode C is defined as the setof all linear combinations, taken modulo q, of k independentvectors from V , where V is the set of all q-ary n-tuples. If C hasminimum distance dmin, C is called a q-ary (n, k, dmin) code.

Definition: A generator matrix G of a linear (n, k) code C is ak × n matrix whose rows form a basis for the k-dimensionalsubspace C of V .

Definition: The q-ary k-tuple u = (u0, u1, . . . uk−1) is used todenote a dataword. In general, it is assumed that there are norestrictions on u, i.e., it may take on all possible qk values and,unless otherwise specified, all these values are equally likely. Othernames for u are message or information word.



Definition: For the encoding operation, any one-to-one associationbetween datawords u and codewords c may be used. For a linearcode with generator matrix G, the most natural encodingprocedure is to use

c = uG .

Example: Code #5. Ternary (n = 5, k = 2) code with generatormatrix

G =

[1 0 2 1 20 1 2 2 1

].

This defines a linear code C whose codewords

C = {00000, 01221, 02112, 10212, 11100, 12021, 20121, 21012, 22200} ,

lie in a 2-dimensional subspace of V , where V is the set of all3-ary 5-tuples.



Example: Code #1 is a linear binary (n, k, dmin) = (5, 2, 3) codewith generator matrix

G =

[1 0 1 0 10 1 0 1 1

].

To verify this, generate the set of codewords by taking all possiblelinear combinations (modulo 2, or modulo q in general) of the rowsof G as follows:

C ={

(0, 0) · G = (0, 0, 0, 0, 0) , (0, 1) · G = (0, 1, 0, 1, 1) ,

(1, 0) · G = (1, 0, 1, 0, 1) , (1, 1) · G = (1, 1, 1, 1, 0)}

.



Example: Code #2 is a linear 5-ary (4,1,4) code with generatormatrix

G =[1 3 4 2

].

The set of codewords is obtained as

C =n

0 · G = (0, 0, 0, 0) , 1 · G = (1, 3, 4, 2) , 2 · G = (2, 1, 3, 4) ,

3 · G = (3, 4, 2, 1) , 4 · G = (4, 2, 1, 3)o

.

Example: Code #3 is a nonlinear binary code with n = 10. Thesum of codewords

(0, 1, 0, 0, 1, 0, 1, 1, 1, 0) + (1, 0, 0, 1, 0, 1, 1, 1, 0, 0) = (1, 1, 0, 1, 1, 1, 0, 0, 1, 0) ,

for instance, is not a codeword of the code. In fact, the 4codewords

C = {0010010111, 0100101110, 1001011100, 1110001001} ,

are linearly independent and form the basis of a 4-dimensionalsubspace of the space of all binary 10-tuples.



Definition: The Hamming weight w(c) of a codeword c ∈ C isequal to the number of nonzero components of c. The minimumHamming weight wmin of a code C is equal to the smallestHamming weight of any nonzero codeword in C.

Definition: The Lee weight w (L)(c) of a codeword c ∈ C is definedas

w (L)(c) = |c0|+ |c1|+ . . . + |cn−1| ,

where the magnitude |v | of a q-ary symbol v is computed moduloq as

|v | = min{v , q − v} .

The minimum Lee weight w(L)min of a code C is equal to the smallest

Lee weight of any nonzero codeword in C.



Theorem: For a linear code dmin = wmin.

Proof: For any x, y ∈ C

dmin = minx 6=y

d(x, y) = minx6=y

d(x− y, 0) = minc6=0

w(c) ,

where c = x− y ∈ C because the code is linear. QED



Standard Array for Decoding. A good conceptual, butpractically inefficient way to visualize the decoding operation for aq-ary linear code under the Hamming distance measure is theso-called standard array. It is set up as follows:

(1) The first row of the array consists of all the codewords of thecode, starting on the left with the all-zero codeword that mustbe present in every linear code.

(2) The first column starts out with all q-ary n-tuples that are inthe decoding sphere St(0) of radius t about the all-zerocodeword c0 = 0, where t is the maximum number of errorsthe code can correct. There are P = |St(0)| n-tuples in thissphere, and, assuming an additive (modulo q) error model, allare correctable error patterns (including the the all-zero“error”). The elements in this column are called the cosetleaders.



Standard Array for Decoding (contd.)

(3) Making use of the linearity of the code, anything that appliesto the all-zero codeword c0 also applies to any other codewordcj by simply translating the origin. Thus, the first P entries ofthe j-th column make up the decoding sphere St(cj). Allentries in the j-th column are obtained by adding (modulo q)the error pattern on the left to the codeword above.

(4) Each of the rows in the standard array is called a coset.Altogether, the first P rows contain the M = qk distinctdecoding spheres St(cj) for j = 0, 1, . . . M − 1. If the code is aperfect code, then P M = qn, else there are(qn − P M) = (Q − P)M, where Q = qn−k , distinct q-aryn-tuples that do not yet appear in the array. These correspondto error patterns with more than t errors, but in general only afew of these are correctable.



Standard Array for Decoding (contd.)

(5) To complete the standard array, organize the (Q − P)M q-aryn-tuples that do not yet appear in the array into Q − Pcosets. For each coset, select an error pattern of smallestHamming weight that has not yet appeared anywhere in thearray as a coset leader, and complete the coset by adding theerror pattern on the left to the codeword above. Because ofthe linearity of the code it can be shown that it is alwayspossible to fill the bottom Q − P rows in this way withdistinct n-tuples and that the set of all Q rows contains all qn

possible q-ary n-tuples.



Decoding using the standard array consists of looking up thereceived n-tuple in the array and returning the codeword above itas the result.

Definition: A decoder which decodes only received n-tuples withindecoding spheres of radius t or less, but not the whole decodingregion, is called an incomplete decoder. If one of the n-tuples inthe bottom Q − P rows of the standard array is received, anincomplete decoder declares a detected but uncorrectable errorpattern. Conversely, a complete decoder assigns a nearbycodeword to every received n-tuple.

Note: For a perfect t-error correcting code P = Q, and no cosetleader has weight greater than t.



The following figure shows the contents of the standard arraygraphically.

decoding sphere decodingof radius t region........... .............

codewords ----> c0= : c1 : c2 ... : cM-1 :00..0 : : : :

-------------------------------------------------------/ e1 : c1+e1 : c2+e1 ... : cM-1+e1 :| .......................................................

coset -----:-> e2 : c1+e2 : c2+e2 ... : cM-1+e2 : : cosetleader | .......................................................

| e3 : c1+e3 : c2+e3 ... : cM-1+e3 :1,2,..,t | . : . : . : . :error | . : . : . : . :

patterns | eP-1 : c1+eP-1 : c2+eP-1 ... : cM-1+eP-1 : P rows\ ........... : : above line

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -/ eP c1+eP c2+eP : cM-1+eP : Q-P rows

more than | . . . : . : below linet errors | . . . : . :

\ eQ-1 c1+eQ-1 c2+eQ-1 ... : cM-1+eQ-1 :.............

1



Example: Code #1 has minimum (Hamming) distance dmin = 3and thus is a single error correcting code. The standard array forthis code is:

decoding sphere decoding

of radius 1 region

......... .........

c0 : c1 : c2 : c3 :

codewords ----> 00000 : 01011 : 10101 : 11110 :

----------------------------------------

/ 00001 : 01010 : 10100 : 11111 :

single | 00010 : 01001 : 10111 : 11100 :

errors < 00100 : 01111 : 10001 : 11010 :

| ........................................

coset -----:-> 01000 : 00011 : 11101 : 10110 : : coset

leader | ........................................

\ 10000 : 11011 : 00101 : 01110 :

......... : :

- - - - - - - - - - - - - - - - - - - - - - -

00110 01101 10011 : 11000 :

01100 00111 11001 : 10010 :

.........

1

The selection of the coset leaders for the last two rows is notunique. For example, 11000 and 10010 could have been usedinstead of 00110 and 01100.



Because the size of the standard array increases exponentially inthe block length n, it is quite clearly not very practical even formoderately large values of n. Therefore the concept of a paritycheck matrix is introduced, which will lead to a more compactdecoding method using syndromes.

Definition: Let C ⊂ V , where V is the set of all q-ary n-tuples, bea linear (n, k) code. Then the dual or orthogonal code of C,denoted C⊥ (“C perp”), is defined by

C⊥ = {u ∈ V | u ·w = 0 for all w ∈ C} .

The orthogonal complement of C has dimension n− k and thus C⊥is a linear (n, n − k) code.



Definition: A parity check matrix H of a linear (n, k) code C is a(n − k)× n matrix whose rows form a basis for the(n− k)-dimensional subspace C⊥ of the set of all q-ary n-tuples V .That is, any parity check matrix of C is a generator matrix of C⊥.

Theorem: Let C be a q-ary linear (n, k) code with generator matrixG and parity check matrix H. Then, using arithmetic modulo q,ci H

T = 0, where T denotes transpose, for all ci ∈ C, andGHT = 0.

Proof: Follows directly from the definition of C⊥ and thedefinition of H. QED



Example: Code #5 (ternary (5,2,3) code) has generator matrix Gand parity check matrix H given by

G =

[1 0 2 1 20 1 2 2 1

], H =

1 1 1 0 02 1 0 1 01 2 0 0 1

.

Thus, C⊥ is the set

C⊥ = {00000, 12001, 21002, 21010, 00011, 12012, 12020, 21021, 00022,

11100, 20101, 02102, 02110, 11111, 20112, 20120, 02121, 11122,

22200, 01201, 10202, 10210, 22211, 01212, 01220, 10221, 22222}



Theorem: A linear code C contains a nonzero codeword ofHamming weight w iff a linearly dependent set of w columns of Hexists.

Proof: Let c ∈ C have weight w . From cHT = 0 we can thus finda set of w linearly dependent columns of H. Conversely, if Hcontains a linearly dependent set of w columns, then we canconstruct a codeword c with nonzero coefficients corresponding tothe w columns, such that cHT = 0. QED

Corollary: A linear code C has minimum weight wmin iff every setof wmin − 1 columns of H is linearly independent and at least oneset of wmin columns of H is linearly dependent.



Definition: Elementary row operations on a matrix are thefollowing:

(i) Interchange of any two rows.

(ii) Multiplication of any row by a nonzero scalar.

(iii) Replacement of any row by the sum of itself and a multiple ofany other row.

Definition: A matrix is said to be in row-echelon form if it satisfiesthe following conditions:

(i) The leading term of every nonzero row is a one.

(ii) Every column with a leading term has all other entries zero.

(iii) The leading term of any row is to the right of the leadingterm in every previous row. All-zero rows (if any) are placedat the bottom.

Note: Under modulo q arithmetic, any matrix can be put intorow-echelon form by elementary row operations if q is a primenumber.



Example: Let q = 11 and consider the matrix

A =

246 9 1 10 6 10 2 01 7 4 9 10 10 8 62 3 9 7 8 4 7 4

35 .

Multiply the first row by 6−1 = 2. Then replace the second row by thedifference of the second row minus the new first row. Next, subtract 2 timesthe new first row from the third row and replace the third row with the resultto obtain

A′ =

241 7 2 9 1 9 4 00 0 2 0 9 1 4 60 0 5 0 6 8 10 4

35 .

Now start by multiplying the second row by 2−1 = 6. Then subtract 2 timesthis row from the first row. Finally, note that the third row is just a multiple ofthe second row, so that it can be replaced by an all zero row. The result is A inrow-echelon form:

A′′ =

241 7 0 9 3 8 0 50 0 1 0 10 6 2 30 0 0 0 0 0 0 0

35 .



Definition: Two codes which are the same except for apermutation of codeword components are called equivalent.

The generator matrices G and G′ of equivalent codes are related asfollows. The code corresponding to G is the set of all linearcombinations of rows of G and is thus unchanged under elementaryrow operations. Permutation of the columns of G corresponds topermutation of codeword components and therefore two codes areequivalent if their generator matrices G and G′ are related by

(i) Column permutations, and

(ii) Elementary row operations.



From this it follows that every generator matrix G of a linear codeis equivalent to one in row-echelon form. Because G is a k × nmatrix whose rows span a k-dimensional subspace, all rows of Gmust be linearly independent. This proves the following

Theorem: Every generator matrix of a q-ary linear code, where q isa prime (or a prime power), is equivalent to one of the form

G = [Ik | P] , or G = [P | Ik ] ,

where Ik is a k × k identity matrix and P is a k × (n − k) matrix.

Definition: A code C with codewords whose first (or last) kcomponents are the unmodified information symbols is called asystematic code. The remaining n − k codeword symbols arecalled parity symbols. A systematic code has generator matrixG = [Ik | P] (or G = [P | Ik ]).



Let u = (u0, u1, . . . , uk−1) be the information word and letc = (c0, c1, . . . , cn−1) be the corresponding codeword. If G is insystematic form, then

G =

1 0 0 . . . 0 p0,k p0,k+1 . . . p0,n−1

0 1 0 . . . 0 p1,k p1,k+1 . . . p1,n−1

0 0 1 . . . 0 p2,k p2,k+1 . . . p2,n−1...

......

. . ....

......

. . ....

0 0 0 . . . 1 pk−1,k pk−1,k+1 . . . pk−1,n−1

,

then the components of c = uG are cj = uj , for 0 ≤ j ≤ k − 1, and

cj = u0 p0,j + u1 p1,j + · · ·+ uk−1 pk−1,j , k ≤ j ≤ n − 1 .

This latter set of equations is known as the set of parity-checkequations of the code.



Theorem: The systematic form of H corresponding to G = [Ik | P]is

H =[− PT | In−k

].

Proof: Multiplying corresponding submatrices in G and HT

together yields

GHT = [Ik | P]

[−P

In−k

]= −Ik P + P In−k = 0 .

Thus, H = [−PT | In−k ] satisfies GHT = 0. QED



Written out explicitly, the systematic form of H is

H =

−p0,k −p1,k . . . −pk−1,k 1 0 0 . . . 0−p0,k+1 −p1,k+1 . . . −pk−1,k+1 0 1 0 . . . 0−p0,k+2 −p1,k+2 . . . −pk−1,k+2 0 0 1 . . . 0

......

. . ....

......

.... . .

...−p0,n−1 −p1,n−1 . . . −pk−1,n−1 0 0 0 . . . 1

.

From cHT = 0 it thus follows for the m-th row of H that

−c0 p0,k+m − c1 p1,k+m − · · · − ck−1 pk−1,k+m + ck+m = 0 , 0 ≤ m ≤ n− k − 1 .

Letting j = k + m and using the fact that for a systematic codeci = ui , 0,≤ i ≤ k − 1, one obtains

cj = u0 p0,j + u1 p1,j + · · ·+ uk−1 pk−1,j , k ≤ j ≤ n − 1 ,

which is the same set of parity check equations as before. Thisshows that a systematic linear (n, k) code is completely specifiedeither by its generator matrix G or by its parity check matrix H.



Example: Let q = 11 and consider the (8, 2) code with generatormatrix

G =

[3 10 0 5 9 2 0 41 7 1 9 2 3 2 8

].

Put G into row-echelon form which yields

G′ =

[1 7 0 9 3 8 0 50 0 1 0 10 6 2 3

].

Note that G and G′ produce exactly the same set of codewords,but using a different set of basis vectors and therefore a differentmapping from datawords u to codewords c. To obtain a generatormatrix in systematic form, permute the second and third columnsof G′ so that

Gsys =

[1 0 7 9 3 8 0 50 1 0 0 10 6 2 3

].



Now, using Hsys = [−PT | In−k ], one easily finds

Hsys =

266666644 0 1 0 0 0 0 02 0 0 1 0 0 0 08 1 0 0 1 0 0 03 5 0 0 0 1 0 00 9 0 0 0 0 1 06 8 0 0 0 0 0 1

37777775 .

Finally, to obtain a parity check matrix H for the original generatormatrix G, all the column permutations that were necessary toobtain Gsys from G need to be undone. Here only columns twoand three need to be permuted to obtain

H =

266666644 1 0 0 0 0 0 02 0 0 1 0 0 0 08 0 1 0 1 0 0 03 0 5 0 0 1 0 00 0 9 0 0 0 1 06 0 8 0 0 0 0 1

37777775 .

A quick check shows that indeed GHT = 0 modulo 11.



Theorem: Singleton bound. The minimum distance of any linear(n, k) code satisfies

dmin ≤ n − k + 1 .

Proof: Any linear code can be converted to systematic form(possibly permuting coordinates which does not affect dmin) andthus G = [Ik | P]. Since P is a k × (n − k) matrix and systematiccodewords with only one nonzero information symbol exist, theresult follows. QED

Note: It can be shown that the Singleton bound also applies tonon-linear codes.



Definition: Any linear code whose dmin satisfies

dmin = n − k + 1 ,

is called maximum distance separable (MDS).

Note: The name maximum distance separable code comes from the fact thatsuch a code has the maximum possible (Hamming) distance betweencodewords and that the codeword symbols can be separated into data symbolsand parity check symbols (i.e., the code has a systematic encoder).

Example: Code #6. The ternary (4, 2) code with generator matrix

G =

»1 0 2 20 1 2 1

–,

is an MDS code. The set of codewords is

C = {0000, 0121, 0212, 1022, 1110, 1201, 2011, 2102, 2220} .

From this it is easily seen that dmin = 3 = n − k + 1, which proves the claim

that this code is MDS.Peter Mathys ECEN 5682 Theory and Practice of Error Control Codes


Definition: Let C be a linear (n, k) code and let v = c + e, wherec ∈ C is a codeword and e is an error vector of length n, be areceived n-tuple. The syndrome s of v is defined by

s = vHT .

Theorem: All vectors in the same coset (cf. standard arraydecomposition of a linear code) have the same syndrome, uniqueto that coset.

Proof: If v and v′ are in the same coset, then v = ci + e and v′ = cj + e forsome e (coset leader) and codewords ci , cj . But, for any codeword c, cHT = 0and therefore

s = v HT = ci HT| {z }

=0

+eHT = eHT

s′ = v′ HT = cj HT| {z }

=0

+eHT = eHT =⇒ s = s′ .

Conversely, suppose that s = s′. Then s− s′ = (v − v′)HT = 0, which implies

that v − v′ is a codeword. But that further implies that v and v′ are in the

same coset. QED



Note: In practice, this theorem has some important consequences.To decode a linear code, one does not need to store the wholestandard array. Only the mapping from the syndrome to the mostlikely error pattern needs to be stored.

Example: Syndrome decoding for code #1. This is a binary codewith parity check matrix

H =

1 0 1 0 00 1 0 1 01 1 0 0 1

.

Computing s = eHT for e = 0, all single error error patterns, andsome double error patterns, the following table that uniquelyrelates syndromes to error patterns is obtained.



Error e Syndrome s

00000 00000001 00100010 01000100 10001000 01101000 101

00110 11001100 111

H =

241 0 1 0 00 1 0 1 01 1 0 0 1

35

Note that an incomplete (or bounded distance) decoder would only use the first

six entries (above the dividing line at the bottom) for decoding. The choice of

the error patterns for the last two entries is somewhat arbitrary (just as it was

in the case of the standard array), and other double error patterns that yield

the same syndromes could have been used. Suppose now that v = (11101) was

received. To decode v, compute s = v HT = (011). From the syndrome lookup

table the corresponding e = (01000). Finally, the corrected codeword c is

obtained as c = v − e = (10101).



Note: To construct a linear q-ary single error correcting code withredundancy r = n − k, one can start from a parity check matrix Hwhose columns are q-ary r -tuples with the property that they areall distinct, even when multiplied by an arbitrary nonzero q-aryscalar. The resulting codes are called Hamming codes.

Example: Natural parameters (n, k, dmin) of commonly used small binary linearcodes are:

(7, 4, 3)(15, 11, 3) (15, 7, 5) (15, 5, 7)

(23, 12, 7)(31, 26, 3) (31, 21, 5) (31, 16, 7) (31, 11, 11)(63, 57, 3) (63, 51, 5) (63, 45, 7) (63, 39, 9) (63, 36, 11)(127, 120, 3) (127, 113, 5) (127, 106, 7) (127, 99, 9) (127, 92, 11)(255, 247, 3) (255, 239, 5) (255, 231, 7) (255, 223, 9) (255, 215, 11)

Note that most of the block lengths of these codes are of the form 2m − 1 for

some integer m. Such block lengths are called primitive block lengths.



Modified Linear Blockcodes. Often the natural parameters ofblock codes are not suitable for a particular application, e.g., forcomputer storage applications the data length is typically amultiple of 8, whereas the natural parameter k of a code may be a“crummy” number like 113 (= 14× 8 + 1). Thus, it may benecessary to change either k or n or both. To explain the differentprocedures, code #7 which is a linear binary (6, 3, 3) code with Gand H as shown below, is used.

G =

1 0 0 1 1 00 1 0 1 0 10 0 1 0 1 1

, H =

1 1 0 1 0 01 0 1 0 1 00 1 1 0 0 1

.

The six different modifications that can be applied to theparameters n and k of a linear block code are: Lenghtening(n+,k+), shortening (n−,k−), extending (n+,k =), puncturing(n−,k =), augmenting (n =,k+), and expurgating (n =,k−).



Lengthening. Increase blocklength n by adding more data symbolswhile keepig redundancy r = n − k fixed. The result is a code thathas n and k increased by the same amount. In the best case dmin

will be unchanged, but it can drop to as low as 1 and needs to bereexamined carefully. Example: Code #7 lengthened by 1 results ina (7, 4, 3) code with generator and parity check matrices

G =

1 0 0 0 1 1 10 1 0 0 1 1 00 0 1 0 1 0 10 0 0 1 0 1 1

, H =

1 1 1 0 1 0 01 1 0 1 0 1 01 0 1 1 0 0 1

.



Shortening. Decrease blocklength n by dropping data symbolswhile keeping the redundancy r fixed. The resulting code has nand k reduced by the same amount. In most cases dmin will beunchanged, occasionally dmin may increase. Example: Code #7shortened by 1 results in a (5, 2, 3) code with generator and paritycheck matrices

G =

[1 0 1 0 10 1 0 1 1

], H =

1 0 1 0 00 1 0 1 01 1 0 0 1

.



Extending. Increase blocklength n by adding more parity checksymbols while keeping k fixed. The result is a code that has n andr = n − k increased by the same amount. The minimum distancemay or may not increase and needs to be reexamined. A commonmethod to extend a code from n to n + 1 is to add an overallparity check. Example: Code #7 extended by 1 results in a(7, 3, 4) code with generator and parity check matrices

G =

1 0 0 1 1 0 10 1 0 1 0 1 10 0 1 0 1 1 1

, H =

1 1 0 1 0 0 01 0 1 0 1 0 00 1 1 0 0 1 01 1 1 0 0 0 1

.



Puncturing. Decrease blocklength n by dropping parity checksymbols while keeping k fixed. The resulting code has n andr = n − k decreased by the same amount. Except for the trivialcase of removing all zero columns from G, the minimum distancedecreases. Example: Code #7 punctured by 1 yields a (5, 3, 2)code with generator and parity check matrices

G =

1 0 0 1 10 1 0 1 00 0 1 0 1

, H =

[1 1 0 1 01 0 1 0 1

].



Augmenting. Increase datalength k while keeping n fixed byreducing the redundancy r = n− k. The result is a code which hask increased by the same amount as r = n − k is decreased.Because of the reduction of r , the minimum distance generallydecreases. Example: Code #7 augmented by 1 gives a (6, 4, 2)code with generator and parity check matrices

G =

1 0 0 1 1 00 1 0 1 0 10 0 1 0 1 10 0 0 1 1 1

, H =

[0 1 1 1 1 01 0 1 1 0 1

].



Expurgating. Decrease datalength k while keeping n fixed byincreasing the redundancy r = n − k. The resulting code has kdecreased by the same amount as r = n − k is increased. Theincrease in r may or may not lead to an increase in dmin. Example:Code #7 expurgated by 1 gives a (6, 2, 4) code with generator andparity check matrices

G =

[1 0 1 1 0 10 1 1 1 1 0

], H =

1 1 1 0 0 01 1 0 1 0 01 0 1 0 1 00 1 1 0 0 1

.



Definition: |u|u + v|-construction. Let u = (u0, u1, . . . , un−1) andv = (v0, v1, . . . , vn−1) be two q-ary n-tuples and define

|u|u + v| = (u0, u1, . . . , un−1, u0 + v0, u1 + v1, . . . un−1 + vn−1) ,

where the addition is modulo q addition.

Let C1 be a q-ary linear (n, k1, dmin = d1) code and let C2 be aq-ary linear (n, k2, dmin = d2) code. A new q-ary code C of length2n is then defined by

C ={|u|u + v| : u ∈ C1, v ∈ C2

}.

The generator matrix of the (2n, k1 + k2) code C is

G =

[G1 G1

0 G2

],

where 0 is a k2 × n all-zero matrix, G1 is the generator matrix ofC1 and G2 is the generator matrix of C2.



Theorem: The minimum distance of the code C obtained from the|u|u + v|-construction is

dmin(C) = min{2d1, d2} .

Proof: Let x = |u|u + v| and y = |u′|u′ + v′| be two distinctcodewords of C. Then

d(x, y) = w(u− u′) + w(u− u′ + v − v′) ,

where d(., .) denotes Hamming distance and w(.) denotesHamming weight.

Case (i): v = v′.

Case (ii): v 6= v′.


ECEN 5682 Theory and Practice of Error Control …ecee.colorado.edu/~mathys/ecen5682/slides/block99.pdfPeter Mathys ECEN 5682 Theory and Practice of Error Control Codes. Introduction

Documents