Lattices for Distributed Source Coding - Reconstruction of a Linear function of Jointly Gaussian Sources -D. Krithivasan and S. Sandeep Pradhan - University.

Lattices for Distributed Source Coding -

Reconstruction of a Linear function of Jointly Gaussian

Sources-D. Krithivasan and S. Sandeep Pradhan

- University of Michigan, Ann Arbor

Presentation Overview

• Problem Formulation• A Straightforward coding scheme using

random coding• A new coding scheme using lattice coding

– Motivation for the coding scheme– An overview of lattices and lattice coding– A lattice based coding scheme

• Results and Extensions• Conclusion

Problem Formulation• Source (X1,X2) is a bivariate Gaussian.

• Encoders observe X1,X2 separately and quantize them at rates R1,R2

• Decoder interested in a linear function Z , X1-cX2

• Lossy reconstruction to within mean square distortion D

• Objective: Achievable rates (R1,R2) at distortion D

X 1;X 2 » N (0;1);E(X 1X 2) =½>0

Pictorial Representation

• Reconstruct , the best estimate of Z = F(X,Y) Z

Features of the proposed scheme

• Multi-terminal source coding problem.• Random coding techniques typically

used to tackle such problems.• Our approach based on “structured”

lattice codes.– Previously, structured codes used only to

achieve known bounds with low complexity.– Our coding scheme relies critically on the

structure of the code.– Structured codes give performance gains not

attainable by random codes.

Berger-Tung based Coding Scheme

• Encoders: Quantize X1 to W1, X2 to W2. Transmit W1 and W2

• Decoder: Reconstruct • Can use Gaussian test channels for

P(W1 j X1) and P(W2 j X2) to derive achievable rates and distortion

• Known to be optimal for c<0 with Gaussian test channels

Z , E(Z j W1;W2)

Motivation for Our Coding Scheme

• Korner and Marton considered lossless coding of Z = X1 © X2

• Coding scheme if the encoding is centralized?– Compute Z = X1 © X2. Compress it to f(Z) and transmit.

– f(¢) is any good source coding scheme.

• Suppose f(¢) distributes over ©?– Compress X1, X2 as f(X1) and f(X2).

– Decoder computes f(X1) © f(X2) = f(X1 © X2) = f(Z).

• No difference between centralized and distributed coding.

• Choosing f(¢) as a linear code will work.

An Overview of Lattices

• An n dimensional lattice ¤ – collection of integer combinations of columns of a n £ n generator matrix G.

• Nearest neighbor quantizer

• Quantization error : x mod ¤ , x – Q¤(x)

• Voronoi region • Second moment ¾2(¤) of lattice ¤ – expected value

per dimension of a random vector uniformly distributed in

• Normalized second moment G(¤) = ¾2(¤) / V2/n`

Q¤ (x) , f ¸ 2 ¤ : k x ¡ ¸ k· k x ¡ ¸0k 8¸

02 ¤g

V0(¤) = fx: Q¤ (x) = 0ng

V0(¤)

Introduction to Lattice Codes

• Lattice code – a subset of the lattice points are used as codewords.

• Have been used for both source and channel coding problems.

• Various notions of goodness:– Good source D-code if log (2¼eG(¤)) · ² and ¾2(¤)=D

– Good channel ¾2Z(¤) if it achieves the Poltyrev

exponent on the unconstrained AWGN channel of variance

• Why Lattice codes?– Lattices achieve the Gaussian rate distortion

bound– Lattice encoding distributes over Z = X1 – c X2

Nested Lattice Codes

• (¤1, ¤2) is a nested lattice if ¤1 ½ ¤2

• Nested lattice codes widely used in multiterminal communications.

• Need nested lattice codes such that ¤1 and ¤2 are good source and channel codes.

• Such nested lattices termed “good nested lattice codes”.

• Known that such nested lattices do exist in sufficiently high dimension.

The Coding Scheme• Good nested lattice code (¤11, ¤12, ¤2), ¤2 ½ ¤11, ¤12

• Dithers • Note: second encoder scales the input before

encoding.

Ui » Unif V0(¤1i ); i = 1;2

The Coding Scheme contd.

• Lattice parameters

• Rate of a nested lattice (¤1,¤2) is

• Encoder rates are

• Sum rate R1 + R2 = log achievable at distortion D.

¾2(¤11) = q1;¾2(¤12) =D¾2Z¾2Z ¡ D

¡ q1;¾2(¤2) =¾4Z

¾2Z ¡ D

2¾2ZD

R = 12 log

¾2(¤ 2)¾2(¤ 1)

R1 = 12 log

¾4Zq1(¾2Z ¡ D ) ;R2 = 1

2 log¾4Z

D¾2Z ¡ q1(¾2Z ¡ D )

The Coding Theorem

• The set of all rate-distortion tuples (R1,R2,D) that satisfy

are achievable.

2¡ 2R 1 +2¡ 2R 2 ·³¾2ZD

´ ¡ 1

Proof Outline• Key Idea: Distributive property of lattice mod

operation

• Using this, one of the mod-¤2 can be removed from the signal path.

• Simplified but equivalent coding scheme is

((x mod ¤) +y) mod ¤ = (x+y) mod ¤

Proof Outline contd.

• eq1, eq2

- subtractive dither quantization noises

independent of the sources.• Decoder operation

• But ¤2 is a good ¾4Z/(¾2

Z-D) channel code.

• Implies ((Z+eq) mod ¤2) = (Z+eq) with high probability

• Decoder reconstruction

• Can be checked that

Z =³¾2Z ¡ D¾2Z

´((Z +eq) mod ¤2)

Z =³¾2Z ¡ D )¾2Z

´(Z +eq)

E(Z ¡ Z)2 =D

Comments about the Coding Scheme

• Larger rate region than the random coding scheme for some source statistics and some range of D.

• Coding scheme is lattice vector quantization followed by “correlated” lattice-structured binning.

• Previously lattice coding used only to achieve known performance bounds.

• Our theorem – first instance of lattice coding being central to improving the rate region.

Extensions to more than Two Sources

• Reconstructing – Some sources coded together using the

“correlated” binning strategy of the lattice coding scheme.

– Others coded independently using the Berger-Tung coding scheme.

– Decoder has some side information – previously decoded sources.

– Possible to present a unified rate region combining all such strategies.

Z ,P K

i=1 ciX i

Comparison of the Rate Regions• Compare sum rates for ½ = 0.8 and c = 0.8

• Lattice based scheme – lower sum rates for small distortions.• Time sharing between the two schemes – Better rate region

than either scheme alone.

Range of values for Lower Sum Rate• Shows where (½,c) should lie for lattice sum rate to be

lower than Berger-Tung sum rate for some distorion D.• Contour marked R – lattice sum rate lower by R units• Improvement only for c > 0.

Conclusion

• Considered lossy reconstruction of a function of the sources in a distributed setting.

• Presented a coding scheme of vector quantization followed by “correlated” binning using lattices.

• Improves upon the rate region of the natural random coding scheme.

• Currently working on extending the scheme to discrete sources and arbitrary functions.