Fixed Block-Length coding for Joint Source-Channel ...

Fixed Block-Length coding for Joint Source-ChannelCommunication

Arun Padakandla

August 27, 2019

Setting : Transmitting Correlated sources over 2−user MAC and IC

S1

S2

WS1S2

~~

S1S2

X1

X2

2-MAC

WY|X1X2

Y

Setting : Transmitting Correlated sources over 2−user MAC and IC

S1

S2

WS1S2

~~

X1

X2

Y1

2-IC

WY1Y2|X1X2

Y2

S1

S2

Problem Setup : Transmitting Correlated sources over 2−user MAC and IC

Shannon-theoretic study

?? Optimal coding technique ??

?? Necessary and sufficient conditions (in terms of WS1S2 ,WY |X1X2(MAC)

WY1Y2|X1X2(IC)) ??

Plain Vanilla Lossless source-channel coding over 2-user MAC or IC.

1. A new coding technique.

2. Characterize performance, derive new sufficient conditions.

?? What is the New idea ??





WY1Y2|X1X2(IC)) ??









WY1Y2|X1X2(IC)) ??





What are the tasks involved?

1. Source Compression/Coding (Lossless)

2. Channel Coding (Over MAC/IC)

3. Co-ordinate channel inputs

Separation sub-optimal

Slepian-Wolf bin indices independent. Disables co-ordination.

What are the tasks involved?



3. Co-ordinate channel inputs Separation sub-optimal

Slepian-Wolf bin indices independent. Disables co-ordination.

Co-ordination can be critical to communication : Toy Example

Source correlation has to be extracted for co-ordination.

Three Tasks - What is the block-length (B-L) of the codes?








05284649Source

Compression

05284649

WY|X

ChannelCoding Codeword





052846494634957335764Source

Compression

052846494634957335764

WY|X






05284649463495733576476723638 …....Source

Compression

Efficiency gets better and better ….

05284649463495733576476723638 …....

Efficiency gets better and better ….

WY|X






Large block-length is good

Large block-length is good

?? block-length ??

Co-ordinating Channel Inputs is by Extracting Correlation from Distributed Sources.

?? optimal block-length for extracting correlation from distributed sources ??

Witsenhausen [1975] Problem Setup

X1X2...Xn...

Y1Y2...Yn...

IID PXY~

~

supfn,gn

=: P(n)

Best fn, gn for 1−bit agreement

.

P(n) = Best Agreement probability for n symbol processing.

P(1) > P(2) > P(3) > · · · · · ·


X1X2...Xn...

Y1Y2...Yn...

IID PXY~

~

Agreement on a non-trivial RVIs a measure of correlation.

supfn,gn

=: P(n)


.


P(1) > P(2) > P(3) > · · · · · ·


X1X2...Xn...

Y1Y2...Yn...

IID PXY

~~

fn X( 1X2...Xn ) = A ∈ {0,1}

gn Y( 1Y2...Yn ) = B ∈ {0,1}

supfn,gn

=: P(n)


.


P(1) > P(2) > P(3) > · · · · · ·


X1X2...Xn...

Y1Y2...Yn...

IID PXY

~~

fn X( 1X2...Xn ) = A ∈ {0,1}

gn Y( 1Y2...Yn ) = B ∈ {0,1}

Prob. of Agreement P (A B= )Is a measure of correlation.

supfn,gn

Binary RVs A and B non-trivial and agree with high prob.

=: P(n)


.


P(1) > P(2) > P(3) > · · · · · ·


X n

Y n

{0,1}fn

gn

X1X2...Xn...

Y1Y2...Yn...

IID PXY~

~

{0,1}

supfn,gn

=: P(n)


.


P(1) > P(2) > P(3) > · · · · · ·


X n

Y n

{0,1}fn

gn

X1X2...Xn...

Y1Y2...Yn...

IID PXY~

~

{0,1}

supfn,gn

P (fn(Xn) = gn(Y

n))

=: P(n)


.


P(1) > P(2) > P(3) > · · · · · ·


X n

Y n

{0,1}fn

gn

X1X2...Xn...

Y1Y2...Yn...

IID PXY~

~

{0,1}

supfn,gn

P (fn(Xn) = gn(Y

n))

=: P(n)

Best fn, gn for 1−bit agreement.


P(1) > P(2) > P(3) > · · · · · ·


X n

Y n

{0,1}fn

gn

X1X2...Xn...

Y1Y2...Yn...

IID PXY~

~

{0,1}

supfn,gn

P (fn(Xn) = gn(Y

n)) =: P(n)

Best fn, gn for 1−bit agreement.


P(1) > P(2) > P(3) > · · · · · ·


X n

Y n

{0,1}fn

gn

X1X2...Xn...

Y1Y2...Yn...

IID PXY~

~

{0,1}

supfn,gn

P (fn(Xn) = gn(Y

n)) =: P(n)


.


P(1) > P(2) > P(3) > · · · · · ·

Extracting Correlation favors Short Block-Lengths!!!

Indeed, · · · a binary symmetric source · · ·

0101001010

0101001010

S1

S2

pS1S2

0.495 0.005

0.005 0.495

0 1

0

1

IID Source S1, S2 with P (S1 = S2) = 0.99

A shorter block agrees with higher prob.

P (Sn1 6= Sn2 ) = 1− P (Sn1 = Sn2 ) = 1− (P (S1 = S2))n = 1− 0.99n →

n→∞1


010100101001010

010100101001000

S1

S2

pS1S2

0.495 0.005

0.005 0.495

0 1

0

1




n→∞1


01010010100101010100101110

01010010100100010100101010

S1

S2

pS1S2

0.495 0.005

0.005 0.495

0 1

0

1




n→∞1


010100101001010101001011101100

010100101001000101001010101110

S1

S2

pS1S2

0.495 0.005

0.005 0.495

0 1

0

1




n→∞1


010100101001010101001011101100 ...

One among the exponentially large conditional typical seqeuences

S1

S2

pS1S2

0.495 0.005

0.005 0.495

0 1

0

1

As n → ∞




n→∞1


010100101001010101001011101100 ...

One among the exponentially large conditional typical seqeuences

S1

S2

pS1S2

0.495 0.005

0.005 0.495

0 1

0

1

As n → ∞




n→∞1

Communicating Distributed Correlated Sources over Networks

S1

S2

WS1S2

~~

S1S2

X1

X2

2-MAC

WY|X1X2

Y


S1

S2

WS1S2

~~

S1S2

X1

X2

2-MAC

WY|X1X2

Y

01010010100100101010101101100011100101101001010

01010010100100101000101101100011100101101001010


S1

S2

WS1S2

~~

X1

X2

2-MAC

WY|X1X2

01010010100100101010101101100011100101101001010

01010010100100101000101101100011100101101001010

01010101010101111000010101010100100

01110100101010101010010101010011110

Source – Chnl Mapping

Source – Chnl

Mapping


S1

S2

WS1S2

~~

S1S2

X1

X2

2-MAC

WY|X1X2

Y

01010010100100101010101101100011100101101001010

01010010100100101000101101100011100101101001010


S1

S2

WS1S2

~~

X1

X2

2-MAC

WY|X1X2

01010010100

01010010100

0111010


Source – Chnl

Mapping

0111010


S1

S2

WS1S2

~~

X1

X2

2-MAC

WY|X1X2

01010010100100101010101101100011100101101001010

01010010100100101000101101100011100101101001010

0101010

0111010


Source – Chnl

Mapping


S1

S2

WS1S2

~~

X1

X2

2-MAC

WY|X1X2

01010010100100101010101101100011100101101001010

01010010100100101000101101100011100101101001010

0111011

0111011

Source – Chnl

Mapping

Sourc

e –

Chnl

Map

ping


S1

S2

WS1S2

~~

X1

X2

2-MAC

WY|X1X2

01010010100100101010101101100011100101101001010

01010010100100101000101101100011100101101001010

10101101

10101101

Shorter Blocks

⇓

Efficient transfer of correlation.

Longer Block

⇓

Efficient Chnl Coding,Source Compression.

A tension in choice of Block-Length (B-L)

?? Coding scheme ?? Info-theoretic analysis ?? Single-lettercharacterization.

Tension in choice of Block-Length (B-L)

Optimal B-L neither too big nor too small

Fixed Block-Length (B-L) coding

A New approach in information theory.

Challenges in the Design and Information-theoretic Analysis

1. Design of Information-theretic coding scheme with shortblock-lengths.

2. Information - theoretic performance analysis requires block-length(B-L) of overall coding scheme to →∞.

3. Fixing the B-L results in multi-letter distribution. How do you get asingle-letter characterization?

?? Design ?? Analysis ?? Performance Characterization ??

Two-layer coding - one fixed B-L, one ∞− B-L

B-L ml.

0101001010010010101010110110001110010110……..1001010

Source 11 ml


B-L ml.

0101001010010010101010110110001110010110……..1001010

010100101001001010101011011000111001011010

0101001

Source 11 ml

m

1

1 l


B-L ml.

0101001010010010101010110110001110010110……..1001010

010100101001001010101011011000111001011010

0101001

Source 11 ml

m

1

1 l

Layer 1 code

Maps row-by row

Fixed B-L l

101010101011011101011111111100000001011010

0101001


B-L ml.

0101001010010010101010110110001110010110……..1001010

Source 11 ml

101011111101110001011111111111010111101000

0101001

Layer 2 codeBlock mapping

B-L ml


B-L ml.

0101001010010010101010110110001110010110……..1001010

010100101001001010101011011000111001011010

0101001

Source 11 ml

m

1

1 l

Layer 1 code

Maps row-by row

Fixed B-L l

101010101011011101011111111100000001011010

0101001

101011111101110001011111111111010111101000

0101001

Layer 2 codeBlock mapping

B-L ml

101011111101110001011111111111010111101000

0101001

MultiplexingTwo layers intoChannel inputs


1. Fixed B-L Layer 1 transfers correlation efficiently

2. ∞− B-L Layer 2 performs source compression, channel coding.

Challenges in Information-theoretic Performance Analysis

1. Multiplexing multiple codewrds of Layer 1 with single codewrd of Layer 2.

I Shirani and Pradhan’s technique of interleaving.

2. Distribution induced by the coding scheme not the same as the chosentest channel

I Constant composition codes and bounding the divergence between therespective distributions.

Fixed Block-Length Coding

A New Framework in Information Theory

Builds on early findings of Dueck 1981, Witsenhaunsen 1975.

Builds on recent findings by Shirani Pradhan [2014] on Distributed SourceCoding.


New Coding Structure for

that strictly outperform all previous known.

Derived new sufficient conditions.

New Results on a Fundamental problem. Remained Stubborn since 1981.

Thank You

A new coding theorem in a simplified form

Theorem(S,WS) is transmissible over IC (X ,Y,WY |X) if there exists (i) a finite set K,maps fj : Sj → K, with Kj = fj(Sj) for j ∈ [2], (ii) l ∈ N, δ > 0, (iii) finite setU ,V1,V2 and pmf pUpV1pV2pX1|UV1

pX2|UV2defined on U × V × X , where pU

is a type of sequences in U l, (iv) A,B ≥ 0, ρ ∈ (0, A) such that φ ∈ [0, 0.5),

A+B ≥ (1 + δ)H(K1), and for j ∈ [2],

B +H(Sj |K1) + LSl (φ, |Sj |) < I(Vj ;Yj)− LCj (φ, |V|),

where,

φ:= gρ,l + ξ[l](K) + τl,δ(K1), ξ[l](K):= P (Kl

1 6= Kl2)

gρ,l:=

2∑j=1

exp{−l(Er(A+ ρ, pU , pYj |U )− ρ)}, (Channel coding error)

τl,δ(K) = 2|K| exp{−2δ2p2K(a∗)l} (Prob of non-typicality)

LCj (φ, |U|) = hb(φ) + φ log |U|+ |Y||U|φ log 1

φ,

LSl (φ, |K|):=1

lhb(φ) + φ log |K|.





A+B ≥ (1 + δ)H(K1), and for j ∈ [2],


where, φ:= gρ,l + ξ[l](K) + τl,δ(K1), ξ[l](K):= P (Kl

1 6= Kl2)

gρ,l:=

2∑j=1




φ,

LSl (φ, |K|):=1






WY1Y2|X1X2(IC)) ??








WY1Y2|X1X2(IC)) ??








WY1Y2|X1X2(IC)) ??




Central element of both problems

? Optimally transferring source correlation onto co-ordinated channel inputs ?

Prior work : Cover, El Gamal and Salehi (CES) [Nov. 1980] technique.Dueck’s example

[Nov. 1980](S1, S2) transmissible over MAC if

H(S1|S2) < I(X1;Y |X2, S2, U), H(S2|S1) < I(X2;Y |X1, S1, U)

H(S1, S2|K) < I(X1X2;Y |U), H(S1, S2) < I(X1X2;Y )

for a valid pmf WS1S2pUpX1|US1pX2|US2

WY |X1X2.

Prior work : Liu and Chen [Dec. 2011]

Incorporated

1. CES technique

2. Random source Partitioning [Han Costa 1987]

3. Message Splitting Via super position coding [Han-Kobayashi 1981]

into a single coding (LC) technique and derived sufficient (LC) conditions.

CES and LC techniques : A common thread/constraint

X1 − S1 − S2 −X2

X1 = g1(S1,W1), X2 = g2(S2,W2). W1 ⊥W2.

CES and LC conditions are sub-optimal

MAC : Single-letter CES technique are (in general) sub-optimal. Dueck [Mar,1981] An ingenious coding technique designed for a particular example.

IC : LC conditions are sub-optimal. (Adapt Dueck’s argument).

This talk :

Understand why CES and LC are sub-optimal via Dueck’s example and developa new coding technique.

Begin with a very simple MAC example

S1, S2 ≡ Sources. Source alphabet = {0, 1, · · · , J}.

Channel inputs ≡ X1, X2 ∈ {0, 1, · · · , L− 1}

Channel output ≡ Y ∈ {0, 1, · · · , L− 1, ∗} .

Begin with a very simple exampleSource alphabet = {0, 1, · · · , J}. Source PMF

S1 = S2 w.p 1.

H(S1) = hb(ε) + ε log J

MAC Channel

Above source, is transmissible as long as

H(S1) = H(S2) ≤ suppX1X2

I(X1X2;Y ) (1)

ACHIEVABLE (CO-ORDINATED) INPUT PMF IS UNCONSTRAINED BYSOURCE PMF.

Can Squeeze ε, blow up J , hold H(S1). Source is transmissible if (??) holds.


S1 = S2 w.p 1. H(S1) = hb(ε) + ε log J

MAC Channel



I(X1X2;Y ) (1)




S1 = S2 w.p 1. H(S1) = hb(ε) + ε log J

MAC Channel



I(X1X2;Y ) (1)




S1 = S2 w.p 1. H(S1) = hb(ε) + ε log J

MAC Channel



I(X1X2;Y ) (1)




S1 = S2 w.p 1. H(S1) = hb(ε) + ε log J

MAC Channel

Source is transmissible as long as H(S1) = H(S2) ≤ logL.



I(X1X2;Y ) (1)




S1 = S2 w.p 1. H(S1) = hb(ε) + ε log J

MAC Channel



I(X1X2;Y ) (1)




S1 = S2 w.p 1. H(S1) = hb(ε) + ε log J

MAC Channel



I(X1X2;Y ) (1)



Gacs Korner part facilitates co-ordination and efficient communication

Gacs Korner part provides considerable flexibility to co-ordinateand communicate efficiently.

H(S1) = logL

Need X1 = X2 uniform and X1 = X2.

For small ε, source highly non-uniform.

What if we tweak the source just a little?

MAC Channel remains same.


Source stripped off its Gacs-Korner part.

Single-Letter (S-L) coding is constrained to

X1 − S1 − S2 −X2.

X1 = g1(S1,W1), X2 = g2(S2,W2). W1 ⊥W2.






X1 − S1 − S2 −X2.

X1 = g1(S1,W1), X2 = g2(S2,W2). W1 ⊥W2.






X1 − S1 − S2 −X2.

X1 = g1(S1,W1), X2 = g2(S2,W2). W1 ⊥W2.






X1 − S1 − S2 −X2.

X1 = g1(S1,W1), X2 = g2(S2,W2). W1 ⊥W2.






X1 − S1 − S2 −X2.

X1 = g1(S1,W1), X2 = g2(S2,W2). W1 ⊥W2.




X1 = g1(S1,W1), X2 = g2(S2,W2). W1 ⊥W2.

W1 and/or W2 non-trivial RVs reduces P (X1 = X2).



Need X1 = X2 uniform andX1 = X2.

X1 = g1(S1, φ), X2 = g2(S2, φ).

?Trivializing W1,W2 and requiring X1 = X2 uniform?

Pool all less likely symbols together.



Need X1 = X2 uniform andX1 = X2.

X1 = g1(S1, φ), X2 = g2(S2, φ).




X1 = g1(S1, φ), X2 = g2(S2, φ).



Can get only (1− ε, ε)


Carefully choose values for ε, θ, J, L, Dueck [1981] proves

1. Single-letter technique of CES [1980] is sub-optimal.

2. Proposes mapping fixed length blocks of source to channel codewords.Proves S1, S2 is transmissible.

Mimic conditional coding via fixed block-length coding

When S1 = S2.

Map large (typical) source sequences to codewordschosen uniformly at random.

We can ensure

X1 = X2 uniform and X1 = X2.

Mimic conditional coding via fixed block-length coding

P (S1 6= S2) = Jθ > 0.

P (Sn1 6= Sn2 ) = 1− (1− Jθ)n −→n→∞

1.

With increasing block-length n, P (Xn1 6= Xn

2 )→ 1.

Conditional coding with FIXED BLOCK-LENGTH l.

P (Sl1 6= Sl2) = 1− (1− Jθ)l ≤ lJθ.

Fixed block-length coding forces a concatenated coding scheme

Source PMF

MAC Channel

Block-length of coding is determined by ε, θ, J, L, not be the desired prob. oferror.

This fixed block-length code results in non-vanishing probability of error.

Necessitates an outer code of arbitrarily large block-length to

1. Correct for errors in the fixed block-length code.

2. Communicate rest of information.

This results in concatenated coding structure.

Part 2 and 3

Concatenated coding scheme for a general problem instance whoseinfo-theoretic performance can be characterized?

What challenges does the finite block-length code throw up?

Revisit CES scheme with common part K

Two phase coding, in the presence of GK common part K.

Phase 1: Common part Kn is decoded perfectly (owing to infinite block-length)

Phase 2: Rest of Information : Sn1 , Sn2 conditioned on common part Kn is

encoded in Xn1 , X

n2 and decoded by Rx.

Kn can be viewed as side-information available at all terminals.


Revisit CES scheme with common part K

Two phase coding, in the presence of GK common part K.

Phase 1: Common part Kn is decoded perfectly (owing to infinite block-length)

Phase 2: Rest of Information : Sn1 , Sn2 conditioned on common part Kn is

encoded in Xn1 , X

n2 and decoded by Rx.

Kn can be viewed as side-information available at all terminals.


Suppose S1, S2 do NOT have common part, but highly correlated.

m× l matrix encoding. l remains fixed, m→∞.

Two phase coding.

Phase 1: Rows of Km×l1 (Km×l

2 ) mapped to Um×l1 (Um×l2 ) via finiteblock-length code. Decoded erroneously as V m×l.

Phase 2: Rest of Information : Sm×l1 , Sm×l2 conditioned on common partV m×l is encoded in Xm×l

1 , Xm×l2 and decoded by Rx.


Suppose S1, S2 do NOT have common part, but highly correlated.

m× l matrix encoding. l remains fixed, m→∞.

Two phase coding.

Phase 1: Rows of Km×l1 (Km×l

2 ) mapped to Um×l1 (Um×l2 ) via finiteblock-length code. Decoded erroneously as V m×l.

Phase 2: Rest of Information : Sm×l1 , Sm×l2 conditioned on common partV m×l is encoded in Xm×l

1 , Xm×l2 and decoded by Rx.

Multi-letter distribution induced by the fixed block-length code

Each row of matrices Um×l1 , Um×l2 , V m×l, Gm×l have a multi-letter pmf. Howdo you perform superposition coding?

Leverage the technique of interleaving suggested in Shirani and Pradhan [ISIT2014].

Note that the inner code of finite block-length is fixed upfront.

As a consequence, the rows of each of the m× l matrices are iid. However,each row has a product distribution.

What is Interleaving?

Interleaving [Shirani and Pradhan 2014]



The same argument holds for each sub-vectorsE(1, λ1(i)) · · ·E(m,λm(i)) : i = 1, · · · , l.

We therefore have l iid sub-vectors E(1, λ1(i)) · · ·E(m,λm(i)) : i = 1, · · · , l.

Independent streams with Gacs-Korner common part

Theorem(S,WS) is transmissible over IC (X ,Y,WY |X) if there exists (i) a finite set K,maps fj : Sj → K, with Kj = fj(Sj) for j ∈ [2] withP (K1 = K2) = φ+ ξ[l] = 0, (i) finite set U ,V1,V2 and pmfpUpV1pV2pX1|UV1

pX2|UV2defined on U × V × X , (iii) A,B ≥ 0,

A+B ≥ H(K1) = H(K2),

A ≤ min{I(U ;Y1), I(U ;Y2)} and for j ∈ [2],

B +H(Sj |K1) < I(Vj ;Yj).





A+B ≥ (1 + δ)H(K1), and for j ∈ [2],


where,

φ:= gρ,l + ξ[l](K) + τl,δ(K1), ξ[l](K):= P (Kl

1 6= Kl2)

gρ,l:=

2∑j=1




φ,

LSl (φ, |K|):=1






A+B ≥ (1 + δ)H(K1), and for j ∈ [2],


where, φ:= gρ,l + ξ[l](K) + τl,δ(K1), ξ[l](K):= P (Kl

1 6= Kl2)

gρ,l:=

2∑j=1




φ,

LSl (φ, |K|):=1


Backup

Finite blck-lngth inner code concatenated with an large blck-lngth outercode

Let U lj be CU codeword chosen by encoder j.

P (U l1 6= U l2) ≤ P (Kl1 6= Kl

2) + P (Kl1 /∈ T lδ(K1))

≤ 1− (1− ξ)l + 2|K| exp{−2p2K1(a∗)l} =: β

Suppose g is the prob. of error of a PTP channel decoder operating over aPTP channel pY |U .

Inner block code of finite block length

Assuming encoders agree, decoder decodes into CU -codebook with a PTPdecoder on pY |U chnl.


V is decoded version of U . G(t, 1 : l) = e−1U (V (t, 1 : l)) is decoded version of

K.


Encoders and decoder agree on a fraction 1− (g+β) of the rows of U -matrices.

Outer code : The approach

Encoder 1 has Encoder 2 has

S1(1 : l) · · ·S1((m− 1)l + 1 : ml) S2(1 : l) · · ·S2((m− 1)l + 1 : ml)

K1(1 : l) · · ·K1((m− 1)l + 1 : ml) K2(1 : l) · · ·K2((m− 1)l + 1 : ml)

U1(1 : l) · · ·U1((m− 1)l + 1 : ml) U2(1 : l) · · ·U2((m− 1)l + 1 : ml)

Decoder has

G(1 : l) · · ·G((m− 1)l + 1 : ml)

V (1 : l) · · ·V ((m− 1)l + 1 : ml)

If these had a single-letter form i.e., distributed as∏mlt=1 pS1K1U1S2K2U2V G,

then one could apply standard info-theoretic techniques.

Challenges

Encoder 1 has Encoder 2 has

S1(1 : l) · · ·S1((m− 1)l + 1 : ml) S2(1 : l) · · ·S2((m− 1)l + 1 : ml)

K1(1 : l) · · ·K1((m− 1)l + 1 : ml) K2(1 : l) · · ·K2((m− 1)l + 1 : ml)

U1(1 : l) · · ·U1((m− 1)l + 1 : ml) U2(1 : l) · · ·U2((m− 1)l + 1 : ml)

Decoder has

G(1 : l) · · ·G((m− 1)l + 1 : ml)

V (1 : l) · · ·V ((m− 1)l + 1 : ml)

I Not guaranteed to have a iid form.

I Even if we can extract sub-vectors that have product iid form, we do notknow the underlying pmf. Hence, cannot express rates in terms of thesepmfs.

The rows are independent and identically distributed

The rows are independent.

And identically distributed.


The rows are independent.And identically distributed.


The rows are independent and identically distributed.



Randomly chosen co-ordinates are iid!!!

For each row t, choose column πt(1), where π1, · · · , πm are randompermutations.


For each row t, choose column πt(1), where π1, · · · , πm are randompermutations.



We therefore have l iid sub-vectors E(1, π1(i)) · · ·E(m,πm(i)) : i = 1, · · · , l.


Co-ordinates with same color coded together within a block in the outer code.l such blocks.







The induced single-letter distributions are unknown!!!

S1(1, π1(1))S1(2, π2(1)) · · ·S1(m,πm(1))

K1(1, π1(1))K1(2, π2(1)) · · ·K1(m,πm(1))

U1(1, π1(1))U1(2, π2(1)) · · ·U1(m,πm(1))

S2(1, π1(1))S2(2, π2(1)) · · ·S2(m,πm(1)) ∼m∏t=1

pS1K1U1S2K2U2V G

K2(1, π1(1))K2(2, π2(1)) · · ·K2(m,πm(1))

U2(1, π1(1))U2(2, π2(1)) · · ·U2(m,πm(1))

V (1, π1(1))V (2, π2(1)) · · ·V (m,πm(1))

G(1, π1(1))G(2, π2(1)) · · ·G(m,πm(1))

The sufficient conditions will look like

H(S1|S2, G) < I(X1;Y X2|V S2), H(S2|S1, G) < I(X2;Y X1|V S1) · · ·

But, we do not know this pmf.

This pmf depends on the pmf of the empiricaldistribution of the inner layer code and the errors in that layer.

The induced single-letter distributions are unknown!!!

S1(1, π1(1))S1(2, π2(1)) · · ·S1(m,πm(1))

K1(1, π1(1))K1(2, π2(1)) · · ·K1(m,πm(1))

U1(1, π1(1))U1(2, π2(1)) · · ·U1(m,πm(1))

S2(1, π1(1))S2(2, π2(1)) · · ·S2(m,πm(1)) ∼m∏t=1

pS1K1U1S2K2U2V G

K2(1, π1(1))K2(2, π2(1)) · · ·K2(m,πm(1))

U2(1, π1(1))U2(2, π2(1)) · · ·U2(m,πm(1))

V (1, π1(1))V (2, π2(1)) · · ·V (m,πm(1))

G(1, π1(1))G(2, π2(1)) · · ·G(m,πm(1))

The sufficient conditions will look like

H(S1|S2, G) < I(X1;Y X2|V S2), H(S2|S1, G) < I(X2;Y X1|V S1) · · ·

But, we do not know this pmf.This pmf depends on the pmf of the empiricaldistribution of the inner layer code and the errors in that layer.

Upperbound the difference using relationship between chosen and actualpmf

Let pUpX1|US1pX2|US2

be chosen pmf.

pUj= pU since symbols of CU chosen iid pU .

P (U1 6= U2) ≤ β

pXj |Uj Sj= pXj |USj

: By choice of coding technique. One can derive an upper

bound using these relations.

Upperbound the difference using relationship between chosen and actualpmf

Let pUpX1|US1pX2|US2

be chosen pmf.

pUj= pU since symbols of CU chosen iid pU .

P (U1 6= U2) ≤ β

pXj |Uj Sj= pXj |USj

: By choice of coding technique. One can derive an upper

bound using these relations.

Fixed Block-Length coding for Joint Source-Channel ...

Documents