presentation

Distributed Arg-max Computation

Jie [email protected]

John MacLaren [email protected]

Adaptive Signal Processing and Information Theory GroupDepartment of Electrical and Computer Engineering

Drexel University, Philadelphia, PA 19104

This research has been supported by the Air Force Research Laboratoryunder agreement number FA9550-12-1-0086.

October 14th, 2015

Jie Ren (Drexel ASPITRG) DAC October 14th, 2015 1 / 12

Introduction

Outline

1 Introduction

2 Lossless One-way

3 Lossless Interactive

4 Compute Two-way Interactive Rate-regions

Jie Ren (Drexel ASPITRG) DAC October 14th, 2015 2 / 12

Introduction

Motivation – Resource Allocation in LTE

MS 1

MS 2BS

Encoder

Encoder

Decoder

Subband index

Usersubband

gain1 2 3

Subband index

Usersubband

gain1 2 3

X(1)1 , . . . , X

(M)1

X(1)2 , . . . , X

(M)2

S1

Subband index

Usersubband

gain1 2 3

Z(1), . . . , Z(M)

Z(j) = argmaxn

X(j)1 , X

(j)2

o

Z = g(X1,X2)

Z = f(S1, S2)S2

Jie Ren (Drexel ASPITRG) DAC October 14th, 2015 3 / 12

Introduction

Problem Model

Z 2 g(XS1 , . . . , XS

N )

limS!1

P (S)e = 0

R1

R2

RN

Enc1

Dec

XS2

XSN

XS1

Enc2

EncN

• Channel capacity: modeled asdiscrete i.i.d. sources

• Assume rateless datatransmission

• The CEO needs to compute{i |Xi = max{Xi : i ∈ [N]}}

• Distortion Measure

dA((X1,s , . . . ,XN,s), ZA(s)) =

{0 if ZA ∈ ZA

ZM(s)− XZA(s),sotherwise

(1)

• Distributed Lossless Computation

E[d((X1, . . . ,XN), Z)

]= 0 (2)

Jie Ren (Drexel ASPITRG) DAC October 14th, 2015 4 / 12

Lossless One-way

Outline

1 Introduction

2 Lossless One-way

3 Lossless Interactive

4 Compute Two-way Interactive Rate-regions

Jie Ren (Drexel ASPITRG) DAC October 14th, 2015 5 / 12

Lossless One-way

Lossless One-way Results

• Candidate arg-max functions

RA = minfN∈FA,N

N∑

n=1

mincn∈C(Gn(fN))

H(cn(Xn)) (3)

• Achievability: build f ∗N recursively, graph coloring

• Converse: graph entropy

• i.i.d. sources: do not need the OR-product graph

Jie Ren (Drexel ASPITRG) DAC October 14th, 2015 6 / 12

Lossless Interactive

Outline

1 Introduction

2 Lossless One-way

3 Lossless Interactive

4 Compute Two-way Interactive Rate-regions

Jie Ren (Drexel ASPITRG) DAC October 14th, 2015 7 / 12

Lossless Interactive

Problem Model

3 dB

2 dB

2 dB

Ut(�t = 3dB)

X1

X2

X3

V 1t = 1

V 2t = 0

V 3t = 0

Notations

• Xi ∈ Xt = {at , . . . , bt}• Ut Broadcasting message at

round t

• V it Replied message from MS i

at round t

Achievable Interaction Scheme

1: CEO broadcasts a threshold λtat round t

2: User i replies a 1 if Xi ≥ λt and0 otherwise

3: Stops when CEO knows arg-maxreliably

Jie Ren (Drexel ASPITRG) DAC October 14th, 2015 8 / 12

Lossless Interactive

Analysis

Aggregate rate

Rt(λ) = H(λ|λ1, · · · , λt−1) + Nt + (Ft(λ))NtR∗(Nt , at , λ)

+Nt∑

i=1

(1− Ft(λ))iFt(λ)

Nt−i Nt !

i !(Nt − i)!R∗(i , λ, bt) (4)

Policy Iteration

λ∗t = argminλ

Rt(λ) (5)

Jie Ren (Drexel ASPITRG) DAC October 14th, 2015 9 / 12

Compute Two-way Interactive Rate-regions

Outline

1 Introduction

2 Lossless One-way

3 Lossless Interactive

4 Compute Two-way Interactive Rate-regions

Jie Ren (Drexel ASPITRG) DAC October 14th, 2015 10 / 12

Compute Two-way Interactive Rate-regions

BackgroundInteractive Communication

User A User B

X Y

U, Ry

V, Rx

f(x, y)

{(rx, ry) :rx ≥ I(V ; X|U, Y ) ry ≥ I(U ; Y |X)

where U − Y − X and V − (U, X) − Y

with E[d(f(x, y), φ(v, y))] ≤ D}

• Interaction for Lossy SourceReproduction (Kaspi 1985)

• Two-way Interaction FunctionComputation (Orlitsky & Roche2001)

• Interaction for functioncomputation (Ishwar & Ma2011)

Jie Ren (Drexel ASPITRG) DAC October 14th, 2015 11 / 12

Compute Two-way Interactive Rate-regions

Numerically Compute Rate Regions

BA for user 1

BA for user 2

Update Estimator

Converge?

Init

Update p(u|y)

Update p(u)

Converge

Update p(v|x,u)

Update p(v)

Converge

• Communication order matters

• Interested in minimum sum rate

• Apply Blahut-ArimotoAlgorithm

• Alternating optimization

• Includes Markov chainconstraints

Jie Ren (Drexel ASPITRG) DAC October 14th, 2015 12 / 12