(P) Baraniuk an Introduction to Compressive Sensing-libre (1)

An Introduction to Compressive Sensing

Collection Editors:

Richard Baraniuk

Mark A. Davenport

Marco F. Duarte

Chinmay Hegde

An Introduction to Compressive Sensing

Collection Editors:

Richard Baraniuk

Mark A. Davenport

Marco F. Duarte

Chinmay Hegde

Authors:

Richard Baraniuk

Mark A. Davenport

Marco F. Duarte

Chinmay Hegde

Jason Laska

Mona Sheikh

Wotao Yin

Online:< http://cnx.org/content/col11133/1.5/ >

C O N N E X I O N S

Rice University, Houston, Texas

s st rrt tt s t s rt r r r

rt r rt t s s r t rt s ttrt s

ttrtsrss

t strtr rs r

P rt tr

r rt ttrt rt r t s t ts t s

tts

trt

trt t rss ss

rs rss s

trt t tr ss

ss rs

rs rrstts

rss ss

s trs

s tr s

s ts

rstrt str rrt

P t P

trs tt sts t P

r

rs r t

rr t sr s rr

rr s

stt rts rst

rsst s trsts

rts r rs r

rs rr rts

tts ts

r rts

tr rts

s ts

ts rss s

r rrss st

rs rrr rrt

r tst t str rts

rss

trt rs

r

rstr

rss rss t

r s rss srts

rss ssr trs

ss

s

ss r rs

trt sr r sss r rs

Pr t P r sss trs

t r

ssr

r

ttrts

r r t s ttrtt

tr

trt

trt t rss ss

r t st t rt tt s r t t t s

ss ssts t rrs t rst trt t ts rt

s t r r t qst ttr s tst

t ss r rsts strt tt ss s s tr

t t rr r st r s ss t t t s qst rt

t t st rq rst t s trst t ts sr s

rss s r t t t t r t rs tt

s t rt ss rss ssts tt r r rst r

sqt r s t tr trrts

s rst ts sss t t t rt ss ssts s r r tr t

trrt rtt rtt r ts t rst qst rt s s

tt t r t ss trt t s t st r s

ss t s qr ss t t ssr rt s st trrr

s tt r t qst rss ss t rs s s

rt sr strs t ss ts t

s trs

rss t st tt s t s s

t t rss s t t st s rrstt s

tt s t trt t strt t st r tqs r s

rss s s trsr t rs ss r r tt rs

srs r rss rrstts r ss ss trst srs rrstt

tt r s t N rrst t t K N r ts rssrrstt tt t s s rt s t K r tst srs rss ss rrst t t rsr t s

ts t rst ts t s s rss s srs rt rs

t t trsr ss tt t s srst rsst t

P P P P strs

r t t trsr rss ss s r s rr

r s qst ssr s s tt r rt t s

tt sts r ss ss tt srs r rss rrstt qst

s tr stts tt rt r ss s rqr rr t rt

tr rtrr t s t t s s srs ss st r

t r srts tt t str sqt ss srs ss t

s tt s t ttrtt

r r t s ttrtt

P

t ttr t sst ss rsts s s t t rtr

t rst s t rt t rss t s t t s t

rt ss t t rss r t r s rt r t

t r s st r r s

tt ts s srs r rss rrstt rr r s

st r t srts s ts srt ss tr

tss t rt t s qst ss r t st tr s t

t ts s rt st ttrt t s rss t

tr ts ts rt t s r s t tt tr Pr rs

rt r t stt t rtrs sst t s r ts

s t rs s t trt t r s rtr

s tt st r t K sss s q tr ts t t = 0 t tr 2K ts t s rrsts r r ss t t r qstrtss K s s t r ss rqs s r t s ts r s r r rts st srst t tt t tr

tts rssr rs s s r qr rt sss

ss sst K ts t r t s s t r sss t r s ttr r rs s s r t ss tt

r r K rtrs s tt ts ss s rr r st 2Kss

rt r ss rr s r rt srt ts rr trsr r

rs t r trt ts srts t tr t tr rr trsr

s tt t s ssts t r ss t rs r rrt

rr t tr rr trsr ts t s r st r ts

rr trsr s r t t s t sst 1 r ss r tt qr rr srts rs rr rs t s t rts s

r rt s r s tt s

srs rrstt rr t r s st r t srts

s rst ssts tt t ss t ss srs ss t r r srts

t rss ss t r tt rs r ss s t rtt

rsts rst rtr t s t s t s ts t ssts t qr

srts t r r rts t t s r r tst ts

s trt ts rs tt rss t s r t s ts tst ts

t t rrs r t r t t s rr t r

rr t r s r t rss srts t qst rr

s rr s tr r s s trt r rss tt rqrs tt

tt s s trrtt

s r t t sr ts s t

r t s ss tr s tr rsr st qt

rr t r t ts rr s sr rsr tt ts t rr

rs rt tts r rs ts sqst tt

rtrs t s rss rttrs t rss ssr

trs t

s rs trs t s ts rss ss r t ts srst

t rsst t trsr r t rsts t

s rr t tr ss tr s t 1tt trt rts r srs rr t t r ts

srst sr s rss rs s s srs rrss st t rrr

rrt t r tst t rss r t s sss

ts rss ss tt rs t ss t

r r t s ttrtt

t t rstr t t ssr

trs t

ts

trs t t t r r Ptr tt t

P r tr rr s rts ts rs s r t trtr

tr

rss s r ts r rst Prss

ttsttstrrtstr

r r t s ttrtt

P

r r t s ttrtt

tr

rs rss s

trt t tr ss

r ts str s rss s s ss r s ssts

tr ssts s r s t s tr t sr s s tt

t ts r strtr s t s rrt t r s rss

ss s trs rrt tr s s trs t r strtr tt

t sr tt t ss ttr t t s

s rr tr ss s t tts ts r tr R3 s s ts

sts s t sr r ss trst s s s r ss

s r ts ss

rt ts rs

trt ss s r ts s tt r tr

ts r srt tr t r t s ssts r s ssr

tr ts rs ss tt t rr s rt rt t t ts

tr ss r r r s t ts tr ss tt

rqr t tr rss ss t r r tr r

tr ss s ts trtr rs t Prss

t r t r tr ss tr ss t r

t s srt t r ss s trs N s st R

N t trs R

N rqt s t p rs r r p [1,] s

xp = {(N

i=1 |xi|p) 1p

, p [1,) ;max

i=1,2,...,N|xi|, p =.

s s sr t str r rt RN t

< x, z >= zTx =Ni=1

xizi.

s r rt s t t 2 r x2 =< x, x > s tts t s s t t t t p rs t t s r p < 1 ts s tr s t sts t tr qt s t s t qsr s

rqt s t tt x0 := |supp (x) | r supp (x) = {i : xi 6= 0} ts t srt

s tt s t ttrtt

trt t rss s ttrtttst

t Prss ttrtttst

r r t s ttrtt

P P P

x |supp (x) | ts t rt supp (x) t tt 0 s t qsr t s s tt

limp0

x0xp = |supp (x) |,

st ts tt p qsrs t rt rrts r rt s p strt ts r s t t sr {x : xp = 1}, tsrs R

2 t tt r p < 1 t rrs t sr s rt t qsrst t tr qt

r t srs R2r t p rs t p = 1, 2, r t p qsr t p =

1

2

t sr r 1 r t sr r 2 r t sr r r t sr r

p qsr

t s rs s sr t strt s r t s rrr r

ss r s x R2 s t rt t s t s s A sr t rt rrr s p r t r ts s t t

x Att s x x p p st t t rrts t rstrt rrr s strt r t t sst t A t x s p r r p sr tr x t t trsts t A s t

t

x A tt s sst t x t rrs p r sr tt rr p ts t sr tt rrr r t t ts sr p s t rrr tt s r strt ts t srs s tt rs t r ss s rtt

r t t tr

r r t s ttrtt

r st rt t R2 s ss s t p rs

r p = 1, 2, t p qsr t p =1

2 rt 1 r rt

2 r rt r rt p qsr

ss rs

st = {i}iI s ss r ts tr s t V t trs t st s V r r t s s tt tr t s rrsts r t ts sr t t tr s st ss trs q s

rtrr t ts ts r t t r rt t s

st trs srt stts sr r ts rt ss r

V = RN I = {1, ..., N}tt s x RN rss s

x =iI

aii,

r r ts r t s ai =< x,i > {i}iI r t trs tt sttt r ss tr t t r ss ts s t rt x r r ss r stss ss s t rstrt r s s r ss ss rtr ss s s st trs = {i}iI tt r ss sts r rt t r tr rs < i, j >= 0 i 6= j trs ts t stss ss s s t rt t ss ss = T t s t s t r t t ss t r sts ss r t

trs rst t s s r r r r s st trs {i}ni=1 Rdd < n rrs t tr Rdn s tt r trs x Rd

Ax22 Tx22 Bx22 t 0 < A B < t tt t t A > 0 s tt t rs st rt A s s s t rst ss B s t sst r ts qts t t t t t r s A B s s A = B t t r s Att A = B = 1 t s Prs r r s qr tr stss > 0 s tt i2 = r i = 1, ..., N t s tr = 1 t s tt tt r s r r ts ss t s r s dN tr A B s rrs t t sst rst s T rst

s tt s t ttrtt

r r t s ttrtt

P P P

rs r rr rrstts t t tr r r s x trst t t trs s tt x = rtr r rs rt t tr r r r sts

T = T = I

s trt r rtr =(T

)1 s rrr t s t r t s s s t rPrs srs t tt s A > 0 rqrs t r t rs sr tt T s rt s tt s s tt st s ts s

d = T(T

)1x.

s tt ts sq s t sst t sq 2 r d2 2 r s tt x = t tt t srs rt t trtr t s s r ss r

r t rrr t s tr r rt tr rst t t tr ts

ts

rs rrstts

rsr s t ss r r t s t rrst s r s

rst rss s s r r t str t trsss qt

s s ts t s t s trst t ss ts t

r ss r r s st ts s rrt ss r t r

r ts s s s tt srs rrstt rs s s s

t rts rss t s rss ss t s t

tt r s s srs ss r r t rr r r s r s

r srts r srs t t r ts t str r trstt

ss t rst ss t r

tt s tt s x s Ksrs t s t st K rs x0 K t

K = {x : x0 K} t t st Ksrs ss t ss tt r t tsssrs t t srs rrstt s ss ts s st rr t x s Ksrs t t rst tt rss x s x = r 0 Krst s t s rss rt tr r tss s s

rss s sttsts r tr s t r

rtt rst s rs rt t tr sttst stt s

t t st t s sst s t

rss tss s t ts t trsr rs r srs rrstts r t

r s r sr s s ts s

s s

rst tr ss ss tt rt s srst r t s

s s r s s s rrst s r tr t ss s

r trrt s s ss s r r ts r ss r ss

s tt s t ttrtt

r r t s ttrtt

t r ts t t s tt ts s r ss ssts s

ts s t rstrt r s r srtt ss s rrstt ts r

ts t t ss rt tr r rt srts rst t

s t rr ss t tr rqrs t r ss trs s rrt

t t st t rqs s r r t r ts

r t srt s trsr t r s tr rqr s

r t r s s r qt s Xk =N1

n=0 xncos(piN

(n+ 12

)k)t k = 0, , N 1 xn t t s t t t trsr

r s s tr rrstts r ss rr ss

ss

r r t s ttrtt

P P P

s s

s s t t t ss s r st r tr tt s s

srs t str s st t s r r s s tr

s r rtr r st r ttr rs rt sr s s t ts

strtr r t r r srs rrst s ts t trsr

t trsr ssts rrs t t ts rq ts

st rq ts r rs s rt t t r rq

ts t t rs s t s t t trsr

t tr s s r s tt st ts r r s t

rt t s stt t s ts t r r trs t ts t

t Ksrs rrstt sr t rt rrr s p r ts rrs t st Ktr rt t r s t st rt t s s K ss ts

r rs rrstt ts t trsr r

t rrstt r ts r rrst t s s ts r

rrst r s sr tt st t t ts r s t r

rst rsts tr ts st s st tr s st s r tt

r tr rs rs t tt trst r rrst t rq ts

rq ts r rt tr strt tr t ts t s

t tr stts r s t tr rqr rq ts r rt

tr t tr t t t s t t trsts r r t s

t rrts t rq t t rss ss t

r

rs s t st Ktr rt s t rst t rtr ss rtrs r s st r srs rt rts ts sr tr ts rs

r r t s ttrtt

rss ss

rsst Ktr rt

rtt sst s t tt rss ss t s tt ss t

r strtr r t strtr sr s srst t t r

r s t s s rrstt rtr ss t s ssr srs t s r s rs t ts r t

strtr ss r tr srs rtr t r rss s s rss ts

srt t ts r sr ts tt t x s srss t ss

x = ,

r r t ts x t ss x s rss t t ts t srtts s sr r

|s| C1sq, s = 1, 2, .... s s rss t s ts r rr q s t str tts t r rss s s r ss s tt r rss

rt ss

s tt s t ttrtt

r r t s ttrtt

P P P

0 1 2 3 4 5 6x 104

0

0.5

1

1.5

Sorted indices

Wav

elet

coe

fficie

nt m

agni

tude

r t r t s s tt s rss s t

s r srt r rst t sst tr s sr st t r t s t

rss s t t s rss ts s ts t ts t r

s t ts tr ts s r rss ss rrst

K N ts st Ktr rt s s t t K rstts r t t t rst r rrr t t tr s tsK tr rts t t Ktr rt rrr K (x) s

K (x) = arg minK

x 2.

r r t s ttrtt

r rss ss sts t r s s

K (x) C2K1/2s. t s tt K(x)2 s K

r t srt ts i s

ir+1/2 r ss ts Ktr rt

r rs rt tr r rt

t t rst t t ts s tr s r

rss t rt ts t trs ts rst t ts

ts t

rsst p ss

ss rsst s rt t t p s t t s s t sq x (n)s t p s r rtr p ts p r s t

x p =(

i

|xi|p) 1

p

P P P

r s t p rss t s t rrs p s s rss s

s s r t t s t ss ss tr ss r t

p r s ss t r q t ts p s rss t p

s tt s s s t t x [n] rr r ts sq t p r ts ts st r rt t q > 1/p rr s s p s t p 1 s r s trr rss

r r t s ttrtt

tr

s trs

s tr s

rr t t sss r rt rstrt r ttt t t str ts

rss ss t s x RN sr srtssts tt qr M r srts rrst ts rss tt s

y = x,

r s M N tr y RM tr rrsts st rt ts R

N r N s r r t RM r M s t sr t N t tt tstr rr ss tt t srts r t tt t rs r t t rs qr srts rt stts t

srt ss t st rr s

t tt t t str rr sss tt x s tt tr t srt s s t r s rt t trst s srt ssts

r qr ts ss s s tst ss r s r s

t x s tt qstrt ss trr r t ss trt qr rss srts tt rst s t t qst rt

r r t trt qsts rst s s t ss tr tsr tt t rsrs t rt t s x rr t r s xr srts y t s r r t s srs t r rss t s tt s trs t M N tt sr tt t rr t t r s rt t s rt rt rts t

ts rt t rs

rst rss t qst t s t ss tr tr t rt rs s rr st sr r sr rrts tt

t s t t s rrt t t rstrt str rrtt r t t r s rtt s tr

strts t tt sts ts rrts

s tt s t ttrtt

trt t rss s ttrtttst

r r t s ttrtt

P

s ts

tr t sts ts t tt s ss tr t s sr t s t

N () = {z : z = 0}. s t t rr srs t ss x r t srts x t t st r tt r r stt trs x, x K = {x : x0 K} st x 6= xs trs t ss t sts x r x s s t srts y rr sr tt x = x t

(x x) = 0 t x x 2K s tt qrrsts x K N () ts trs 2K r r qt s rtr ts rrt t st s s t sr

sr

t

sr tr s t sst r s tt r r t

s t s s t s t strtrr rt

r rr

r tr y RM tr sts t st s x K s tt y = x spark () > 2KPr

rst ss tt r y RM tr sts t st s x K s tt y = x ss r t s trt tt spark () 2K s s tt tr stss st t st 2K s tt r r t tr s tt trsts h N () s tt h 2K ts s s h 2K rt h = x xr x, x K s s h N () tt

(x x) = 0 x = x tts trts r sst tt tr sts t st s x K s tt y = xrr st tt spark () > 2K ss tt spark () > 2K ss tt r s y tr st x, x K s tt

y = x = x trr tt (x x) = 0 tt h = x x rt ts s

h = 0 spark () > 2K sts t 2K s r r t trr h = 0 s tr s x = x r t tr

t s s t s tt spark () [2,M + 1] rr r rr st rqrt M 2K

s rrt

t t srs trs t sr rs t rtrt srs r

r s ss r t rt srs t ss st tr

st r rstrt ts t s s st s srtt N () s t t trs tt r t rss t t trs tt r srs rr t stt t r t t tt tt r t s trt

ts rs

s tt {1, 2, , N} s sst s t c = {1, 2, , N} \ x t t t N tr t stt t trs x

ct r

s tt s t ttrtt

trt t rss s ttrtttst

r r t s ttrtt

r t t M N tr t stt t s ct r

t

tr stss t s rrt P rr K tr sts stt C > 0s tt

h2 Chc1

K

s r h N () r s tt || K P qts t t tt trs t s s t t trt s sst s r tr h s t Ksrs t tr sts s tthc1 = 0 s tt h = 0 s s tr stss t P t t Ksrs tr N () s h = 0 strt t ts t P t tt srs rr r sss

sr t rr srs rr rts t r srs

x rs ts t : RM RN rrst r s rr t s rr rts t r

(x) x2 CK(x)1

K

r x r r tt

K(x)p = min

xK

x x p.

s rts t rr ss Ksrs ss t s srs r rstss tsrs ss tt rt s t ss r rt Ksrs trs rts r stt s t rt t rr r st x s stss t r rts tt r s sst ss ss s s srs

r rss ss t qt t rt ts t t rtr x s r s rrr t s r rts s t r r xr rs s st rtrr s sr t rstrt rrr

s tr p rs p r t t s rts r ss s tt t trt rts t P r st rr t r

t rts t s st s tt sr t rt rrr

s K(x)1/K rtr t s st K(x)2 r s tr ts rs

tt

s rt s t t ss tt t rt r r srts

tt rrsts t st ss rt t t s stt rts

rst t

tr ts rs s tt t P rr 2K s st t sts rt tr r rt rr rt s sr s rr t rr t

tt tr strts tt tr sts rr rt sts

t st ssr sts t P rr 2K

r r

t : RN RM t ss tr : RM RN t rtrr rrrt t r (,) stss t stss t P rr 2K

t tt ts tt s s t rr t t t || tr t t trsrrs t r t M || tr t t s rrs t s s rr t tt t t tr s sstt r t t t

trt t rss s ttrtttst

r r t s ttrtt

P

Pr

s h N () t t s rrs t t 2K rst trs h t st t 0 1 r |0| = |1| = K t x = h1 + hc x = h0 s tth = x x strt x K t t x =

(x

) rr

s h N ()

h = (x x) = 0 s tt x = x s x = (x) tt

h2 h2 = x x2 = x(x) 2 CK(x)1

K=2C

hc12K

,

r t st qt s r

rstrt str rrt

s rrt t P s t ssr st r sts rts

t r

(x) x2 CK(x)1

K,

t ts rts t t r s t srts r tt t s r

rrt s rrr s s qtt t s t sr st strr ts

s tr t str t trs sts tsrtt r rss ss t

t

tr stss t rstrt str rrt P rr K tr sts K (0, 1)s tt

(1 K) x22 x22 (1 + K) x22, s r x K = {x : x0 K} tr stss t P rr 2K t trrt s s tt rtrsrs t st t r Ksrs trs s r t tsr rstss t s

t s rtt t t tt r t t P ss s tt r str t

t ts s r r tt rt st sr rtrr s

x22 x22 x22 r 0 < < s s s s s tt t stss t str t t

2/ ( + ) rst tt stss t stt K = ( ) / ( + ) t t s ts t tt ttrs ts rs s t sst tt stss t P t s s tr stss s tt stss t P s s s s t sts s t rstrt r ttt t ts sr

t s tt stss t P rr K t stt K t r K < K tt tt stss t P rr K t stt K K rr t s s tt

s tt s t ttrtt

r r t s ttrtt

stss t P rr K t st s stt t t s tt sts t P rr K r rt t t st rs stt

rr

s tt stss t P rr K t stt K t st tr stss t P rr K = K2 t stt K < K r ts t rrtr

s s tr r = 1, 2 t r 3 K 4 ts s s t t r P rrK t r rrs t r tt K st st s rr r t rst t s

P stt

s tr ts rs

tt tr stss t P t ts s st r rt rts t t t sss rr srs s r s srts rst

r t sr t tr t P s t ssr t s r tt t

r t P s ssr t s t t rr srs ss x r tsrts x r t s rss tt t P s ssr s r t t sst t P sr t t stt

t

t : RN RM t ss tr : RM RN t rr rt s tt t r (,) s Cst r x K e RM tt

(x+ e) x2 Ce.

s t s ss tt s t s t t srts t t t

ts t rr s s t rtrr r r strts

tt t st rt tt rt tt st rr r s

srts rqrs tt sts t r t stt tr C

r

t r (,) s Cst t

1

Cx2 x2 r x 2K Pr

P x, z K

ex =(z x)

2and ez =

(x z)2

,

t tt

x+ ex = z + ez =(x+ z)

2.

trt t rss s ttrtttst

r r t s ttrtt

P

t

x= (x+ ex) = (z + ez) r t tr qt t t Cstt tt

x z2 = x

x +

x z2 x x 2 +

x z2 Cex+ Cez2= Cx z2.

ts s r x, z K t rst s

t tt s C 1 tt st sts t r t K = 1 1/C2 0s sr t r t t s r rr s t st st s tt tstss t r t ttr stt

t rs t ts rst r tt s t r s t ssr

rs s rs s tt s s stss t P t 2K < 1 t rs rs sts r stt C stts r t s t s s t r ts s t s r s st t t s rt rsrts rs ts s t t t s t rtrr

sts rts s tt t t s s r t t s

r rt t t t rs t rtrr r rr rt stts t s s t t r ss tt t s tr e rrstsqtt s r t r qtr t B ts s t srts t tr [T, T ] st t qtr t tr ts r rs t t srts t [T, T ] st s t r r qtr ts s t rst qtt rrr s s e rt trstrt rrr

srt s

s sr srts r ssr t t P r t t

s t ss t r N M K t sts s r rst r rr tt t r t tr

t K N sts K < N/2 r sts st X K s tt r x X x2

K r x, z X t x 6= z

x z2 K/2,

log|X| K2log

(N

K

).

Pr

sr t st

U = {x {0,+1,1}N : x0 = K}. strt x22 = K r x U s strt X ts r U t tt x2

K

r r t s ttrtt

t sr tt |U | =(NK

)2K t s tt x z0 x z22 ts x z22

K/2 t x z0 K/2 r ts sr tt r x U {z U : x z22 K/2} |{z U : x z0 K/2}|

(N

K/2

)3K/2.

s ss strt t st X trt s ts tt sts tr j ts t t st tr r t st(

N

K

)2K j

(N

K/2

)3K/2

ts t t r s strt st s |X| r tt

|X|(

N

K/2

)3K/2

(N

K

)2K

t sr tt(NK

)(

NK/2

) = (K/2)! (N K/2)!K! (N K)! =

K/2i=1

N K + iK/2 + i

(N

K 1

2

)K/2,

r t qt s r t t tt (nK + i) / (K/2 + i) s rs s t i s st |X| = (N/K)K/2 t

|X|(3

4

)K/2=

(3N

4K

)K/2=

(N

K N

4K

)K/2(N

K 1

2

)K/2

(NK

)(

NK/2

) . s r |X| = (N/K)K/2 stss t

s ts sts t t rqr r srts t

sts t P

r

t M N tr tt stss t P rr 2K t stt (0, 12] M CKlog

(N

K

)

r C = 1/2log(

24 + 1) 0.28Pr

rst t tt s stss t P t r t st ts X

x z2 1 x z2

K/4

r x, z X s x z 2K 12 r s

x2 1 + x2

3K/2

r x Xr t r s tt r r ts x, z X tr s rsK/4/2 =

K/16 t x z t ts s st tr t r ts

r r t s ttrtt

P

s tt t tr st s s ts t t rr rs

3K/2 +

K/16 t BM (r) = {x RM : x2 r} ts s tt

Vol(BM

(3K/2 +

K/16

)) |X| Vol

(BM

(K/16

)),(

3K/2 +K/16

)M |X|

(K/16

)M,(

24 + 1)M |X|,M log|X|

log(24+1)

.

tr s t r |X| r

t tt t rstrt t 12 s rtrr s r r r tst t rt sts s r max r max < 1 rr t rtt t t stts t s rt t tt t r r qt rs

t t r s st ss rt sts sr rst t N K t t t 1 r t ts rst r t tr t rs M t sr P stt rr t qtts t rt rsts r t sstrss rs

s t sts ts s ss t s s tt

r t t p ts s t ts ts RM s tt t sqr 2st t r ts s rsr t tr 1 t st tt

M c0log (p)2

,

r c0 > 0 s stt sstrss s s rt t t P s trs tt sts

t P t tt rr tt s r rt r strsr

r t s s t strt tr tt stss t P rr

t s s tt tr stss t P rr K = c1log (p) t stt t st strt strsr r p ts t = /4 ts t

M c0log (p)2

=16c0K

c12.

s r s t r srts rqr t sr tt stss t P rr K rrt t K/2 st r t Klog (N/K) r rtr ts

P t P

t s tt tr stss t rstrt str rrt t P t t s

stss t s rrt t P s t P s strt strr t t P

r

s tt stss t P rr 2K t 2K | t r qt t u1 u2sgn (u) 2 r s s sgn (u) s t K r trs q t1 s u K ts sgn (u) =

K r s t sr tt t K r trs u r u

stt t s tt rr t r r

s rst s r rst s r rtrr h t st trs h N () t s r tt h N () t rt s sr rts r s s tr t t r srs rr r s

srts t tr ts rs

ts sts t ts rt

stt t r s r 1 t r t

s tt stss t P rr 2K t h RN h 6= 0 rtrr t 0 sst {1, 2, ..., N} s tt |0| K 1 s t st rrs t t Ktrs hc

0t rst t st = 0 1

h2 hc

01

K+

|< h,h >|h2

,

r

=

22K

1 2K , =1

1 2K .

t tt s r rtrr h rr t r r r t t t s r h N ()rs ts ss tt h N () t s st t s tt

h2 Chc1

K

s r t s r s t st rrs t t 2K rst trs h s t 0 t t st rrs t t K rst trs h

s tr ss s h = 0 ts

h2 hc

01

K.

s

hc01= h11 + hc1

Kh12 + hc1

trt t rss s ttrtttst

r r t s ttrtt

P

rst

h2 (h12 +

hc1K

).

h12 h2 tt

(1 ) h2 hc1

K.

sst 2K 0 t st M = 2K r r t ss strt trt sst 2K s r t ssts 2K s 1 2K r 2K > 0 r ss s t t stt 2K rr t t t stt st sr ss

(NK

)Ks sss RN

r tt rst ts s ss r rst s N K s s t r t P rr 2K r s stt 2K r trtt s s ts r rst sr tr r t rr ss r trs

sr tt t tr sts t P s t ts t r strt

rst rqr tt t strt tr tt s rrsr rqr tt

E(2ij)=

1

M,

t r t strt s 1/M rqr tt t strt s sssstrt t tt tr sts stt c > 0 s tt

E(eijt

) ec2t2/2 r t R s ss tt t trt t r strt s t tt ss strt s s qt t rqr tt ts r strt t st s

st s t ts ss strt s sss strts t ss

s tt s t ttrtt

r r t s ttrtt

strt t r strt t s 1/M r r strt t srt ss r rs t r r ts

r t t t ss t r t ssst ss

tt t trs r strt sss s tt t sts t c2 = E(2ij)=

1M r rsts t t r r sss strts t t s t

stts rstrt r rst ttt t t strt sss ts s t s rst s r trt sr r sss r rs t

rr

s tt s M N tr s trs ij r t ij r r t strt sss strt t c2 = 1/M t Y = x r x RN r > 0 x RN

E( Y 22) = x 22

P( Y 22 x 22 x 22) 2exp

(M

2

)

t = 2/ (1 log (2)) 6.52s ts s tt t r sss r tr str trts t ts s

ts rst Pr t P r sss trs t r s r s

tt tt sss trs sts t P

r

(0, 1) t M N r tr s trs ij r t ij rr t strt sss strt t c2 = 1/M

M 1Klog(N

K

),

t stss t P rr K t t rsr t rt 1 2e2M r 1 s rtrr 2 =

2/2 log (42e/) /1t tt t t srt s rstrt str rrt t s

tt s t t r srts t stt

s r trs t strt s r t ts strt ts s t P rst s tt r r strts t srts r rt

tt t s ss t rr s s st r sst t srts

s s r rst t t ss r rrt s rt tsrts rs r st rt r t r trst t

stt r x s srs t rst t s ss ts s t t rqr s tt trt stss t P r t s trst strt t t tt t t r strt t s s r ts srtr s s r t ss strt s rtr ss t s s tt s ss strt s r tt M s st sts t P t rt st s r t ss s sr rsts rsss strts s s rrt sts rrr t s rst sttts

st t s r trs t strt t tt s t r tr r s sts rt t r

r sr rr rttrs t r rs tt r sr

ts t qr rt stts s t r tr t

r tr t t rtr r t t

rss tr s rttrs t s r t rss r

r r t s ttrtt

P

trs tt st r strtr t r tr Prs st srrs t s t t qt s s s t r s r tt

ts strts s sts t P

r

t sr t s rrt t P rstrt str rrt

t P r rts r t rr srs t ss r tt

r tr stss ts rrts s tr tt t s

s st sst sr

(NK

)strs stts t s rr t s rrts

tt r s t t r r rt rr rts r tr s s rrt

t

r tr () s t rst st r rt t t si j

() = max1i| i2 j 2

.

t s ss t s tt t r tr s s t r () [

NMM(N1) , 1

] t

r s s t t tt N M t r srt () 1/M sts rt r t t sr P P r t r sr

rrts tr rt t rsr r tr

r r

s N N tr M t trs mij 1 i, j N t N ssdi = di (ci, ri) 1 i N tr t ci = mii t rs ri =

j 6=i|mij | ts tr t r tr G = T s t t strtrr rst

r tr

spark () 1 + 1 ()

.

Pr

spark () s t t s t s ss tt ss rt tt s tr s t {1, ..., N} t || = p tr st s sr t rstrt r tr G = T stss t rrts

gii = 1 1 i p |gij | () 1 i, j p i 6= j

r r r

j 6=i|gij | < |gii| t t tr G s stt s tt t s r r t s t sr t s(p 1) () < 1 r qt p < 1 + 1/ () r p < spark () spark () 1 + 1/ ()

r r r s ts t t s

t t tt rts qss

s tt s t ttrtt

r r t s ttrtt

r r

K |h2

.

sst tt 2K |

h2.

P r s t sr stts

stss rrr r t ss 1 t rts sr t srt tr sts t P rr t t s s r

rt s B (y) st rqr x B (y) ts |< h,h >| s t s sr srts t t tr

r r

s tt stss t P rr 2K t 2K |h2

.

rtrr s x,

x B (y) s tt y = x = x h = 0 rr ts tr ss t t sr rst

r r s rtr rr sr t s r x K ={x : x0 K} s tt r stss t P s s rr s r s sO (Klog (N/K)) srts rr Ksrs xt s rst ss r ts s t t tt t rr sst t s t s t

tt s s t strt tt ts r s t st

t tt r r sss tt stss t P s t rt t r ts t t sst tt stss t s rrt t P st r trst t sss stt s h s t s t ss ts r r t t sts i s tt stss t P t t stss t P s s P t P t iit P s t s rs s r rt rrrs tt

s t s rt s t r r r t s

strtrr t s tt stss t P t t t s rrr

rr s

t t rt rstrt srs t s r sr t srts

rrsts rs rst r st rr ssts t srts r t

tt s r s r st rr t rss t tr st

t rrst t s t r ts t srts t st t

qtt rrr rr ssts r t s rr st t

rt rt ts s t stt

Prs st srrs s tt t s ss t

x= argminz

z1 subject to z B (y) . t st rr srs ss r rt s s s t

t t rstrt str rrt t P s tr s sts rr

rts s

r ss rt s r sr s rr t s

rt t r r

s tt stss t P rr 2K t 2K |h2

.

s tt s t ttrtt

r r t s ttrtt

P P

r

C0 = 21 (12) 2K1 (1 +2) 2K , C1 =

2

1 (1 +2) 2K .

s

rst r t rsts rr r r s s rst stt

r r

s tt stss t P rr 2K t 2K | ts sr tt

h2 = (

x x)2

= x y + y x2

x y2 + y x2 2

r t st qt s s x,

x B (y) ts t t P t r qt t

|< h,h >| h2h2 21 + 2Kh2. s

h2 C0K(x)1

K+ C12

1 + 2K = C0

K(x)1K

+ C2,

t t r

rr t ts rst tt sr rr srs tr x t r t K ts t r ts t 0 s s rrr t s tr sttr ts s tr r s t rstrt t s s s srs

x0 = 0y =

(T00

)1T0y

xc0

= 0.

r r t s ttrtt

t sst s tt 0 s r r sr t sr 0 s t M K tr t t s c0 r s tt tr s q stt t qt y = 0x0 t ts t rr rrr s

x x2 = (T00

)1T0 (x+ e) x2 =

(T00

)1T0e2. sr t rsts r ts rrr s str rrts t sr

st t s strtrr t s tt stss t P rr 2K t stt 2K

t t rst sr 0 s t r[1/1 + 2K , 1/

1 2K

] s sr t

rsts rr rrr r e s tt e2 t t rr rrr

1 + 2K x x2

1 2K

.

rr x s t Ksrs t t rt r t srs rr t s rt t tr srt x t r t r r r t stt

st rt s rs r r

ss tt t s r e2 s s t tr s rt rr rt s t t str t s r T e s s s tt ts t s ss t rr ts rt ss s

r

s tt stss t P rr 2K t 2K | rst ttt

Th T(

x y)

+ T (y x) 2

r t st qt s s x,

x B (y) t t tt h = h s ts t r qt t t

|< h,h >| =< h,Th > h2Th2. s Th 2 tt r t Th s t st 2 tsTh2

2K (2) s

h2 C0K(x)1

K+ C12

2K = C0

K(x)1K

+ C3K,

r r t s ttrtt

P P

s sr

ss s

s t rr ts rs t rs ss s s

ss s s rst sr t rr 0 t t ssrts s tt r r r

r t r sr rts r 1 t s r sss rstrtr ttt t t s r x K = {x : x0 K} s tt K(x)1 = 0 t rrr s r r r t s e ss tt t ts e RM r r t ss strt t r r 2 t ss strt s ts sss rsts ss rr r trt sr r sss r rs t t s

tt tr sts stt c0 > 0 s tt r > 0

P

(e2 (1 + )

M

) exp (c02M) . ts rst t r r t = 1 t t rstr t s s ss s

rr

s tt stss t P rr 2K t 2K vi r ej ts t tr rs t r t jt tr s s tt hj = PN ej r PN ts rt rt tN () PN ej22 + PN ej22 = ej22 = 1 tt hj2 1 s stt = {j}r hj sr tt

NMi=1

|vi (j) |22

= |hj (j) |2 (1 1

C2

)hj22 1

1

C2.

r j = 1, 2, ..., N t

N1 1/C2

Nj=1

NMi=1

|vi (j) |2 =NMi=1

Nj=1

|vi (j) |2 =NMi=1

vi22 = N M,

ts M (11 1/C2)N s srs t t r tt s r ss x t stt C 1 trrss t rr rt s t t M N srts r ss ts rst s r ssst s tt t rsts st sts r s

rr s t s t r ts tt sst trt t rt

rrr s s

rs ts t tt t rsts r 1 t stt ts r r trststt rts tt sts t x tr tt stss t rstrtstr rrt t P s s rtt trt rrt t s t trs

tt sts t P t rt t s r t t t trst rt tt t

tr stss t P rtr strts tt r rss r t stst P t rt s r r r trs tt sts t P

t s t r t st r rsts tt t rt

r

(0, 1) t M N r tr s trs ij r t ij rr t strt sss strt t t c2 = 1/M

M 1Klog(N

K

),

t stss t P rr K t t rsr t rt 1 2e2M r 1 s rtrr 2 =

2/2 log (42e/) /1 t t ss rst rsts tr r t stt rs t r s

t rst strt tr tt sts t P t rt t

t rs rsts ts tr s s rr tt t rt sts

trst rt t ss ss x r rst s tt stts tt s x r r tr t rt t rt rr rtt s x s t rt s sts stt rt stt ssst tr r t t r r r s x s rttstt rt t ss r t t tt t s rss t r tr r x t s tt r r st ssst rr t sts r rst r t t rs s trs tt sts t P

t tt sss trs rsr t r rtrr tr t rt

st t rr r trs tt sts t P t ss

tt r x RN s r t t rr r t s tt

P( x 22 2 x 22) exp (3M) r r t s ttrtt

t 3 = 4/ s ts t t rst

r

t x RN t 2K 0 rtr s trrrr r sttst tqs s srsst

r rr rts t st s J H r s s s sJ (x) = x 1 t 1r x H (x, y) = 12 x y 22 t 2r t rrr t tsr srts t r rts t trt tr x sttsts ts Hst t x 1 s s t ss r r r J () ts s rrt tr r tr r ts r t sr s s

stt sts srs rrstt r ss trsr ts s s rrt tr

J (x) = TV (x) + x 1 t t tt t s t tt s r t rts

rtss t rs s t s r s t rt

rs tr t rr ts r r rs

rst 10241024 s s t tt r rs t r str tt str t t t s st str st

tqs t r rsts r ts rs t rts t

tr trts r t t r s q s tr

t tt t sr trst r srs rr rts t

tt t

r rr

t sss t s t 1t t r t sstttJ (x) = x 1 rst s r rr P t qt strts s s t O

(N3) s str trrt ts s s t rst s

rstrt rt s r rr s str trt rts s s rr

s tt s t ttrtt

r r t s ttrtt

ts rs t t s s t r rst s srr rr

P t qrt strts Ps Ps s r trst tt rsr

rtss tr t rt rs s t t t t tt t t s

s N t r strts M r r srs t tt t Ps Ps rrs t t str rts r s s rst rr

trrt ts

r rt s s s trrt rt tt ss rt

t rt P t t rt s r ssts trtt rr

rt ts t strtr t ss t strt tr rtr ts ts s s

rr t tt rsts s tt t r P sts r st r t

rt rstrt s t s t s t sr t rstrr ts t

str ss r t tr trt s s srs

t tt

s s t s t str rt trt r s t s t str

rt s t r s 1t rs t r

minx

x 1 +H (x) , r rt H s trt rr s sr s st trss t tt s t qrt H ts t s t rs t rtr st r t

r s ss t s ts s sr rtr

sr t s s

shrink (t, ) = {t ift > ,0 if t , and

t+ ift < .

s t s t t s rtr t s rt rtt s

t t trt r i = 1, ..., N t ith t x t t (k + 1)tht st s

xk+1i = shrink((xk [U+25BD]H (xk))

i,

)

r > 0 srs s stt r rt st r t k s s s t sr t s s t s tt t rr s t rr t st t xk+1 xk r qrt t tr H () t rt [U+25BD]H s t s r t xkts trt sst s t s r trtr tts

st t trt r s qt t r tt s s

s stt rs ts ts rts t ts r

rs t t r t t s trt t t ts t t

rs s J r t s trt t rt ttt tt r t strt s r rs

sq s r t rtr t t trt trts trs t t sts rr s s tt t rr rt s Pt

tt P s r r t tr sr t s rt Pr

t r rs strt P s st t s t str

tt r s t t rtr s sss r rr s tr rrr r t sss str rt s t rrs str

t s r r

r r t s ttrtt

P P

t s rr r s rss srts s r t

st t H (x) s t sqr t r t rs s

H (x) = y x 22[U+25BD]H (x) = 2 (y x) .

r ts rtr t t rs t t trt

xk+1i = shrink((xk [U+25BD]H(y xk)

i,

)

s r t r t t rt s t s r

ts tr s srts y rtr sq n

tts stt

x

t

x0 = 0 r = y k = 0 t rtr s

k k + 1 xx Tr t rt st

x shrink (x, k ) rr st trs r y x t srt rs

rtr

xx

r trt ts

t trs t tt t t t t t st t t str tt r

s s s r t str rs t r s

srs r rrr t s r trts s rs rtt s s

yk+1 = yk + y xkxk+1 = argmin J (x) + 2 x yk+1

2.

r t s st s t rts r r trts r

tr r str tt rt t rs s r t r r

s ts J (x) t s t r J (x) = x 1 s t r t r sts r > 0 r rt t r trts s t ssrt r t t trt r tt ss tr rt s t str

r t r r r trts r t r st sts str

sss

t ts sss ts st t t s t 1r r ss st s s rt r t t t tr rs

tt t s tr stss t ts s rr 1 tt tt ts rr t r s x t rt ts s rt st rr rrt t rr r s t P r t

str rt

r r t s ttrtt

r rts

t

s s t s ss tt s tt t rr

trt r t srs rr t s t ts srs rt tt

t srs rr s t rr t srsst tr x s t r srts y tr rs t s t r

minI{|I| : y =

iI

ixi},

r I ts rtr sst t s i = 1, ..., N i ts t ith t s tt sr r t r st r t s r t t sst I tsst rt s Pr st ss srs rt ts t ts r

r st s r sss ttr rts t y

t Prst

t Prst P tr t t s rss t t

s trt r rt tt ss s t r t

ts r tr srs rr ts tr s r t s tr RMN s srs rrstt x r s yP s t r s qtt P s t rs r RM t rs rrstst st rt t srts t trt t rt st tr

r t tr tt s rrt t t rs r

k = argmax

< rk, >

2.

ts s st ssss ttr rrstt t s s t

k s t r s rt s t t t rs trt s s

rk = rk1 kk2 ,

xk =

xk+ < rk1, k > .

rt t trt st st rtr s t r r s sr t sqtt P s sr s r

ts srt tr s srts y

tts rs s

x

t

x0 = 0 r = y i = 0 t rtr s

i i+ 1 b Tr r rs s stt

xi xi 1 +T (1) t rst t t r r xi t srt rs

rtr

x xi

s tt s t ttrtt

r r t s ttrtt

P P

t P s tt rt rt t s t sssss t r

rs t rs rts trs rr rrr t s t t t s

strtr rst t tr t rqr r trts rqr qt r t P s O (MNT ) r T s t r P trts

rt t Prst P

t Prst P r t tt s r rs s t t

P rs r t r trts T s t P t r P trts r s s t trt k st strt ttrt t tr t t t rs r s rrt t trt r t t rt ss t t r s t rrt st tr tss qtt ts ttr rrsts t rt t rs s strt r r tr rs t rss s rt s t str r t s stt t st t t rts r rr

xk = argminx y x 2,

t = xt,

rt = y t.

s sts r rt t r s s s rt t Prst P

r rt r tt P s t rr srs s t rt s

rss srts rt rs t st K trts r s t srst trqrs t tt st rtt t trt t tt t

P s t O (MNK) . P s r st s t t t rr t rts

P r srs rr r r t ts sst t tt tqs t

rtr t rstrt rts r t r t t s tt s srt

tr t M = CKlogN rs s t rr r ss Ksrs s t M = CKlogNsrts t t s ss t t s r rts t s t t t

r srts r s tr ss t P s rstss t s t s

tr t st t P rtr st t t s t

s t srts rtss P s t t r rr s t

s srst K s s rrstt P s s

ts srt tr s srts y

tts rs rrstt

x

t

0 = 0 r = y = i = 0 t rtr s

i i+ 1 b Tr r rs s stt supp (T (b, 1)) rss rst t tr t s srt

xi| x xi| C 0 r s stt r y xi t srt rs

rtr

x xi

r r t s ttrtt

ts rt t Prst tP

rt t Prst s t t s s t r srs s t tt st

rss qrt t t r rs K ts stt ts rt tPrst tP s ttr r rt srs ss rs stt

tP rs sr tt ts r 1 t t rtt Prst r r s rs t srs sts rt strts t t rs

r0 = y ts t st rts T rk1 t t kth st s P r st s tr t t ss trs rtr t tr t t st st s s rrts t t rrt rs rs s t s st sqrsstt t s s ts st s st s r

P t r trts tP s s r S = 10 s r r t t tP s O (KNlogN) st rt r Pr tP s t r ts rstrt rts tP s s rt

r rqrts r t P r t rtt rqrs t t s

trt t tr ts

rss t Prst P

r rst rts s s P P t t ss tt t

tr tts srs rr t s t sst str rts r r s

rr rqst r srts t s t t s tr ts

r rts r rst t s r srt s

r s rt ttts t r rts r P

rss t Prst P s Prst tt r ts

t rt t tr t rstrt str rrt t P

t tt 1 t t s tr r s rts t tr stss t P rr K ts s tt r sst K s ttr s rt rtr s rrt s s t r str r rsts ts

r ts

rt s r s t P rt trst tr

P s tt P P tP s s stt s s

t r t rrt st s s trst r rst rts sr r t t tt

s r qt s t r t tr rs t s rrsttt t r srt P s s s t t strt trt i ss t

s stt s

xi1

r s rs stt e T r t st 2K ts t s rs e ts st s r srts T supp

(

xi1

)

r s stt b ss rt b|T T y b|TC 0 Pr b rt ts K rst ts ts stt xi t srt rs r y xis rr s sr s r

ts srt tr srts y s srst K

tt Ksrs rt

xt tr s rrstt x

t

x0 = 0 r = y i = 0 t rtr s

r r t s ttrtt

P P

e Tr r s rs stt supp (T (e, 2K)) r s rs stt T supp

(

xi 1

)r srts

b|T Ty b|TC r s stt

xi T (b,K) r s stt r y xi t srt rs

rtr

x xis sss t tt sss r P r t rt t s rs

t t s r ss rt t s stt st r rt r ssts

t tt st P s t O (MN) s t t srst tr s s rrsts rt r t r rts s s ts

P r rrsts t stt t rt srs rr rt rr t

sssss r t rt rqrs rr t srst K t trt s rrt t srst t rs rt t t t rrr rr

r rt s s P stt s P sr tt t rrr t

rrt rtr s t t s t t ts s trst t rt

trt r rs

trt r rs s rt r s r rs rs

strtr s s strt t t stt

x0 trt r trs ts sq stts s t trt

xi+1 = T

(

xi +T

(y xi

),K

).

st s r tt ts sq trts rs t t

x rtr

t tr sssss t P t s tt t rr s

xstss sttt

rt t t sr rr t rts s s t r tq r

rst t s tt r r t t tr rts s s P P

sss

tt tqs r r ts r t srs rrstts tr r

s rt rtrt ts r s s rs r rts r trt

rt t s ts srt tr trt t t srt t

s t r rtr s t r trt t r stt t srs

s t trt t r t st t t sr t r ts

t s t rr rts tt t ts t r tt

rs t s t r sstt r rts r rr sr t ts

s r 1 t sr rs t r t tqs rqr t rrr rts r sstt rt r s st trt tqs r srs rr

s ss ss ss r srs r s t s r rts s s

ts rt trrt s ss ss ts

r r t s ttrtt

tr rts

t t tt t r rst t rs tr s tr

rtt ss srs rr rts tt rr t s tr rts s

rts st t trt tr s t ss rt t

rss ss t trtr t r rt t t srs s rr r

t

t

st tr rts r t tt r tst t

r tst r ss tt tr r N tt ts sst K tsr s t t r t s t t t rts

str stt r t sst ss tss ss tt t ts ss

t tr x ts ts r s xi 6= 0 r t K s ts xi = 0trs r s t s t tsts tt s t t t srt ss t

s t rs x s t r tsts rr t sst rtstt ts tsts r rrst r tr s trs ij r q t t j

th

t s s t ith tst t tt t tst s r t rst t t ts t t r rr t tr x s sst t s s t str srs rr rtr t r tr rts r s s tt t

strs s tt xi rrsts t r ts ss tr tr rtrt stt i str t tr x s t s s t tt r ssstts rrst t P rss s N = 232 s st ttt t str x rt str y = x r s M N tr t M N ts tt t tr y s t st t tt ts r y s t rt r t t rssss tt t tr tr t r sr xi rt rtr sr rts t xi t t stt i sss tr t rtr s strty trt t ith t y t sr rt t xi sy = x s r t tr tr s t tr t s r stts ttr x s rss ts t r rr x r t st x s sst ts s t srs rr r

r tr rts r srs rr t trtr

st st s rr Prst rs qt t

Prst t r sss ts rts st sr t s

ts tt t t rs tr srs rr t t

t st

H s t st srt ts h : {1, ..., N} {1, ...,m} t tt H s t st s mN t h H s r rtrst tr (h) s m N t r tr t t t t t j r j = h (i) strtt r s tr s d ts h1, ..., hd t r t r strt H rt tt tr rtrst trs s M = md s rtr s M N t t t d s s x qr r srts y = x t s s t s t srts t t rrts rst t ts t srt tr y r tr rr t t tr r ts {h1, ..., hd} sr t ith t t srt tr y rrss t t tr r t h t rss r yi s s

s tt s t ttrtt

r r t s ttrtt

P P

yi =

j:h(j)=i

xj .

tr rs r s t j srt yi s rss ssts srt xj rrt tr s ts t t s i t t h rr sst ssts stt t s s r ts rrt srts

t rt s s t s s r t trs t r s r st

srts y s t s tr s strt t stt t jth str s

xj = minlyi : hl (j) = i.

tt ts s tt t stt xj s r s t srts tt rs xj rrt tr s s t t t st t st t st ts rt t s r r sttt rt d = ClogN

m = 4/K t t rt t rr s

xstss

x x /K x x 1, r x rrsts t st Ktr rt x t 1 ss

t st

r t r stt t ts t r s tr st r t sr

rt s t s st t t srts

t t ts srts tt r rs rrt rs xj rt s t s t stt

xj = medianl

yi : hl (j) = i.

rr rts r t r sr t tt r t t rt

t r rt stt rtt tr t t t s tt t

rqr tt t srts rt sss trst t ttr rts

trt s ts srt s

r

t tt s t rr srs s r s r r srts t

ts stts tr r s rtt rs t s stts t rss ss

stt st ts rs

rst ts stts t s tr t ss tt t sr t

rstrt rt s s tr r s ts r t s r tt rst t tt rqr t rr rr r t s t s t s s ttt s rs t ss tr ts s s srs r st ts

r strt t ss tr t s trst t t tt r ts tt r t tr sts r t s s t rstrt str

rrt t s t r r t st str rts

trt t rss s ttrtttst

r r t s ttrtt

t tt t tt t t ts r rts

sr s s t st r N s rr t rr x st t st r t tt st r t N trs x s t t rssss ts t ts s rt N s tr r s t tr tr ts tt s t rts s t s r t t

t rrstt t s ts srst K ts s t rt s t rtr trstrt x t st rtrs ts K rst ts tr s s srrs s t s s rts r ss r s

s ts

t

rt ts rs

st s r t trst s rr

tr rs r s x s s t st ss ts st rt r tsrr ss tt t srs t r rss t s trst rss

r rt strt ss srst rt rrs t ts x rr r t stst srts y = x rt strt r t x r s r t r s ts r srs rr t

rts sss ts st strt rss r t t srs rr

tqs t s rss ss t t tt ts rts r

rts t r srts rqr r t t s rstrt

s s rr tr s t rstrt rrr

r s ts r st t rr rts r r sss ss

sr rt trst

rs rr rt

s s tr ts rs tr r st rs t r t rrr rrt s

srs rr rtr srs s s s P s r sss

t tt srs trs t ss tr t

rts s trsr t t t s srs ss trs

t srst t tr s qt t t srst P rs

s tt s t ttrtt

trt t rss s ttrtttst

r r t s ttrtt

P P

r tr r t t rts t t rs s

P r s r t strt s r t

ss tr tt s t rt t t s x srts y rrsts rtt r s t s x (i) srt s y (i) trr r rrsts t rts t t s ts srts t

r

s rt st s rt trst ts t ss trst

t s rss s s t strt srs s r s

tstt ss tr strt t s t t tr r r s

t stt ss tt t ts x r t s tt s tsr r rqt t t r ts tr strts ss t tstt ss

s s t t ts r t rr t ts x tt s t stt x y r ts t r s r r t t rt t r strts t x (i)ts t t sr srts y (i) t stt t stt r t Pstrr P stts t ts r tr strts

s srt r s s rt ts r t r rt

t P t s r t ts rt t t r

rtrr r s s P r r r s P s srs st t trs t tr r q t r

rs s r

tr rst r s t stt t ts x s s tr ss s sst s r t tt rs srs sst r

t s r ss ts r r tr tt ts r r

ts t trst rr rr t r t rst ts s

t t tr t ts srs x r t t ss ts rs ts st s rr rrs rst ss rr s ss t t N ts x ssqt rr s ss t t rsr i t i

thss rr rr

r r t s ttrtt

i trs t strt t rr ts sst t xi x s t srs tr t rstrt ts sst ss rr s

p (x|) =Ni=1

N (xi|0, 1i ) t rr s rtt s

p (|a, b) =Ni=1

(i|a, b)

r rr x t t t t ttt strt st t xi = 0 t rrt a b s s t sr st x t srs r s s r sr t t rs rr

rt t rs rr rt s t sr srts yt strr st xi s stt trt rt r t r ts r t ss t t srt s rr rr t

trt t t t st t rrtrs a b s s ssss srt s t r 2 rt t r r t t rt r rt rtt t stts r

L () = logp (y|, 2) = log p (y|x, 2) p (y|) dx.

s rss ss

rtt t t t r st s t t rs

N N tr rr t rts t t O (N3) st trt rt rt s t s t r s t rrs rt

st rst rt t t O(NM2

) r ss ts r sqt

t ts t strt t tr srst t s x s t t t s s s st r t s

t s rss s rt t t t t strr sts xi t t rt s tt t s t rrr rs stt

t x ts s t rs ts stts s rrr rs st t st t r rts t rs t tr t r rtt t ss rs tr t t r tqs s s rt

s t r

r r t s ttrtt

P P

r r t s ttrtt

tr

ts rss s

r rrss st

t srs rr rts t sr s r ts rs

r r

t rss t r srs r rrss st sttsts ts stt

r s t sst st t rs rss rs ss tt

tr r tt N t rs sr tt M t rss rs rrst t st t r srts s M N tr t st rss rsrts s M 1 tr y r rrss t s ss tt y rt s r t t t rs tr sts x s tt y x r t r t rs s r r tt r srts M N ts s tr s s t sttN rtrs r r r t N srts r ts ss t r t rt t s tt t rs r t ssr t rt t rss r

ts s t x tt s t stt s srs t tqs tt r s r r srs rr t stt x ts stt t s srst s r t rrss t t s rrs st t t st rt rs

rt t rss

rs rrr rrt

ts rrr rrt rrs t ss tt tt rrt rrrs t t

tt r t t strt t trsss tr rs r rrr rrt r

rtt ss r s r rst r sr sr t rtr s

s x t M trs s t tN r t rs {1, ....M}t N > M s t s t trs x s ts r ss s tN y =

Mm=1 ixi = f r s tr tt s t rt rs r s sstt t trsss rrts t trs y t s tt t r t sy = x+ e r e s rrr tr tqs r srs rr t t tt rss ss

t r r ts t stt t rrr tr e trr t ss trrt t t t s x e s st srs t stt t rrr tr tt s ss r t rt ss t t s t tr (N M)N

s tt s t ttrtt

trt t rss s ttrtttst

s tt s t ttrtt

r r t s ttrtt

P PP P

tr tt s = 0 s tr s t t srts t t t t tr t t y = y = x+e = e t tr s st r t stss t s s t rstrt str rrt t e s st srs

t t rrr tr e stt rt s t stt

es t t rrrr

srts stt s

y= y e t s rr s x= y= y es t rs m r t t strt ssst trs t Ksrs rrr rrt M < N CKlogN/K r stt Cs trs tt sts t P t

r tst t str rts

tr sr r rss ss t srs rr rts t

tt s s t tt r tst t rt r tt t strs

r tst

t str st srs rr rts r t tt tr

r tst ts r ss tt tr r N tt ts K sts tt s t r t s t t t rts str

stt r t sst ss tss ss tt t ts ss t tr

x ts ts r s xi 6= 0 r t K s ts xi = 0 trsr s t s t tsts tt s t t t srt ss t s t

rs x s t r tsts rr t sst rt stt tststs r rrst r tr s trs ij r q t t j

tht s s

t ith tst t tt t tst s r t rst t t ts t t r rrt tr x s sst t s s t str srs rr r rss ss

tt t strs

tr t r s rt t rss ss r s s tt

t strs s t t str r ss tt xi rrstst r ts ss tr tr rtr t stt i str t tr x st s s t tt r ss stts rrst t P rss s

N = 232 s st ttt t str x rt str y = x r s M N trt M N ts tt t tr y s t st t tt ts r y s t rt r t t rss ss tt t tr tr

t r sr xi rt rtr sr rts t xi t t stt i ssstr t rtr s strt y trt t ith t y t sr rt t xi s y = x s r t tr tr s t trt s r stts t tr x s rss ts t r rr x rt st x s sst t s s t srs rr r rss ss

rss

rstrt

t s s tq s t r r tt rts

tr s t tss t s rt s

s tt s t ttrtt

s tt s t ttrtt

r r t s ttrtt

t s t s tr s t t srt t rrt tss

ss rs tt rrqs ss tt t t t s t

s s t ss r st rt rt t rt s

rq r r s rst strt r t tts

t s rss t t sr qst rrss t rr t

t t t t t rr rs rr trsr t

s sr rr ss

r tt t rss s t r rt t t r sr t

ss s ts rt ss r r sst t s t rtts

srt t t tt r s rr s t tt rs

qt s rt t t t t

tr rss ss t t rstrt

t t trsr srst s str r

strt rs t rr rsts s rtts t s rstrt

r trsr rrs t t srs t r rss t

t rstrt s srs rr t ts t srt s

t trsrs r s t rstrt ts s t s tt rt r

t s rs qt rstrt

trr

trr tr r t r s ts

t rtr r t sr s tr tt strts t s t

r r r tr rst t s ts rtrst

r t tt s t stt t rrt srs s t r s

rs r

s r rt srs sst tt t r tt s t t st tt

s ts rts rtr st t r s t t

srs ss srs t s rtr r

s rrs r stt rr s t srs st

rt sss rrt t r rrt t rtrs t t t r s

tt t rrt s rt r t t srts ts t st r

r r t st t rs trt st rt rs t sr strt

t r rtrs rqr t sr sr rrts s t t r

srts tt trs t sts t rt rs t

trt rs

sr t t rss ss t t t r s

sst tt qr tst s x (t) t tt rtr tt s t s x (t) t ts qst rt x (t) s srs tstt ss tt x (t) s s srs t strtr t rr tt t s st t t tt t str s t sss t rt

ss s s r tt tr ts strtr rtr t r t

s tt r srt sst t st t s rr s ss

sr rstrts t s rts rr

s tt s t ttrtt

r r t s ttrtt

P PP P

srt

r rt s r t tst s x (t) st s srtst tst ts {j (t)}Mj=1 t sr t t s t [0, T ] t t M srts t r

y [j] =

T0

x (t)j (t) dt.

sst t t s srts rqr tr ts

rr r rt t tst ss j (t) M rrtrs tt t t s x (t) t rst j (t) M trtrs t r t stt

t s qt t tt t trtrs t t t srts y [j] rs ts st stt t s r tt t r

r str r j (t) s t r rt r rrrtrr j (t) rr rr t rttr sr rqrs M rrtrtrtr rs rt r tt rt s t r

st s s sts s s s t r P

s rst tr r rts t s sr rttrs r

s strtr j (t) sst t sr str rst s t s j (t) = (t tj) r {tj}Mj=1 ts sq M ts t t t s ts x (t) t r srts r qr s r t t qstrt tts ts t s r s t tr [0, T ] t st r s ttt ts r s rqrs s trt t t t t s r

r t rqrt rM r rrtrtrtr rs r s ssts st tr tts ts t rr r tr st s st

rts r t rr tr r rrsrs ss rt ss ss t

r sq ts tt r strtr t r t t t s s s

sr s s trtr rs rts sr t ts r str

t tr st rrs r t s rr t ss s rr

trsrs r st r t r rq s s ss s r s

ttrs s st s rtt ts rs r t

sst t ttr r rrr t t t ss r t

sr rtr rt r t r tr

rttr t r tr s t r t x (t) s rrtt sr sqr s 1s t sq pc (t) trts ts t rt Na r Na s t st s st s t qst rt x (t) ss trt r t r 1/Ma s trt trt tMa Na ts s r srts r

y [j] =

j/Ma(j1)/Ma

pc (t)x (t) dt.

rt t s rss t s r T N = NaT s t r ts t sq M = MaT s t r srts sss t srtt ts t t srt s tt t rrtr sq rt t

st rt t rts t rt rr t s sr t rt

tr sq t t rt t ssts r s

ts ts rt r r s st t s s r tr t

t ss s s

rrtr s s s tr t ts st t t r ss

r r t s ttrtt

r tr r

srt rt

t t r tr rt qrs rss srts tt rst s x (t)t s qt t sst rst ss x (t) t ts qstrt t srtt tr x t s tr t t t srts y = x s ts t pc [n] t t sq 1 s t rt t s pc (t) pc (t) = pc [n] r t [(n 1) /Na, n/Na] s srt rst srt r t s j = 1 ts s t [0, 1/Ma] s tt pc (t) s tr pc [n]r n = 1, 2, ..., Na/Ma s r t

y [1] = 1/Ma0

pc (t)x (t) dt

=Na/Ma

n=1 pc [n] n/Na(n1)/Na x (t) dt.

t s Na s t qstrt x (t) n/Na(n1)/Na x (t) dt s ts t r x (t) t nth tr s t x [n] s t

y [1] =

Na/Man=1

pc [n]x [n] .

r r srt rss s qt t t t s x t t r sq 1s pc [n] t s r sqt Na/Ma ts rrst ts s tr t Na/Ma sr 1s r r r t N = 12 M = 4 T = 1 s s rss s

=

1 +1 +1

1 +1 1+1 +1 1

+1 1 1

.

r M rs r t N/M rs t tt trs sts tsstrtr r tr t t rqr O (N) tts r t O (MN) tr s s s tr s r rr

r r t s ttrtt

P PP P

t ss t r tr st t rrts ts trs t

rtr s t s s tt rrsts t NN r srt rrtrsr tr t t tr sts t rstrt str rrt t P t rt r tt

M = O(Klog2 (N/K)

),

r t rt s t t rst t t r pc [n] s s tt x (t) s r r tt s s tt t s s t s srs r rss t ss t t s ss t rr x (t) r t srts r t r trrr t s r strt tt 1 t t t t rtr rr Ksrs ss t

M CKlog (N/K + 1) srts r C 1.7 t tt t s sr s st rstrt s r t t

srs ss t rq s t q t k/Na r s tr k Prs rrst s s t t s r t s

s ss t t ts K s t B r KB s ss t ttt ss t t rs tr t r tr t t rr

s ss rr tr s st tr sr rttrs tt t r ts

ts rs t r rsr s r r

t r rs rs rr t strt t tt t t s ts rs

t t r tt rs

r

rttr

r rr rttrs rs tt t tr rss ss t

stt s t s s r

s r s t tr tt sqt srs t r

rts y [j] =< x, j > t N s rs t t t r t sr t x st N tst ts {j}Mj=1 rttr s strt r r t r t s s r s s ts rs

t t s s s s r t t r s rr t

rtr t st t tr ts t tst t r t

tr s s trsss tt tr sss r s rts t s

r trsr tr s q rst s rtr

trt t rss s ttrtttst

s tt s t ttrtt

r r t s ttrtt

r r r t t rrs t t sr

x s rt t rrrr rr s rrr rtts r t

r t t sr ttr j s r r rtr rt

rrr ttr rs t t t s t tt rrss t srt y [j]

s strts t rrrr s rt tt st r

rts rts t t ssts rr trs trstt tt

rrrrs r rrr t rr s ss stt r ss r

rrr rtts t st t stts 10 rsr rt r t t s t t ts t t

rt t rts t rtt t rrrs

t t rrss t rtr t j ts rrs xr j rt t rrs t t tr trs rrs t t tt t j r r rrs t t tt t j s s s r s s s ts t rt t ss t t s t ttr t

s tt trts t rt x j t t t srt y [j] =< x, j > s tstt t s t s t t rtr s j t t tr t rrrs rt r t t trt t rs xt tr t j t r trs ts ts sst s t t rty = x r r rrss t j t r srts st trrr rtts j r s sr r rtr sr y [j] t rtt rss M ts t t t srt tr y

r r t s ttrtt

P PP P

r r t s r t

s s rs t rqr s t st t t ttr rr

t s t s t s t ttrs tt ss t t

r ttrs ttr t r t r t t

t s sr ts sst t rt t ts r t

ss strtr r rstr s

t t ss t t rt ts t s s t ts

tt t qt t s r t tt t ssrs t r

rr tt t tr r rs tt rr s

t rtt t t t s tt srt rs t N/2 tsr ts t r ssr st rs strt r r s

rt s

s s s t t ss t rs s str r t

s ss rstr s r t tst ts {j} r sq t ts [n j] tt tr rrr tr r r sstt ts t rt rtr t rstr s

r tt srts M r rtr t N r rstr s st r rs r r t sss ts sss

r strts trt t t rtt x rstrt

xt t s r rtt r s N = 256 256 M = N/50r strts N = 256 256 r s tr rtt t r tst t r t ts s r trs ttr t t M = N/10 t ss t s r rstrt s tt rt t s s rt t

t t 1 t t

r r t s ttrtt

r rstrts r s r 256256 t t rstrt r M = 1300 s r srts 50sqst 256 256 r rstrt rtt t r tst t stt s s ttr t ssr r trs M = 6500 rsrts

srt rt

t rr s rr rtrr tst ts j r rstrt t j t {0, 1} str ts ttrs r r s N s rt rtrrs t sr tt rt r rr s rtrt r r s sr tt t st trsrs tt tr strs s r r st trsr sr t s

trsr r t rs strt

ss tt N s r t Wlog2N t t N N s trsr tr stt W0 = 1 Wj rrs s

Wj =12

Wj1 Wj1

Wj1 Wj1

. s strt rs rtr tr t trs 1/N tt ts st ttrqr O (NlogN) tts t s t tt

W1 =12

1 1

1 1

r r t s ttrtt

P PP P

W2 =1

2

1 1 1 1

1 1 1 11 1 1 11 1 1 1

.

t ts strts s s s tt N = 2B rt WB t I t M N str t tt I t r st M rs s tt IWB s tstr WB sst t rs WB rtrr t D t r N Nrtt tr rt s

=

(1

2

NIWB +

1

2

)D.

t tt

12

NIWB +

12 r rss sts IWB t {0, 1} trs r tt r rs t tr rs tt s t s t rrrtrr D tt s tr rt t s r rt t s WB sst s s t rss s s t rs t s tr r rrt

t rs s t ss ts t rt t s ts ts strtr t tt

t ts t t strt rts tt s t t ss rt sts t rstrt str rrt t t ts r ss t r rt

rstr

tr t r s s trst sst tr ts r r

t t tst r t s tr rt rs ts s t

s t t rstr tstr ss ts r t t t

qr ssts trs t tt s t st ss x y strs s trs t s t f (x, y, ) tt rrst s st tsts t s t rt ts t s s r ts trs s

s s str str s t st ts s t str

s f (x, y) = {f (x, y, )} str str t rt tt rrs t t s tt s t tr ts r r s str strs

t s ss t t t t tr sr r tt s r s tr r ts

st

ts r s s t str r s rst t s

t t r str s sr r sr strtr s rst t

sr t st strtr tr s s s sr rstr

s str str s s st

s tt s t ttrtt

r r t s ttrtt

r rstr t t ss

rss ss t rttrs r rstr rr rs

rts tt t t st t str r t t tr

rttrs s s tr ss s t srs rstr ts

rss rstr rttrs

rstr r

s r t ss s tttr t rr r rts t t

t r t t t rt r rts tr sq s

rstr r rqrs t t t tt s rt rss t ts trst

s s ssr tt rr t sr str s srt r s s

r

s ssr ssts s strtr tt ss t ssr t r

rs t t strtr rrs t tst t t rt t tr

t s t rrrr rs rtt r ts r

r rr t r trt s rt t t t tr srt str

t tr tt rts t t t t t srts s rrst s

=

x,y 0 00 x,y 0

0 0 x,y

.

s rttr rrs t t st ss x y s tr s t rt str s t s

r r t s ttrtt

P PP P

r r r s rstr r t s r

strtr tt trs t t t tst r str s r t

srts

srsr rtr sst str r

srsr rtr sst str r s r s r

ttr tt s srt t t st str s t ss

t ssr rr ts tt t str s

rst srs t srts t rt str s st r t st

s trs ts t srs t t t str s s r t rs

stt t t s st s rst t s t s s t

rtr s t s t s t sr t rt s t

sqt s srs t ts t s sr t r ts t

srs t st rt t rt t sr t rst srs t rst

t s rt t trs sr s t s tt s t

rst t s t r ssr rr tts t str s

sr t s t ts r t r t rss t trt

r t stts s s s ts t srts sst s

t st x y ss t rt t t str s

r r t s ttrtt

r srsr rtr sst str r t t

ts strt t t rss rr t ts

srsr rtr sst str r

s srsr rtr sst str r s r s s

t t rttr t rst srs t s r s t

t r t t ssrs s t rs t trt rtrr s t sr s t

rrs t r t sr t s rr t tt t r t rqr

st rr r s r t tr

r srsr rtr sst str r t

t ts strt t t rss rr t ts

r r t s ttrtt

P PP P

rst strtrs r rstr ts

ts Prtt

s srst t strtr sss tt t str str r s r

s s t stt tt s tt t t s s stt t st rrs t st

ss t s t t r st sqrs t stt

str str ssr t rt rt t t s r rstrt

rt t srs r t s st t t t st sqrs tt rts

rss srts s t ts sr

r sqr rtt r s st stt t

t srst

s srst strtr rts str srt sss t s t srst

strtr r srst ss s r r ts s s s s

ts rts r t srt s ss ss rts t rst

srst ss r t t rrst s tr

=

x,y 0 00 x,y 0

0 0 x,y

.

r r t s ttrtt

rr rt srst

s srst strtr s srt srst ss r t st ss t str s

s srst ss r t t s t rr rt ts t

= x,y =

[1, 1]x,y [1, 2]x,y [2, 1]x,y [2, 2]x,y

.

ts r t t srst ss sts rs t st str strtr

tt srst r t t ss t st srsts r t srt str

ss t strtr t t ts strtr tr tr

srst

r

rss ss t t rst t ts t r r

s tsts tt t sr ts strtr rstr s

s ts t ssr rttrs t strtr s sr ts s t

rs r ts rt s sts ss rss s

t r str ss rttrs t t strtrs sr s trsr

s rss ss

rss rss t

r t r ts s t tr t s t

tt sssss r K trs rs r ss t Ks t s t s t s t t t rrst

s sst K s rt r st ss sttr r rst N srs tr strt t st s tr t r

rs s t s RN tr s t st s tr

stt r sr tt rtts ss r

s tt s t ttrtt

r r t s ttrtt

P PP P

r rtt s rs r ts rs t s

s strt F rtr Ks tr

ts t s t t rtr t strtr trt t t

rtrs t r st sr ss s s s r s

t st sr st r t st sr rs r rts

s ss s r s s s

s r t s t sr tst st rr r

sr s t rss r t st strt t t t sr r

ts t rr s s s tst rs s trst

s

r r t s ttrtt

r t t sst s s st K = 2 ssrtr rtt tr (1, 2) r 1 2 s t s t s r r r t R

N

r t st rt tq s rt tr tt

s t tq r rts sr Ks M t t sRN ts rt t r ss s M = CKlog (N) t tt K < M < < N rst s tt t rs tr strtr s ts rM s rsr t rr rt r R

Nt R

M s s s t t rst r rss ss t

srs t ss s rstrt str rrt t r t r s

tt t r rts rqr t rsr t s strtr s t t srst

t s t rtr t s t r

s rst s r r ts t ssts tt rt s rss tss

rr rt t r rts qr ts s ts s ss

str rss sts rtr ts s r t r t rt

r strts t rr s t trst s tst r r

rs r rts

r r t s ttrtt

P PP P

r s s r r r rts t s st sqrs

r rts r rts r rts t

ts r rts t t ss r t r t s t

t s RN r sr rss tr rs sr stt s st

s tr t rs s s s t

s rr t ss t sts t t

t r trsts ts rst tr s N t tr rsst

r r t s ttrtt

Prrss tr rssr stts t trs sK t r

r tr rssr rrs r t t ts r st s

s t Ks s s t stt K r t rs st

stts r N s r t t s t t t sts t rst trssst st r t t s rr r

rss s s P t r trstt t t tr rss r

ts rqrs rss r t ssr t rss t

r t r st ts t r tt sts

s trt r r ts t tr rt r rt s

r r rts t t t t tr rssr s r rts r

t r rts t t ts r t r stts t

ss s t str trst rss s r rt rss t ss s

t r rts s ts r t r sr t st s

s t trt s rs s t s ss t t stts

r r s tt s tr str trst r s r

srts s t

r s rss srts

t rss ss t trtr s s st s rs

s rstrtrt t ts s rqt t ssr r st

s rss ts tr s t ts rr ssts ss

r qr r t rs tt r sst s ss s s tt

t rqr rstrt t s t rqr stts t rt st sttsts

r t r t

s s ss sr sst s rss srs t t

rs rss stt r rt rt t trt r t t qr

ts sst r t tt t rs r t t ts rs t rst t

r r rts r r t s ts

strt t t s str srs rr t tqs strtr sr rts t rstrt s

r tst rts rt t rss srts tt rrstrt t s

r rrt tt s t s rss r rts s t rt st

rrt rss srts s rrt rss r t t srts

r rt s s t t r r t t st ss trstrsrs rt t rr r tss t sr srts

r r srts rqr t rr t r ts s s t tr t r ts r sr tt r sstt tss rqr r

srts

tr ss r rs r rts tt rt rt t rss

srts tt tr t rs r s rtr

s sst ss t sr s t t r r s sss rtr

stt t t t sr s

s tt s t ttrtt

r r t s ttrtt

P PP P

tt

tt s ss t sr t qst s s rst t srts

s ts r t ss t stt rt st sttst s trt sr

qt t s ss t s tt s st sttsts r tt r rt

stt r r rts r t qt ts stt s t s t s rt

ssts t s trst s ssts tt s t s t s s r rts r strt t tt stt tt t rt strtr

t s r t ss ss trst rrss t s ss trt

s srs rt s tt rt ts s s

s trs t rt r tt r t r r t

srts s t rt ss ts r s t rqr rt stt t

s s t rtr tr t s t ss trst r t t r srts

ssr s sr t tt rqr r rstrt s s r

r Prr r r s s r

str rr trr tt Prt rrr t rstrt tt r ss

str ss trr SIR = 6 s r rts ts s tt rqrs 3 r srts 4 r tts t rstrt r qtrt sss r

sst

r r rts s r rt sst str rs

sstrss s t t ts stt t t rt rst rs

r r t s ttrtt

s tr rt t sst rst tt r rts rst str

s s t r ts r t sr sst rts stts

sst s rr r rt s r s r t rt sss

s t s ss trst s s t

t t s s s t s tt r r rts rt tr

rrts t s ss r rsr t s strt r trt 2sts r rsr s s t s sst rts t rt

s rt s s t ss tr s r q strt sss

ss s stt t ss tr s qt t rstr ssr

t srt rtr t s s tt r Ks M = O (KlogN)srts r st t rr r rss sst s t r srts

ss s t s t s ss s s t t srst t s

rsts r s r

r sts r ss tr sst rtr stt rts

sst rts r stt rrr r r r srts M s

s r st s sr ts r r sts sM rss t sts t

t s rs s ts rs t s tr r rt stt

sst s t sst stt rrs r s rss M

rss ss r

stt

sr s x RN ss tt s t stt s t f (x) t sr tsrts y = x r s M N tr t str t ts rs ts r r ts s s r ts p rs strs s stts r t s s sts tt s r

rts

s t s r f s r t s tt t stt rrr rtt t rs x f stt tr M s rst s r ss r trs s strtrr sq t s trt

sr qt t tt s r s r r t

r r t s ttrtt

P PP P

Prtr stt s rr t s ss s s s

s sr s x rtr Ks rtr tr r K N t s tt t (KlogN) srts t rtr tr t ts t t rss rsts r s

r

rss ssr trs

rs t rss t ss r rst ssr tr ts

s s rt tr s rr sr rss ss

t s rrts tt t ttrt ts stts s s ts t ss

rss ts rst ts r rt s rst t s s qr

r s t rtr t ss s t trst Pt rs s t r t

tr r s s tr rts r r ss r srt t

t t rr s t tr s r st t s st

s t s r rss ssrs tt rr r r rt r

ts

r ts r s ssr trs rs tr ts ss t

trt t tr r r t r t r t r sst s

trst tr ts rqr rt t trstt t tr t tr t

r tr t t rt tt st trstt r t r rqr t s r

sr ss

tr rts

tr rts t t rss srts t ssr t tr

ts s r ts r tr ts trrs t tr r rt

ss

r ss ssr ts M r rt ts t s t r st s st rt rr t srts r t st

r s ss st t rt s rr s ts rts

rt s t t ss t t M srts s rrt s r rs r ss r rr t t ss tt rs t srst

r t st rsst s rqr s s t r rts s r ts rt

s t t t t s ss

t s ssr srs rss s ts s

ssr ts t r rts t t trsts t s r ss

t t rst t tr tt r ts tq s t t str rqr r

ssr s t sr r r trs ts s ssr str t t r

sst t ssrs r r ssrs rt s t t

srts r t t t tr t stt rrr s t tr s rs t

r trssss s (kMn2

) r k rrsts t r rs t ts rtt M s t r s sr r ssr r t r s ttr rsts r s r ss rts

s tt s t ttrtt

r r t s ttrtt

strt srs r rts

s t s t r ss r t t r ts

st rr rr t r r r st t ts Mrts ts t ss rt t s st L rs s t srtst rst srts r srs s N L rs trs r tss tsrts st s s srts t qt sr t tt r rts

t srt tr r t t s s srs rt s t t

s rr t r r rqr r trsss s r

tr rts

tr rts r s t ss t st rt t s t ts rttr

s ssr trs r r s s rr ss r tr t

rrs t rss

rss rss ss

rss rss ss ss t s srs t t r t trs

ss r ssr ssr ts s rt tr t s

ssr t trsts t t t trsss t

r rts r rt t t tr t t r t t rtr

s ts t s tr rs t srts r t rr

rstrt s t r rts

r ts t s t rqr rt srt t s str t

r t s s r strt trr t rs t r r t t

tr s tss st rt sr t t trst tr t

t sss tt t s r t rt q sts r t s tr

sst tt s t t rr s r r t ssr tr s

s rs t t t trsss rts trr r t sss s

r t t ts t

ts tts r rss r st t s r rt r ts

r tt s t t t srst ss strt rrt t

M2/(2+1) r s s st stt s t tr strtr t r rrrt t t ss t tr ts strts rrt t M2

strt rss ss

strt rss s rs sr s r r srs ss

r t t tt s srs ss sr t tr t tt s tr t

srst s s t t t t

t t t t r srt

rt r rt t srst s s

t s ts r srs

rs ts t sr srst strtr

rs ts s s

t s rr t t rt t t t s

tt s r r ssr trs t s s ts s

s s r rr s t t r ssrs t tr

r s t rss t t r tr ssr t trst t sr

ts t tr t t rs str t tr tqs r srt

trsss tr t t s tr t

r r t s ttrtt

P PP P

ss

ss ts s rsr r sq rt r ss r

s t tt t t sr s tr s rt tr

ts rss t t ss s rss ss t t

rss s rrr rrr t t t st srts

rrrs r rqt st r r ss t st

r ttrs t tr t t ss rss r rrr

ssts t ssrs r sts t sqs tr rs r t rst

rrr sq s sq r ss A T G C tt t t t tr tr rs A t T G t C rr ssq trt rss t s t t r r t ts tr ssq

rrr t r st strtr trt s t t s rst t r

t s s r t rrr trs s s s tt t s t

t rr rr s t s s sr t t s t trr rs

t sts r s rr s ttr rr sts rs s t s t

r t t t tst s

r rt trt rrr s str rt t

q t tr t rt rrr sts

r r tr sss t t trt rrr s st ssts rs tt

q t trt trst st ts t s r r rstss

rst r t ts s s tt r t t trts tst s sr s sqs

s t t r t t r r s r s rssrt ts

t rrrs t rr rt rrt rrr s ts t rss rssts t

sr sqs

s r s q tr s rs s ts rstrt t r

rss tt t t ss ts t rss st t

trr r r trts rqrs rrr t r r sts t tr

r r r rs t r t strs tt st

rss s rrr t s rt rt t ts r sts rrst st

r r r t rrrs ssrs s sq rt ssts r

trt rrs t tr r t s tr ts rqr

t rst t

tr r s t t tt t r r rr sts trt rrrs

t t r tt ts s s r r t ts r t t rst

st trt t t t r rtrs s t tst

s tt s t ttrtt

r r t s ttrtt

rr trt s rrr s rt sts t t t

rrs t t trts rst

t ts rs rss s rrr ss tr tst

ssrs rr t r t r ssr sts st t ts

r trt rss sr sts ttr rt q ttr tr r s trt

s r tst t str rts t s t rs tt rr

ts tr ss s t ss t rrr s rss t t sr

ts

t t srt s s r t r tr

s tt t rt r s rst t t trts ts r

rts r rrstt rs rt t r stss t ts trts ts

r t trts tt r t ts r t t t r rt s t st

t rrr s r t rt ts t ts r sq

t trt ts

r trtr t ss tr t M sts t N trts

r strts t ss rss r ss tr r M sts t N trts r r sts t trt ts r 1 i M 1 j N t r t st irs t trt j t t i,j trt j rs t tst s t trt

xj s tt t tt rt st i s yi =N

j=1 i,jxj = ix r j x r r tr rst rst sr rrr s tst tr y = {yi}i = 1, ...,M ts t srt y = x r tst s rs rs r rrrs t srst t trt s

s t s r r rr r tst s ts

rt st r t t st tt t stt t trt s t r

r srts sr t tt r tst s s rtt s tr r s

t qtts rt ts t rt t t s tr r trts tt

r t s rtrr r t s rr ts s s Prt

t tt x t r rt s srt rts tss trt s

r r t s ttrtt

P PP P

r r t s ttrtt

tr

s

ss r rs

r strts t ss r r t sts rt trt

sr t qts ts r r r rst

sr t ss sss strts

t

r r X s sss tr sts stt c > 0 s tt

E (exp (Xt)) exp (c2t2/2) s r t R s t tt X Sub (c2) t t tt X stss t E (exp (Xt)) s t trt t X t r st trt t ss r r s sss strt s

s trt t s tt ss r r trs r

sss strts t tr r t rtr rtt s

X N (0, 2) X s r ss r r t r 2 t X Sub

(2) s t t trt t ss s

E (exp (Xt)) = exp(2t2/2

) ts s tr sts

X s r r r r tr sts stt B stt |X| B t rt t X Sub (B2) t rtr sss r rs s tr tr ts

sr t r t tr r

X Sub (c2) tE (X) = 0

E(X2) c2.

s tt s t ttrtt

r r t s ttrtt

P PP

ss tt X Sub (c2) t E (X2) c2 sstts t s t sr r rstrt ss r rs r ts qt

s qt

t

r r X s strt sss X Sub (2) r 2 = E (X2) tqt

E (exp (Xt)) exp (2t2/2) s r t R t tt X s strt sss t r 2 s ttt X SSub (2)

X N (0, 2) t X SSub (2)

X U (1, 1) X s r strt t tr [1, 1] t X SSub (1/3)

sr t r r t strt s tt

P (X = 1) = P (X = 1) = 1 s2

, P (X = 0) = s, s [0, 1) . r s [0, 2/3] X SSub (1 s) r s (2/3, 1) X s t strt sss r qt rtrt r sss strt sss r rs

r tt strts tr trt sr r

r

r r X Sub (c2) tr sts t0 0 stt a 0 s ttP (|X| t) 2exp

( t

2

2a2

)

r t t0 rr X SSub(2) t s r t > 0 t a =

sss strts s sts t t rrts ss str

t t s t sss r rs s ts sss r r s rst s

sts r rt t

s tt X = [X1, X2, ..., XN ] r Xi s t t strt t Xi Sub

(c2) r RN < X, > Sub (c2 22) r

Xi SSub(2) t r RN < X, > SSub (2 22)Pr

t Xi r t t strt trs ss s

E

(exp

(tN

i=1 iXi

))= E

(Ni=1 exp (tiXi)

)=N

i=1 E (exp (tiXi))

Ni=1 exp(c2(it)2/2)= exp

((Ni=1

2i

)c2t2/2

).

t Xi r strt sss t t rst s stt c2 = 2 sr tt

E(< X, >2

)= 2 22

r r t s ttrtt

trt sr r sss r rs

ss strts t s rts t t trt sr

strt ts t tt r r ss

r rs t t t t s r t ss sss r r

s r r rss r t r t t st t r tr sss

r rs rtr X s tr r Xi s t Xi Sub (c) t t X 2 ts r ts rr t sts t rst s rs qt r t r rs

rs qt

r t r r X t > 0

P (X t) E (X)t

.

Pr

t f (x) t t rt st t r X

E (X) =

0

xf (x) dx t

xf (x) dx t

tf (x) dx = tP (X t) .

t rqr t t t t sss r

r

s X Sub (c2) E(exp

(X2/2c2

)) 11 ,

r [0, 1)Pr

rst sr tt = 0 t t s tr s ss tt (0, 1) tf (x) t t rt st t r X X s sss ttt

exp (tx) f (x) dx exp (c2t2/2) r t R t exp (c2t2/2) t t

exp

(tx c2t2/2) f (x) dx exp (c2t2 ( 1) /2) . trt t ss t rst t t t

(

exp(tx c2t2/2) dt) f (x) dx

exp

(c2t2 ( 1) /2) dt, rs t

1

c

2

exp(x2/2c2

)f (x) dx 1

c

2

1 .

s tt s t ttrtt

r r t s ttrtt

P PP

s ss t r t

stt r tr rs t rsts ss sstt t s

r tq

r

s tt X = [X1, X2, ..., XM ] r Xi s t Xi Sub(c2) E

(X2i)= 2

E( X 22) = M2. rr r (0, 1) r [c2/2, max] tr sts stt 4 max t rt

2/c2 s tt

P( X 22 M2) exp(M(1 )2/)

P( X 22 M2) exp(M( 1)2/) . Pr

t Xi r t t

E( X 22) = M

i=1

E(X2i)=

Mi=1

2 = M2

s tr t t s rst sr

rs qt

P( X 22 M2) = P (exp ( X 22) exp (M2))

E(exp(X2

2))exp(M2)

=QMi=1 E(exp(X

2

i ))exp(M2) .

Xi Sub(c2) r tt

E(exp

(X2i

))= E

(exp

(2c2X2i /2c

2)) 1

1 2c2.

s

Mi=1

E(exp

(X2i

)) ( 11 2c2

)M/2

P( X 22 M2)

(exp

(22)1 2c2

)M/2.

stt t rt t r s r s tt t t s

=2 c2

2c22 (1 + ).

r r t s ttrtt

P ts t

P( X 22 M2)

(2

c2exp

(1

2

c2

))M/2.

r

P( X 22 M2)

(2

c2exp

(1

2

c2

))M/2.

rr t s ts qts t tt

= max

(4, 2

(max

2/c 1)2(max2/c 1) log (max2/c)

)

t tt r [0, max2/c] t log () ( 1) 2( 1)

2

,

exp(( 1) 2( 1)

2

).

stt = 2/c2 rs t r stt = 2/c2 stss

s rst ts s tt tr t trs r r t sss strt

t t r t tr t trt r ts t M2 t t rt s M rs t r tt t r s r s t t c2/2 1 s r r sss strt t rtrrtt trt r r tt r strt sss strts tt c2 = 2 tr s s rstrt rr r strt sss strts s t

s rr

rr

s tt X = [X1, X2, ..., XM ] r Xi s t Xi SSub(2)

E( X 22) = M2 r > 0

P( X 22 M2 M2) 2exp

(M

2

)

t = 2/ (1 log (2)) 6.52Pr

Xi SSub(2) tt Xi Sub

(2) E

(X2i)= 2 s r t = 1 = 1 + s s s t s

rr ts t strtss t strt sss strt t rst t sr tt (1, 2]s ss r r t t s t tt s sts rt rs tsrr r strt sss trs t r sr r rstrt r r tt c2/2 < 2r s st t strts trst ts tss r strt sss t rs ts rt

t s tt s trst r s t tr s sr r r rt t stt C

r r t s ttrtt

P PP

t s t t s r t srt tt1 + 2 s tt st max = 2

r rr s t t s rr s rst

rs t rsts t t rr r strt sss strts

sr r

rr

s tt s M N tr s trs ij r t ij SSub (1/M) tY = x r x RN r > 0 x RN

E( Y 22) = x 22

P( Y 22 x 22 x 22) 2exp

(M

2

)

t = 2/ (1 log (2)) 6.52Pr

t i t t ithr sr tt Yi ts t rst t Y t Yi =< i, x > ts r ss r rs t Yi SSub

( x 22 /M) rr t t M s r tr Y t

Pr t P r sss trs

s t t t trt sr t rrts sss str

ts t t r s r tt sss trs sts t rstrt str

rrt t P s t s tt strt M N tr tr t M st r t t rt tr sts K (0, 1) s tt

(1 K) x22 x22 (1 + K) x22 s r x K r K ts t st ss x t t st K rs sr tt rqr s tt 2K > 0 t st M = 2K r r t ss strt r ts rt strt ts s t

rt sst 2K s r t ssts 2K s 1 2K r 2K > 0 r ss s t t stt 2K rr t t t stt st sr ss

(NK

)Ks sss RN

r tt rst ts s ss r rst s N K rr t t r s sr rr ts rs t t 2K ts s s t rs t s s st t r t P rr 2K r s stt2K sr tt t tr sts t P s t ts t r strt

rst rqr tt t strt s sss rr t s r rt s t

sr rsts stt rr r trt sr r sss r rs

t r r

s tt s t ttrtt

r r t s ttrtt

rr

s tt s M N tr s trs ij r t ij SSub (1/M) tY = x r x RN r > 0 x RN

E( Y 22) = x 22

P( Y 22 x 22 x 22) 2exp

(M

2

)

t = 2/ (1 log (2)) 6.52 t ts tr ss tt t strt s t strt s strt ssss s s t r rt stts rt s t sts

sr rst r r sss strts st s r r trt

sr r sss r rs t

r s t s tt rqr tt t strt tr tt s rt r

rsr rqr tt

E(2ij)=

1

M,

t r s 1/M s s t trt sr qt rr s

ttr t r rts t r t P r sss r trs r r r

t strt ts ts Ks ss t ts ts tr t t t rst r r t st ts t ss Ks sss rr t r t rst rqr t r t r ts rqr

t strt t ts ts r r rsts sr t rs rt

trt sr rsts rr t rr t t t trt

t (0, 1) r sts st ts Q s tt |Q| (3/)K r x RKt x 2 1 tr s t q Q sts x q 2 Pr

strt Q r s rst st rtrr t q1 RK t q12 1 t t ts t Q s tt t st i t qi RK t qi2 1 stss qi qj2 > r j < i s ts t r ts r x RK t x 2 1 tr s t q Q sts x q 2 s t |Q| sr tt tr s rs /2 t t Q t tss r st t rs 1 + /2 s BK (r) ts rs r R

K t

|Q| (BK (/2)) (BK (1 + /2))

|Q| (BK(1+/2))(BK(/2))

= (1+/2)K

(/2)K

(3/)K .

tr t r tr s s t r

r r t s ttrtt

P PP

r

(0, 1) t M N r tr s trs ij r t ij SSub (1/M)

M 1Klog(N

K

),

t stss t P rr K t t rsr t rt 1 2e2M r 1 s rtrr 2 =

2/2 log (42e/) /1Pr

rst t tt t s t r t s x 2 = 1 s s r t st T {1, 2, ..., N} t |T | = K t XT t t Ks ss s t s T s t st ts QT s tt QT XT q 2 1 r q QT r x XT t x 2 1

minqQT

x q 2 /14.

r s s st QT t |QT | (42/)K t rt tsrss r ss st T t t sts QT ttr

Q =

T :|T |=KQT .

r r

(NK

)ss sts T ts r

(N

K

)=

N (N 1) (N 2) (N K + 1)K!

NK

K!(eN

K

)K,

r t st qt s s r trs rt K! (K/e)K |Q| (42eN/K)K t trs r r r t strt sssstrt r rr t s t t

t ts st ts t = /2 t t rst tt t rt

1 2(42eN/K)K eM2/2 , (

1 /2) q 22 q 22

(1 + /

2) q 22, for all q Q. sr tt M stss t

log

(42eN

K

)K Klog

(N

K

)log

(42e

) Mlog (42e/)

1

ts s 1 2e2M s sr A s t sst r s tt

x 2 1 +A x 2, for all x K , x 2 1. r s t s tt A r ts r tt r x K t x 2 1 q Q s tt x q 2 /14 s tt x q K s x XT q QT XT sts x q 2 /14 ts s

x 2 q 2 + (x q) 2 1 + /

2 +

1 +A /14.

r r t s ttrtt

t A s t sst r r s t1 +A

1 + /2 +

1 +A /14 rr

1 +A

1 + /

2

1 /14 1 + ,

s sr r t r qt r qt s r ts

s

x 2 q 2 (x q) 2 1 /

21 + /14 1 , ts t r

r tt t P s t rt t tr s r r t strt sss strt r r t trst ss tt r srs r rss

s rtr ss 6= I s t tr t sts t P tsstt t s s t s tt s r t ts t Ks sss s sts K s r sts t P r r r r sssstrt s rst t rst t t srst ss s ttrt rrt tts t sr r t ss strt s str rts t s s

rrt r r sss strts t s tt t rt s

sts sts t P t rst t t r ss

t r

sts t r s tt s trt ts rs

s s sts t rts t t P t P t s s sts

rsts 1 t t t tt srs rr t t sr t s t stts rr t sts sts t rr

s

s u, v r rt trs

u2 + v2 2u+ v2. Pr

t 21 tr w = [u2, v2]T str s p rs r P t P t t K = 2 w1

2w2r ts t

u2 + v2 2u22 + v22. u v r rt u22 + v22 = u+ v22 s t sr rst

stss t rstrt str rrt t P rr 2K t r r trs u, v K t st srt

|< u,v >| 2Ku2v2.

s tt s t ttrtt

trt t rss s ttrtttst

r r t s ttrtt

P PP

Pr

s u, v K t st srt tt u2 = v2 = 1. u v 2K u v22 = 2 s t P

2 (1 2K) u v22 2 (1 + 2K) . t rr tt

|< u,v >| 14

u+v22 u v22 2K stss t

t 0 rtrr sst {1, 2, ..., N} s tt |0| K r tr u RN 1 s t st rrs t t K rst trs uc

0 st 2 s t st rrs t t t K