Li-Wei Kang and Chun-Shien LuInstitute of Information Science, Academia Sinica
Taipei, Taiwan, ROC{lwkang, lcs}@iis.sinica.edu.tw
April 20092009 IEEE International Conference on Acoustics, Speech, and Signal Processing
(ICASSP2009, Taipei, Taiwan, ROC)
Distributed Compressive Video Sensing
Distributed Compressive Video Sensing April 24, 2009 2
Distributed Source Coding
[bits]XR
[bits]YR
H X
H Y
|H Y X
|H X Y
,X YR R H X Y
Vanishing error probabilityfor long sequences
Vanishing error probabilityfor long sequences
No errorsNo errors
[Slepian and Wolf, 1973]
Distributed Compressive Video Sensing April 24, 2009 3
Distributed Video Coding
“Motion JPEG”
Decoder
“Motion JPEG”
Encoder
X’X
Wyner-ZivInterframe Decoder
Wyner-ZivIntraframe Encoder
Side Information
Y
[Girod, 2006]
Distributed Compressive Video Sensing April 24, 2009 4
Distributed Video Coding
• The statistical dependency between X and YLaplacian distribution
,2
ii YXii eYXp
ii YX
2
Distributed Compressive Video Sensing April 24, 2009 5
Compressive Sensing
• When data is sparse/compressible, one can directly acquire a condensed representation with no/little information loss
• Random projection will work
[Baraniuk, 2008]
Distributed Compressive Video Sensing April 24, 2009 6
Compressive Sensing
• Directly acquire “compressed” data
• Replace samples by more general “measurements”
[Baraniuk, 2008]
Distributed Compressive Video Sensing April 24, 2009 7
Compressive Sensing
• y = Фx = ФΨθ = Aθ
y Ф Ψ θ
x = Ψθ
A = ФΨ
N×1N×N
M×NM×1
[Baraniuk, 2008]
Distributed Compressive Video Sensing April 24, 2009 8
Measurement Matrix
• Scrambled block Hadamard ensemble (SBHE)partial block hadamard transform and random column
permutation
Ф = QMWPN
L. Gan, T. T. Do, and T. D. Tran, “Fast compressive imaging using scrambled hadamard ensemble,” in Proc. of European Signal Processing Conf., Lausanne, Switzerland, August 2008 (EUSIPCO2008).
Distributed Compressive Video Sensing April 24, 2009 9
Signal Reconstruction
• The convex unconstrained optimization problem
• Can be seen as a maximum a posteriori criterion for estimating θ from
y = A θ + n, where n is white Gaussian noise
1
2
2A
2
1min
y
Distributed Compressive Video Sensing April 24, 2009 10
Signal Reconstruction
• Signal recovery from random measurementsGradient projection for sparse reconstruction (GPSR)Two-step iterative shrinkage/thresholding algorithm (TwIST)Orthogonal matching pursuit (OMP)
M. A. T. Figueiredo, R. D. Nowak, and S. J. Wright, “Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems,” IEEE J. of Selected Topics in Signal Processing, vol. 1,no. 4, pp. 586-597, Dec. 2007.
J. M. Bioucas-Dias and M. A. T. Figueiredo, “A new TwIST: two-step iterative shrinkage/thresholding algorithms for image restoration,” IEEE Trans. on Image Processing, vol. 16, no. 12, pp. 2992-3004, Dec. 2007.
T. Blumensath and M. E. Davies, “Gradient pursuits,” IEEE Trans. on Signal Processing, vol. 56, June 2008.
Distributed Compressive Video Sensing April 24, 2009 11
Distributed Compressive Video Sensing
• Measurement matrix Ф: scrambled block Hadamard ensemble (SBHE)
• Sparse basis matrix Ψ: DWT• Video signal sensing (encoder): general random
projection• Video signal recovery (decoder)
Key frame: GPSR with default settingsCS frame
side information generation (motion compensated interpolation)GPSR with the proposed initialization and the proposed
termination criteria
Distributed Compressive Video Sensing April 24, 2009 12
Distributed Compressive Video Sensing
CS measurementyt = Ф xt
Each frame xtMeasurement vector (compressed frame) yt
Measurement vector yt
for each non-key frameInitialization by SI generation
Reconstructed previous key frames
GPSR optimization
Stopping criteria (a)-(c)
Non-stop Stop
ttx ~~ Reconstructed non-key frame
tx~
Compressive video sensing
Video signal recovery
Distributed Compressive Video Sensing April 24, 2009 13
Distributed Compressive Video Sensing
• At the decoder, for a CS frame xt = Ψθt
its side information St = ΨθSt can be generated from its previous reconstructed key frames
• Proposed initializationinitial solution at the 0-th iteration:
• α(xt, St): the Laplacian parameter of (xt- St)
,~ 0
Stt ,~ 0tt Sx
Distributed Compressive Video Sensing April 24, 2009 14
Key frame (t - 1)
Non-key frame t
Key frame (t + 1)
yt-1 = Φxt-1
with higher MR
yt = Φxt
with lower MR
yt+1 = Φxt+1
with higher MR
GPSR reconstruction
GPSR reconstruction
Proposed Modified
GPSR reconstruction
Side information (t)
Reconstructed frame (t)
Reconstructed frame (t-1)
Side information generation
Reconstructed frame (t+1)
Distributed Compressive Video Sensing April 24, 2009 15
Distributed Compressive Video Sensing itx~
xt St
α(xt, )
α(xt, St)
,2
1min
1
2
21 tttt AyFt
22 StttF
it
it
it FWFWF ~~~
2211
itx~
α( , St) itx~
Distributed Compressive Video Sensing April 24, 2009 16
Proposed Termination Criterion
• First:
• Second:
• Third:
T
Sx
SxSx
ti
t
ti
tti
t
,~
,~,~
1
1
0~~ 1 i
ti
t FF
Fi
t
it
it
TF
FF
1
1
~
~~
Distributed Compressive Video Sensing April 24, 2009 17
Proposed Termination Criterion
• MR is low (MR ≤ 20%): if the First criterion with Tα = 0.9 is satisfied, the algorithm will stop
• MR is middle (20% < MR ≤ 70%): if the First criterion with Tα = 0.05 or the Second criterion is satisfied, the algorithm will stop
• MR is high (MR > 70%): if the Third criterion with TF = 0.001 is satisfied, the algorithm will stop
Distributed Compressive Video Sensing April 24, 2009 18
Simulation Results
• Foreman and Coastguard CIF video sequences with 300 Y frames (352×288 = 101376 samples for each Y frame) and GOP size = 3 (Key, Non-key, Non-key, Key, …)
• The three approaches for comparison (all with default settings)GPSR, TwIST, OMP
• For OMP, block size = 32×32 suggested by V. Stankovic, L. Stankovic, and S. Cheng, “Compressive video sampling,” in Proc.
of European Signal Processing Conf., Lausanne, Switzerland, August 2008 (EUSIPCO2008).
Distributed Compressive Video Sensing April 24, 2009 19
Simulation Results
Distributed Compressive Video Sensing April 24, 2009 20
Simulation Results
Distributed Compressive Video Sensing April 24, 2009 21
Simulation Results
The reconstruction complexities for the Foreman sequence
Distributed Compressive Video Sensing April 24, 2009 22
Simulation Results
The PSNR performance at different reconstruction complexities for the Foreman sequence
Distributed Compressive Video Sensing April 24, 2009 23
Simulation Results
(a) Side information (b) Reconstructed frame
Distributed Compressive Video Sensing April 24, 2009 24
Simulation Results
The reconstructed Foreman sequences (352×288 for each frame) at measurement rate (MR) = 0.3 using (a) GPSR (gradient projection for sparse reconstruction) (average PSNR = 27.68 dB) (average reconstruction time = 15.14 seconds per frame); and (b) our DCVS (average PSNR = 29.48 dB) (average reconstruction time = 3.68 seconds per frame) (This example shows the 54-th frame).
Distributed Compressive Video Sensing April 24, 2009 25
Conclusions
• The proposed DCVS approach exploits the two characteristicsdistributed video coding (DVC)compressive sensing (CS)
• The proposed DCVS can outperform or be comparable with the three existing approaches for comparison, especially at lower measurement rates
• The proposed DCVS can significant outperform the three existing approaches at the same reconstruction complexity