Department of Electrical Engineering Stanford University Xiaoqing Zhu EE372 Final Project Congestion-aware Bite Allocation For Multi-path Video Streaming Over Ad Hoc Wireless Networks
Department of Electrical EngineeringStanford University
Xiaoqing Zhu
EE372 Final Project
Congestion-aware Bite Allocation For Multi-path Video Streaming Over Ad Hoc Wireless Networks
X. Zhu: EE372 Project Presentation December 3, 2003
Outline Video streaming over ad hoc wireless network
Bit allocation over multiple paths, across many frames Problem formulation Modeling of encoder distortion-rate function Modeling of network congestion effect Two allocation modes: scalable vs. repetition
Optimal bit allocation via Two heuristic schemes Gradient descent Iterative descent
Experimental results Gaussian samples over M/M/1 channels Video sequence over simulated network
X. Zhu: EE372 Project Presentation December 3, 2003
Video Over Wireless Ad Hoc Networks Ad Hoc Wireless Networks:
Wireless nodes self-configure into a network Restricted bandwidth and power resources, adverse channel
conditions=> limited and varying transmission rate on each link
Many applications with multimedia: search-and-rescue, highway automation, Internet access etc.
Video streaming challenges: High bandwidth requirement Stringent delay constraint
Combined: the self-congesting effect
X. Zhu: EE372 Project Presentation December 3, 2003
Multi-path Video Streaming
Benefit of path diversity: Expansion of available bandwidth More robust to correlated packet losses
[J. Apostolopoulos, 1999][S. Mao et al., 2003]
Need bit allocation over multiple paths,
across different frames
X. Zhu: EE372 Project Presentation December 3, 2003
Problem Formulation: Notations
Bit allocation matrix [Rml] Path index: m = 1, … M Frame index: l = 1, …, L
Packet playout deadline: Tplayout,
Packet size: B Video frame rate: Vfps
Number of nodes in the network: N Link capacity from node i to node j: Cij, i,j = 1, …, N
Cross traffic bitrate from node i to node j: Fij, i,j = 1, …, N
X. Zhu: EE372 Project Presentation December 3, 2003
Problem Formulation: Optimization
To find the optimal rate allocation matrix R :
Minimize Exp [Ddec] = f(D(l)enc(R), Pm(R, Vfps, Tplayout,
B))Subject to: Rml ¸ 0
Rml · Rml(max)
(Vfps/L)Ll=1(Rml) · Cij-Fij, (i,j)2Pm
m = 1, …, M, l = 1, …, L
Where D(l)
enc(.) is encoder distortion-rate function for the matrix R
Pm(.) is the function for packet drop rate due to congestion
f(.) denotes taking expectation over D(l)enc according to Pm(.)
X. Zhu: EE372 Project Presentation December 3, 2003
Wavelet Video Coder
TemporalWavelet
Transform
TemporalWavelet
Transform
Spatial Wavelet
Transform
Spatial Wavelet
Transform
7654
321
0
HH
LLL LLHLH
LH
Original
video framesH
HHHH
HHHH
HHHH
HHH
EmbeddedQuantization &Entropy Coding
EmbeddedQuantization &Entropy Coding
[Taubman & Zakhor, 1994] [Ohm, 1994] [Choi & Woods, 1999] [Hsiang & Woods, VCIP ’99] . . .
Scalable bitstream
X. Zhu: EE372 Project Presentation December 3, 2003
Encoder Distortion Model Now l indexes the sub-band image in the wavelet coefficient
domain High rate approximation of transform vector quantization:
D(R) = Mi=1Ki2-2Bi/Ni, where Ki , (Ni/N)Cni\
||pi||Ni/(2+Ni) ,
[M. Goldburg 1993] Therefore, model the RD curve of each sub-band image as
sum of exponentials:
d (l)enc(r) = K
i=1i e-i r , l = 1, …, L
Further assume additive contribution
from each sub-band image:
Exp(Ddec) = Ll=1Exp(D(l)
enc(R))
X. Zhu: EE372 Project Presentation December 3, 2003
Network Congestion Model
Assume independent delays on each path, congestion dominated by the bottleneck link Cm, Fm for path m.
M/M/1 queuing model for delay distribution: Exponential: Xm ~ fX(x)=1/m e-mXm
Average delay: Exp(Xm)=1/m = B/(Cm-Fm-Sm)
Drop rate: Pm = e-m Tplayout = e -(Cm-Fm-Sm)Tplayout/B
Usually the drop rate is overestimated, need a correction factor
X. Zhu: EE372 Project Presentation December 3, 2003
Code Mode: Scalable vs. Repetitive
Scalable rate allocation: Non-overlaping information on each path Higher efficiency in error-free situation D(l)
enc(Ra, Rb) = d(l)enc(Ra + Rb)
Repetitive rate allocation: Repeating the starting part of the bitstream on both
paths Sacrifice coding efficiency Better robustness to channel errors and packet loss D(l)
enc(Ra, Rb) = min(d(l)enc(Ra), d(l)
enc( Rb))
Path 1
Path 2
Path 1
Path 2
X. Zhu: EE372 Project Presentation December 3, 2003
Heuristic Scheme
Rate allocation across all frames minimizing the Lagrangian cost function for all sub-band images:
R(l)total= argminR D(l)
enc(R) + R
Rate allocation over each path w.r.t. the bottleneck link A: proportional to residual capacity B: inverse water-filling: equal residual capacity
Optimal source coding strategy, heuristic allocation over multiple paths
X. Zhu: EE372 Project Presentation December 3, 2003
Cost function non-convex w.r.t. the rate matrix R Cost function convex w.r.t. individual Rml’s
Initialize the rate matrix using the heuristic scheme
Search for a local minimum: Gradient descent algorithm Iterative descent algorithm
Alternatives: random initialization
Optimal Scheme Via Gradient Descent
X. Zhu: EE372 Project Presentation December 3, 2003
Numerical calculation with i.i.d Gaussian samples: Denc(R) = 2 e-2R
Pm(R) = e -(Cm-Fm-Sm)Tplayout/B, where Sm =(Vfps/L) Ll=1Rml
Randomly chosen capacity and cross traffic rate for M=2 channels
Network simulation with video: Foreman QCIF video sequence at Vfps=30 frame/second 3-D wavelet-based coded scalable bistream, L = 300 frames A network with 15 randomly placed nodes, M = 2 paths Link capacities calculated based on Shannon’s formula for
AWGN Cross traffic randomly fills up to 70% of link capacity Simulation performed with ns-2 (http://www.isi.edu/nsnam/ns/
)
Experimental Setup
X. Zhu: EE372 Project Presentation December 3, 2003
Gaussian Results: Scalable Mode
0 1 2 3 4 5 6 7 80
10
20
30
40
50
60
70
80
90
100
Rate (bit)
Dist
ortio
n E(
X2 )
Distortion-Rate Performance for Gaussian, Residual Channel Rate = (4.29,3.16), Scalable
encoderheuristic
Aheuristic
Boptimal
0 1 2 3 4 5 60
10
20
30
40
50
60
70
80
90
100
Rate (bit)
Dis
torti
on E
(X2 )
Distortion-Rate Performance for Gaussian, Residual Channel Rate = (1.74,3.61), Scalable
encoderheuristic
Aheuristic
Boptimal
Variance: 2 = 100;Residual link capacity:Channel 1: 4.29 bits/useChannel 2: 3.16 bits/use
Variance: 2 = 100;Residual link capacity:Channel 1: 1.74 bits/useChannel 2: 3.61 bits/use
X. Zhu: EE372 Project Presentation December 3, 2003
Gaussian Results: Repetition Mode
0 1 2 3 4 5 6 7 8 90
20
40
60
80
100
120
Rate (bit)
Dis
torti
on E
(X2 )
Distortion-Rate Performance for Gaussian, Residual Channel Rate =( 5.47, 2.95), Repetition
encoderheuristic
Aheuristic
Boptimal
0 2 4 6 8 10 12 140
20
40
60
80
100
120
Rate (bit)
Dis
torti
on E
(X2 )
Distortion-Rate Performance for Gaussian, Residual Channel Rate =( 6.53, 5.86), Repetition
encoderheuristic
Aheuristic
Boptimal
Variance: 2 = 100;Residual link capacity:Channel 1: 5.47 bits/useChannel 2: 2.95 bits/use
Variance: 2 = 100;Residual link capacity:Channel 1: 6.53 bits/useChannel 2: 5.86 bits/use
X. Zhu: EE372 Project Presentation December 3, 2003
Video Results: Repetition Mode
Sequence : Foreman QCIF (Y)
Sequence length : 300
Frame rate : 30 fps
GOP length: 16 frames
Temporal trans.: 1-level Haar
Spatial transf.: 3-level bi-9/7
Playout deadline : 150 ms
Packet Size: 1000 bytes
Random loss rate: 0 %
Simulation time: 10,000 seconds
No. of realizations: 39
Figure: scalable
0 50 100 150 200 250 300 350 40010
15
20
25
30
35rate vs. psnr
Rate (kbps)
PS
NR
(dB
)
encoderheuristicoptimal
Residual channel rates = [141.89, 258.15] T
playout = 150 ms
Packet size = 1000 bytes Code mode = 'repetition'
PSNR(dB)
Model simulation
Heuristic 30.45 30.49Optimal 31.09 31.28
X. Zhu: EE372 Project Presentation December 3, 2003
Video Results: Scalable Mode
0 50 100 150 200 250 300 350 4000
5
10
15
20
25
30
35rate vs. psnr
Rate (kbps)
PS
NR
(dB
)
encoderheuristicoptimal
Residual channel rates: [141.89, 258.15] kbps T
playout = 150 ms
Packet size = 1000 byte Code mode = 'scalable'
Sequence : Foreman QCIF (Y)
Sequence length : 300
Frame rate : 30 fps
GOP length: 16 frames
Temporal trans.: 1-level Haar
Spatial transf.: 3-level bi-9/7
Playout deadline : 150 ms
Packet Size: 1000 bytes
Random loss rate: 0 %
Simulation time: 10,000 seconds
No. of realizations: 39
PSNR(dB)
Model simulation
Heuristic 32.76 33.02Optimal 33.30 33.59
X. Zhu: EE372 Project Presentation December 3, 2003
Summary Optimize rate-allocation considering both the encoder rate-
distortion trade-off and the effect of packet drop rates due to network congestion
Modeling of encoder distortion-rate performance based on high-rate quantization theory
Modeling of network congestion effect based on M/M/1 queuing theory
Initialize from a heuristic scheme Optimization via gradient descent or iterative descent
Numerical results for Gaussian random samples Experimental results for video streaming over simulated ad hoc
wireless network, gain from heuristic scheme around 0.5dB in PSNR for received video quality
X. Zhu: EE372 Project Presentation December 3, 2003
Problem Formulation: Objective Function
On the mth path: Data rate: Sm = Sm(Vfps, R(m,:)) Set of links: Pm = {(i,j)|path m includes nodes i and j} End-to-end delay distribution:X ~ fX(x) depends on
{Cij|(i,j)2 Pm}, {Fij2 Pm}, and Sm
Packet drop rate: P(m)= Pr{X>Tplayout} =Pm(R, Vfps, Tplayout ,B)
For the lth frame: Encoder performance: D(l)
enc(r)
Expected distortion at the decoder: Ddec = f(Denc(R), P(R, Vfps, Tplayout, B)) Depends on the code mode: scalable vs. repetition