Ph.D. Dissertation Defense Modeling and Evaluating Feedback- Based Error Control for Video Transfer PhD Candidate: Yubing Wang - Computer Science, WPI, EMC Corp. Committee: Prof. Mark Claypool - Computer Science, WPI Prof. Robert Kinicki - Computer Science, WPI Prof. Dan Dougherty - Computer Science, WPI Prof. Ketan Mayer-Patel – Computer Science, UNC at Chapel Hill
38
Embed
Modeling and Evaluating Feedback-Based Error Control for Video Transfer
Modeling and Evaluating Feedback-Based Error Control for Video Transfer. PhD Candidate: Yubing Wang - Computer Science, WPI, EMC Corp. Committee: Prof. Mark Claypool - Computer Science, WPI Prof. Robert Kinicki - Computer Science, WPI Prof. Dan Dougherty - Computer Science, WPI - PowerPoint PPT Presentation
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Ph.D. Dissertation Defense
Modeling and Evaluating Feedback-Based Error Control for Video
Transfer PhD Candidate:
Yubing Wang - Computer Science, WPI, EMC Corp.
Committee:
Prof. Mark Claypool - Computer Science, WPI
Prof. Robert Kinicki - Computer Science, WPI
Prof. Dan Dougherty - Computer Science, WPI
Prof. Ketan Mayer-Patel – Computer Science, UNC at Chapel Hill
2Ph.D. Dissertation Defense
Video TransferVideo Transfer
5
Video Frames
InternetInternet
Client
4 3 2 1
Frame Loss
Capacity Constraint
Server
5 3 1
Delay Constraint
Too Late
Error Propagation
4 3 2 15
3Ph.D. Dissertation Defense
Error ControlError Control
5
Video Frames
InternetInternet
Client
4 3 2 1
Server
Retransmission
NACK
3
Change Coding Parameter
Local Concealment
3
4Ph.D. Dissertation Defense
MotivationMotivation
Frame loss degrades video qualityFeedback-based error control techniques use information from decoder to repair
Feedback indicates damage location.Encoder and decoder cooperate in error control process.Better than error control techniques where no interaction between encoder and decoder Major techniques: RPS, Intra Update, RetransmissionChoice and Effectiveness depends on packet loss, RTT, video content and GOP size
No systematic exploration and comparison of impact of video and network conditions on the performance of feedback-based error control techniques
5Ph.D. Dissertation Defense
The DissertationThe Dissertation
Analyze video quality with feedback based error controlDevelop analytical models to predict quality of videos streamed with RPS NACK, RPS ACK, Intra Update or RetransmissionConduct systematic study of effects of reference distance on video qualityValidate analytical models through simulationsAnalysis of loss rate, round-trip time, video content, Group Of Pictures (GOP)Determine choice between RPS NACK, RPS ACK, Intra Update or RetransmissionPublications
“Impact of Reference Distance for Motion Compensation Prediction on Video Quality”, MMCN07“An Analytic Comparison of RPS Video Repair”, MMCN08“Modeling RPS and Evaluating Video Repair with VQM”, IEEE Transactions on Multimedia, 2009, (to appear)
The decoder acknowledges all correctly received frames Only the acknowledged frames are used as a reference Error propagation is avoided entirely Distance from reference frame is reference distance Reference distance increases with round-trip delay Coding efficiency decreases as reference distance increases Video quality degrades as coding efficiency decreases
AssumptionsAssumptionsEach GOB is independent from other GOBs in the same frame.An independent video sub-sequence is referred to as a reference chain.Each GOB is carried in a single network packet.Reliable transmission of feedback messages are assumed. Erroneously-decoded GOBs are repaired by local concealment.Make no assumption on specific local concealment techniques.
1 2 3 4 5 6 7
Assume independent packet loss with a random loss distribution.
In this talk, GOB and Frame is exchangeable.
19Ph.D. Dissertation Defense
Model ParametersModel Parameters
20Ph.D. Dissertation Defense
Modeling of RPS ACKModeling of RPS ACK
The probability of decoding GOB (n) correctly using GOB (n-δ-i) as a reference:
The probability of GOB (n) being successfully decoded is:
p Packet loss probability
Probability of GOB (n-δ-i) being successfully decoded
Round-trip time
Time-interval between two frames
inq
RTTt
INTt
10,)1( niqpp ini
INT
RTT
t
t
pqn 1
ACK(1)
1 2 3 4 5
ACK(2)
21Ph.D. Dissertation Defense
RPS ACK Modeling (cont.)RPS ACK Modeling (cont.)
The expected video quality for n-th GOB:
Average video quality for a GOB encoded using the GOB that is r GOBs backward.
Average video quality for a Intra-Coded GOB
Average PSNR value for a GOB that is repaired using local concealment
nUpUp
nUpqppUQ
n
iin
ii
n
,*)1(
,*)1(
'0
'1
0
rU
0U'U
22Ph.D. Dissertation Defense
RPS NACK -- ModelRPS NACK -- Model The probability of GOB (n) being successfully decoded:
np
nqqq
n
n
iinn
n
,)1(
,1
0,1, inq , --- the probability of decoding GOB (n) correctly
using GOB (n- δ -i) as a reference
1- p
1- p
1- p
1- p1- p1- p1- p
1- p
p
p p
p pp
[1]
[1](1)
[2](1)(1)
[1]
(2)
(2)(1)(1) (3) [3]
GOB 1
GOB 2
GOB 3
GOB 4
(2)
p
p
root
[1]
A
B
C
D
NACK(1)
1 2 3 4 5
NACK(2)
GOB Dependency Tree
23Ph.D. Dissertation Defense
Intra Update -- ModelIntra Update -- Model The probability of GOB (n) being successfully decoded:
np
nqqq
n
INTRAnn
n,)1(
,,1, INTRAnq , -- the probability of decoding GOB (n) correctly using Intra coding
1 2 3 4 5
NACK
Intra-coded
1- p
1- p
1- p
1- p1- p1- p1- p
1- p
p
p p
p pp
GOB 1
GOB 2
GOB 3
GOB 4
p
p
A E
B
root
C
D
F
GOB Dependency Tree
24Ph.D. Dissertation Defense
RetransmissionRetransmission
pNN
pNCC
CNp
pNNC
N
NpppC
RRG
GE
G
RRGE
G
RRE
)1(*
,)1(
)(*)]1(*)(1[* 32
2),1(
11),1(1'1
1
'1
RRNN
RRnn
nNnqUqU
NnqUqUQ
RRRR
Capacity constraint:
The n-th GOB in the reference chain being successfully decoded:
11, RRn
n Nnqq
2,1 RR
Nn Nnqq RR
The expected video quality for GOB (n):
25Ph.D. Dissertation Defense
OutlineOutline
IntroductionBackgroundImpact of Ref. Distance on Video QualityAnalytical Models and Results
Randomly drop controllable number of frames in input sequence based on given loss probabilityBased on given round-trip time and randomly selected lost frames, regenerate video sequenceEncode video sequence generated in step 2 using H.264Measure average PSNR and VQM for encoded H.264 video sequenceCalculate average PSNR and VQM based upon video quality measured in step 4
1(I) 2(P) 5(P) 6(P) 7(P)
RPS NACK, round-trip time = 2 frames, frame 3 is lost
33Ph.D. Dissertation Defense
Validation – RPS NACKValidation – RPS NACK
25
27
29
31
33
35
37
39
41
43
45
0 0.05 0.1 0.15 0.2 0.25 0.3
Loss (Fraction)
PS
NR
(db
)
RTT=80 ms(simulation) RTT=240 ms(simulation)
RTT=80 ms(model) RTT=240 ms(model)
Error bar represents 95% confidence interval
As loss probability or round-trip time increases, the variance is increased
Simulation results are consistent with values predicted by analytical model for both PSNR and VQM
0.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
0 0.05 0.1 0.15 0.2
Loss (Fraction)
VQ
M
RTT=80ms (experiment) RTT=240ms (experiment)
RTT=80ms (model) RTT=240ms (model)
34Ph.D. Dissertation Defense
OutlineOutline
IntroductionBackgroundImpact of Ref. Distance on Video QualityAnalytical Models and ResultsModel ValidationsConclusions
35Ph.D. Dissertation Defense
Major ContributionsMajor Contributions
1. Systematic study of effects of reference distance on video quality for a range of video coding conditions
2. Two utility functions that characterize impact of reference distance on video quality based upon study
3. Modeling prediction dependency among GOBs for RPS NACK and Intra Update using binary tree
4. Analytical models for feedback-based error control techniques including Full Retransmission, Partial Retransmission, RPS ACK, RPS NACK and Intra Update
5. Simulations that verify accuracy of our analytical models
6. Analytic experiments over a range of loss rates, round-trip times and video content using our models
36Ph.D. Dissertation Defense
Future WorkFuture Work
Explore and incorporate other existing video quality metrics or develop a new quality metricInvestigate how local concealment may affect the choice of feedback-based repair techniquesInvestigate the impact of the extra bandwidth consumed by feedback messages on performanceBuild a videoconference system that automatically adapts to the best repair techniques
37Ph.D. Dissertation Defense
ConclusionsConclusionsDegree of video quality degradation is affected by video content
High-motion video sequences starts with lower quality, degrade slower.Low-motion video sequences starts with higher quality, degrade more rapidly.Mathematical Characterization of the relationship between video quality and reference distance:
PSNR:
VQM:
Analytical models reveal: RPS NACK performs best in low lossRPS ACK performs best in high loss, worst in low lossRPS NACK outperforms RPS ACK over a wider range for low motion videos than for high motion videosRetransmission performs worst in high loss Intra Update performs as well as RPS NACK as RTT increases
bxay )ln(baxy
38Ph.D. Dissertation Defense
AcknowledgeAcknowledge
Prof. Claypool and Prof. Kinicki
Prof. Dougherty
Prof. Mayer-Patel from UNC at Chapel Hill
Faculty/Staff of Computer Science Dept., WPI
Huahui Wu, Mingze Li, Feng Li, and everyone from PEDS and CC groups