YOU ARE DOWNLOADING DOCUMENT

Please tick the box to continue:

Transcript
Page 1: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

The Role of Sensory Psychology to VoIP Rate Adaptation: A Study on Skype Calls

Skype Group, NSLABINFOCOMM2012(Hopefully)

Page 2: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Page 3: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Page 4: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Page 5: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Tx/Rx ContentBitrateJitterPacket Loss RateQuality of

Service(QoS)

Page 6: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Page 7: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Mean Opinion Score (MOS)

Reaction TimeReactivity/

ResponsivenessQuality of Experience

(QoE)

Page 8: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

QoEMOSReaction

TimeReactivity

QoSTx/Rx

ContentBitrateJitterPacket

Loss

Page 9: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Page 10: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Related WorksOn the TCP-Friendliness of VoIP Traffic, Tian Bu et al., INFOCOM2006Disprove the conjecture that VoIP is not TCP-

Friendly after taking the user back-off mechanism into account.

User back-off: real time appswill drop out completely ifthe user perceived unacceptable quality due tonetwork congestion.

Page 11: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Related WorksQuantifying Skype User Satisfaction, K. T. Chen et al., SIGCOMM2006The User Satisfaction

Index(USI)Using traditional metrics

(RTT, jitter, bitrate) to infer user-centric metrics (reactivity, duration, MOS.)

Allow real-time and user-centric adaptation .

Page 12: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Related WorksCould Skype be More Satisfying?, T. Y. Huang et al., IEEE Network 2010Skype’s adaptation does not take the

individual codec and packet loss patterns into consideration.

The inconsistency in voice quality results in over-utilization of bandwidth.

Page 13: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Related WorksAn Experimental Investigation of the CongestionControl Used by Skype, L. D. Cicco et al., WWIC 2007Skype’s slow adaptation to bandwidth drop

causes coexisting TCP flows to be suppressed.

Skype’s over-utilization of bandwidth causes massive fluctuation on bitrate, which may result in user frustration.

Page 14: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

MotivationClearly, there are many to be improved on

Skype’s rate adaptation algorithm.Skype’s over-utilization of bandwidth is

1) wasting network resource and2) threating other applications at the risk of3) producing massive fluctuation on quality.

Our major assumption: This selfish deed of Skype is actually NOT helping user satisfaction. Users dislike changes on audio quality, even if they actually increase the average rate.

Page 15: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Roadmap

Preliminary

Experiments

Large-Scale

Experiments

Evaluation

Conclusion

Page 16: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Roadmap

Preliminary

Experiments

Large-Scale Experiments

Evaluation

Conclusion

Page 17: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

GoalConfirm our assumption about user’s

impression towards audio quality fluctuation.Get a ballpark idea of the possible

relationships between parameter and MOS. (formulation)

Page 18: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

MethodExploit audio encoder/decoder to create

audio track with fluctuating qualities (bitrates.)

We will focus on Silk in all following experiments due to its1) potential of domination of VoIP codec and2) flexibility on fine-tuning bitrate.

Page 19: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Test Tracks

High rate

Low rate

Time

∆T ∆T

Bitrate

Page 20: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

PCM

PCM

PCMPCM

Test Tracks SetupHeade

rPCM

Encoder

Encoder

Decoder

Decoder

High rate

Low rateCombin

e

∆T

Header

Page 21: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Formulation: GoalWe target three variables, High Rate, Low

Rate, ∆T, that affect the user’s perception.Interactions between the three variables.Exp1: Find the relation between fixed bitrate

and MOS.

Exp2: Find the formula that combines the three dimensions with MOS.𝑓 𝐹𝐿𝑈𝐶 (h𝑟 , 𝑙𝑟 ,∆𝑇 )=𝑀𝑂𝑆

𝑓 𝐹𝐼𝑋 (𝑏𝑟 )=𝑀𝑂𝑆

Page 22: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Formulation: Test Tracks SetupThe maximum

and minimum bitrate of Silk are 40.6 and 5.6 kbps.

We chose 10 rates uniformly from the interval.

Experiment 1: Fixed Rate vs. MOSq1 40.6 kbpsq2 36.6 kbpsq3 32.8 kbpsq4 28.9 kbpsq5 25.0 kbpsq6 21.1 kbpsq7 17.2 kbpsq8 13.3 kbpsq9 9.5 kbpsq10 5.6 kbps

Page 23: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Formulation: Test Tracks SetupThe source track

is 30 seconds long. We set ∆T as its factors.

We picked 4 rates (q1, q4, q7, q10) to be the candidates of high and low rates.

HR LR

∆T10 sec5 sec3 sec2 sec1 sec

{40.6, 28.9, 17.2, 5.6} kbps

Experiment 2

Page 24: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Formulation: Test Tracks SetupFollows the ITU

recommendations.Four voices: 2 male and 2 female.Sentences with no coherent plot.30 seconds, 44.1 kbpsReference tracks (original 44.1 kbps) are

inserted in the test cases in order to provide a standard of rating.

The tracks of Exp1&2 are mixed up and the order of rating for each subject is randomly picked.

Page 25: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Formulation: Results (Exp1)

Page 26: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Formulation: Analysis (Exp1)The plot can be fitted by a shifted logarithm

function.The shift is due to the lower boundary of

human audio perception.Observed rapid MOS drop with lower bitrate.

𝑓 𝐹𝐼𝑋 (𝑏𝑟 )=γ ×ln (𝑏𝑟 −𝛼 )+ 𝛽

Page 27: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Why Logarithms?Weber–Fechner law

 The smallest noticeable difference in stimulus (the least difference that the test person can still perceive as a difference,) was proportional to the starting value.

The law is shown plausible in a wide range of human perceptions including hearing, vision, taste, sense of touch and heat, and even temporal and spatial cognitions.

𝑑𝑝=𝑘 𝑑𝑆𝑆 ❑

⇒𝑝=𝑘× ln (𝑆 )+𝐶

Page 28: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Formulation: Results (Exp2)

Adapting to an “optimal rate” and ignoring how users feel about changes might be over-optimistic.

Page 29: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Formulation: Results (Exp2)

Page 30: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Formulation: Analysis (Exp2)R2 of logarithm regression of each track are

generally higher than 0.9.An outlier is discovered: 28.9+17.2. This is

attributed to:1) the similarity of the two bitrates and 2) they both reside in middle- or low-level qualities.

The phenomena is also supported by the ANOVA test on the similarity of 28.9 and 17.2 kbps data sets (p = 0.2155).

Page 31: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Formulation: Analysis (Exp2)In short, the MOS to frequency of rate

change relationship, although shows logarithmic behavior in general, depends on the magnitude of rate changes.

𝑓 𝐹𝐿𝑈𝐶 (h𝑟 , 𝑙𝑟 ,∆𝑇 )=𝑆𝐶𝐴𝐿𝐸 (h𝑟 , 𝑙𝑟 )× ln (∆𝑇 )+𝑆𝐻𝐼𝐹𝑇 (h𝑟 , 𝑙𝑟 )

Page 32: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Some Guessing About the SubroutinesSCALE()Directly associated with the

difference between hr and lr. The results in Fig. 7 provide evidence to this inference: same average bitrate, different magnitudes.

Positive correlation between the scale of regression function and rate change magnitude.

Another intention of SCALE() is to deal with small magnitude tracks that does not fit well.

Page 33: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Some Guessing About the SubroutinesSHIFT()Cope with human’s expectation. As ∆T grows, the effect of fluctuation

decreases and the variable-rate case will become indiscernible to a fixed-rate version.

We call this imaginary, fixed rate equivalent the dominant quality of the fluctuation. (dominant quality ≠ average quality)

The dominant quality is the exact quality a user expects to observe when the negative impact of fluctuation diminishes.

Page 34: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Roadmap

Preliminary

Experiments

Large-Scale Experiments

Evaluation

Conclusion

Page 35: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Large-Scale Experiments: GoalWe need massive data to construct the detail

of our formulas:- verify the structures of our formulas.- factors in the fixed-rate formula:

- subroutines in the variable-rate formula: SCALE(hr,lr) & SHIFT(hr,lr)

𝑓 𝐹𝐼𝑋 (𝑏𝑟 )=γ ×ln (𝑏𝑟−𝛼 )+ 𝛽

Page 36: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

MethodSame source track.Nine levels of quality are exponentially

chosen.Five levels of rate changing frequency

{1,2,3,5,10}.127 participants.Score calibration with hidden reference

track.ITU Recommendations

Page 37: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Results: Formula Structure

Figural support:Non-parallel plots

Statistic support:ANOVA of interactivity (p=8e-14)

𝑓 𝐹𝐿𝑈𝐶 (h𝑟 , 𝑙𝑟 , ∆𝑇 )=𝑆𝐶𝐴𝐿𝐸 (h𝑟 , 𝑙𝑟 )× ln (∆𝑇 )+𝑆𝐻𝐼𝐹𝑇 (h𝑟 , 𝑙𝑟 )

Page 38: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Results: Fixed-rate Formulaα=4.091β=1.515γ=1.000

Another interesting discovery: lower bound of Silk.

Page 39: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Results: SCALENot surprisingly, SCALE subroutine is

positively correlated with magnitude.

Page 40: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Results: SHIFTThis is more tricky… due its relationship with

user expectation.Base on our definition of dominant quality:

Where D(hr,lr) is the MOS of the dominant quality of rate changing pair: (hr,lr)

Page 41: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

SHIFT (Conti.)First we plot the estimated MOS of fixed hr,

fixed lr, and D.There is an apparent difference when

hr<14.1.Not surprising, we

have already seen this reaction of MOS when a track ispaired by two similar, inferior rates.

Page 42: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

SHIFT (Conti.)We plot them again in

percentages:hr = 100%lr = 0%

We can then see a clear pattern when we group the tracks by their MOS magnitudes.

Page 43: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

SHIFT (Conti.)Finally…

𝑓 𝐹𝐿𝑈𝐶 (h𝑟 , 𝑙𝑟 ,∆𝑇 )=𝑆𝐶𝐴𝐿𝐸 (h𝑟 , 𝑙𝑟 )× ln (∆𝑇 )+𝑆𝐻𝐼𝐹𝑇 (h𝑟 , 𝑙𝑟 )

Page 44: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Roadmap

Preliminary

Experiments

Large-Scale Experiments

Evaluation

Conclusion

Page 45: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

EvaluationIt is not surprising that the formula outcomes

of preliminary and large-scale experiments fit their ground truth.

We need a third dataset for verifying purpose.

The Verifying ExperimentsDifferent source track: conversation of two

males.Different length: 60 secondsDifferent rates: {44.1, 11.8, 6.4} kpbsDifferent frequencies: {1,5,10} seconds

Page 46: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Results

Page 47: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Roadmap

Preliminary

Experiments

Large-Scale

Experiments

Evaluation

Conclusion

Page 48: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

ConclusionVerified the user experience versus

magnitude of rate change relationship exhibits the log-like behavior, echoing the Weber’s theory.

Discovered that experience versus frequency of rate change relationship also exhibits the log-like behavior.

Derived the closed form model of user experience to rate changes with 97%+ goodness of fit.

Page 49: The Role of Sensory Psychology to VoIP  Rate Adaptation :  A  Study on Skype Calls

Network and Systems Laboratorynslab.ee.ntu.edu.tw

Thanks For Your Attention


Related Documents