Presentation Title 1 1/31/2014 Lucent Technologies - Proprietary RTP Scalability in Large Multicast Groups Jonathan Rosenberg High Speed Networks Research.

Presentation Title104/10/23

Lucent Technologies - Proprietary

RTP Scalability in Large Multicast Groups

Jonathan Rosenberg

High Speed Networks Research11/10/97

Presentation Title204/10/23

Lucent Technologies - Proprietary

Contents• RTP and RTCP Primer

• Large scale applications

• Problems– Congestion

– State Storage

– Delay

– BYE Storms

– Premature Timeouts

• Reconsideration– Conditional

– Unconditional

– Reverse

– BYE

• Polling

• Group Size Estimation

• Conclusions

Presentation Title304/10/23

Lucent Technologies - Proprietary

RTP and RTCP• RTP provides for

– Real time transport

– Resequencing

– Payload type identification

– Intra and Inter media synchronization

– Encryption

– Multicast• Per User demultiplexing - SSRC

• RTP does not– Provide QoS

– Require RSVP

• RTP is a framework– Specific payload formats defined

for H.263, MPEG etc.

– UDP Port numbers based on application

Presentation Title404/10/23

Lucent Technologies - Proprietary

RTCP• Real Time Control Protocol

• RTP port + 1

• Used for– QoS Reporting

• Sender reports: packets sent, bytes sent

• Receiver reports (per sender): loss, delay, jitter observed; instantaneous and cumulative

– Media Synchronization• NTP and RTP Timestamp correlation

– Loose Session Control• Hello, Bye messages

• SDES - email, username, CNAME, etc.

Presentation Title504/10/23

Lucent Technologies - Proprietary

Large Scale Applications• Cable Distribution

– one sender, tens of thousands of receivers

– QoS reporting critical

– Nielsen - group size estimation

• Distributed real time sensors

• Van Jacobson RLM - Receiver Driven Layered Multicast

• MBONE concerts, talks

Presentation Title604/10/23

Lucent Technologies - Proprietary

Achieving Scalability• RTP

– RTP scales if number of data senders is small or controlled

– Generally not a problem

• RTCP– Both data senders AND

receivers send reports

– Reports are periodic

– Period must scale with group size to avoid implosion

– Period is defined to scale linearly

• Each system listens to group and counts number of other users heard from - L(t) is group size estimate

• Period is set to CL(t); C depends on application

• Works fine if group size is static

• $10M question: How does this work when L(t) is dynamic?

Presentation Title704/10/23

Lucent Technologies - Proprietary

Other Applications• Problem is general; applies when

– Some unkown number of users wish to share a bandwidth resource equally

– The users are on a shared medium, and can hear each other, like multicast

– The number of users grows/shrinks over time

• Examples– CBR traffic on Ethernet

– Route updates among BGP peers

– Distributed Sensors

Presentation Title804/10/23

Lucent Technologies - Proprietary

RTCP Intervals• Require RTCP reporting to scale with

multicast group size

• Total RTCP bw should be no more than 5% of session bw

• Divide 5% among participants - 75% senders, 25% receivers

• Algorithm:– At time t, emit packet

– Compute interval T

– Schedule packet for t+T

– Repeat at t+T

• TMIN is 2.5s for first packet, 5s for all others

• Add randomization to avoid synchronization

T TN pktsz

bw

T T

T T

d MIN

d

d

max ,. .

,

,

05 75

1

2

3

2

1 1

Presentation Title904/10/23

Lucent Technologies - Proprietary

Problems• Congestion

– Simultaneous Joins (ER at 10pm)

– RTCP Interval based on group size estimate

– N users, each think there is only ONE group member

– Multicast flood over initial 2.5s interval - all N users send

– Packet losses -> further congestion

• State Storage– To count group size, must store state for each other group member

– Set top boxes - memory size limitations

Presentation Title1004/10/23

Lucent Technologies - Proprietary

Problems• Delay

– Frequency of reports from any user becomes small

– Average loss reports over large intervals useless

– Time to converge on group size exceeds membership times

– DVMRP cache times -> EVERY RTCP packet is flooded!

• Premature Timeouts– Must timeout members if you haven’t heard from them in M transmission

intervals

– Drop in group membership reduces transmission interval

– Sharp drop can cause a user to prematurely timeout other users

• BYE Storms– Many users simultaneously leave and send a BYE message

Presentation Title1104/10/23

Lucent Technologies - Proprietary

Reconsideration• Avoid Congestion

• Last packet send time was tn-1

• About to send at time tn

• Reconsider if number of users changed since tn-1

• Compute interval based on new observation -> T’, add to tn-1. If result is less than current time, send now, else reschedule for tn-1 + T’

• Conditional: As above

• Unconditional: Recompute even if number of users unchanged

• Backwards compatible

• Effectiveness grows with usage

• Safe

tn-1 tn

3 new users observed

tn-1 + T’

Presentation Title1204/10/23

Lucent Technologies - Proprietary

Simulation Assumptions• 28.8 kbps access links downstream

• Infinitely fast upstream links

• Access server buffers = 100kB per user

• Network adds delay uniform between 0 and 600ms

• Full connectivity - multicast join latency not considered

• Step join 10K users at t=0. Each have a homogeneous view of state

Title: rtcp_net_.figCreator: fig2dev Version 3.1 Patchlevel 2CreationDate: Sat Feb 1 16:11:38 1997

Presentation Title1304/10/23

Lucent Technologies - Proprietary

Packets SentTitle: Creator: gnuplotCreationDate:

Presentation Title1404/10/23

Lucent Technologies - Proprietary

Analytical Results• Can derive a differential

equation for L(t) given a step join!

• For small t, solution is linear when there is no network delays

• Can also show that packet rate is roughly derivative of L(t)

• Gives an estimate of network bandwidth

L tt

C

1

Presentation Title1504/10/23

Lucent Technologies - Proprietary

Add Delay• Real network has delay - significant impact on

performance

• Arrival of packets used to increase group size estimate and back-off -> network delays cause initial bunch of senders not to see packets

• Result is a spike of packets

• After packets arrive, causes rapid increase in group size estimates

• Result is a cessation of packet transmission: plateau effect

• What is size of spike?

• Again, analytic solutions exist!

• Let D be fixed network delays, m is modem rate in packets/s

• In limit of N large, distribution of delays doesn’t matter!

• Conditional - first a linear growth in packets, then a growth as t2 - much smaller

NN

TD

mpMIN

2

1

t

Packets S

ent

Np

Presentation Title1604/10/23

Lucent Technologies - Proprietary

Unconditional• Unconditional works much

better - growth in number of packets consists only of small t2 component

• Total packets sent grows with square of delay to TMIN quotient

• Reason - unconditional skews distribution of packet transmissions towards later in time

NN

TD

mpMIN

8

12 2

2

t

Packets S

ent

Np

Presentation Title1704/10/23

Lucent Technologies - Proprietary

State Storage• Problem:

– Must maintain group size estimate

– To distinguish new users from old, must maintain SSRC list for all users

– In a group of thousands, can be problematic

• Solution:– Dynamic sampling

– Given memory size M, store all SSRC until memory full

– Discard all SSRC with MSB=1 (roughly 1/2)

– Sample probability is 2-k

– Group size estimate is Ns 2k

Presentation Title1804/10/23

Lucent Technologies - Proprietary

Delay• May not desire RTCP in current

distributed form– When you know group size is large

apriori

• Central monitor polls stations for reports

• Monitor sends SSRC mask, and interval T

• Stations whose SSRC match mask respond to poller within T seconds

• Must gradually decrease size of mask to avoid flood (Bolot, Wakeman)

• Standard sampling theorems allow group size estimation and confidence computations

• IDEA - Use non-uniform response distribution, and allow poller to cancel poll

• If poller gets lots of responses early, it can expect tons later, so cancel the poll, decrease probability

• Like the beach parking lot problem!

Presentation Title1904/10/23

Lucent Technologies - Proprietary

Premature Timeouts• Subtle problem

– Timeout if no packet seen in MCL(t)

– Rapid leave causes L(t) to drop; timeout interval drops

– Remaining users won’t send packets immediately; change in interval size takes effect after next packet transmission

Original Timeout Window

Timeout Window after Flood

RTCP Packets

BYE Flood

0 1 2 3

Presentation Title2004/10/23

Lucent Technologies - Proprietary

Solution• Two-fold

• Reverse Reconsideration– When BYE arrives, each user

reschedules their next packet a little earlier

– Allows packet transmission rate to closely track group membership

• Group Size filtering– Don’t use L(t) for timeout - use

filtered version!

– How to filter?

– Optimal timeout window size topt is solution to:

pcp

ccp

cnp

ccn

ttn

ntt

ttn

ntt

MCdttL

t

tt opt

1

Presentation Title2104/10/23

Lucent Technologies - Proprietary

BYE Storms• Users simultaneously leave also!

• Usually send a BYE - flood of packets

• Solution: Apply reconsideration to BYE packets– At leave time, schedule BYE packet

– Count number of BYE’s from other users since my leave time

– Use that as “group size estimate” in reconsideration equations

– Works very well

Presentation Title2204/10/23

Lucent Technologies - Proprietary

Conclusions• RTP Scalability to large multicast groups has several

components– Step joins and leaves

– State storage

– Premature timeouts

• Problem is general– Desire equal bandwidth sharing among users

– Number of users is known in a distributed way by observing transmission

– Number of users is very dynamic