Presentation Title 1 06/26/22 Lucent Technologies - Proprietary RTP Scalability in Large Multicast Groups Jonathan Rosenberg High Speed Networks Research 11/10/97
Mar 27, 2015
Presentation Title104/10/23
Lucent Technologies - Proprietary
RTP Scalability in Large Multicast Groups
Jonathan Rosenberg
High Speed Networks Research11/10/97
Presentation Title204/10/23
Lucent Technologies - Proprietary
Contents• RTP and RTCP Primer
• Large scale applications
• Problems– Congestion
– State Storage
– Delay
– BYE Storms
– Premature Timeouts
• Reconsideration– Conditional
– Unconditional
– Reverse
– BYE
• Polling
• Group Size Estimation
• Conclusions
Presentation Title304/10/23
Lucent Technologies - Proprietary
RTP and RTCP• RTP provides for
– Real time transport
– Resequencing
– Payload type identification
– Intra and Inter media synchronization
– Encryption
– Multicast• Per User demultiplexing - SSRC
• RTP does not– Provide QoS
– Require RSVP
• RTP is a framework– Specific payload formats defined
for H.263, MPEG etc.
– UDP Port numbers based on application
Presentation Title404/10/23
Lucent Technologies - Proprietary
RTCP• Real Time Control Protocol
• RTP port + 1
• Used for– QoS Reporting
• Sender reports: packets sent, bytes sent
• Receiver reports (per sender): loss, delay, jitter observed; instantaneous and cumulative
– Media Synchronization• NTP and RTP Timestamp correlation
– Loose Session Control• Hello, Bye messages
• SDES - email, username, CNAME, etc.
Presentation Title504/10/23
Lucent Technologies - Proprietary
Large Scale Applications• Cable Distribution
– one sender, tens of thousands of receivers
– QoS reporting critical
– Nielsen - group size estimation
• Distributed real time sensors
• Van Jacobson RLM - Receiver Driven Layered Multicast
• MBONE concerts, talks
Presentation Title604/10/23
Lucent Technologies - Proprietary
Achieving Scalability• RTP
– RTP scales if number of data senders is small or controlled
– Generally not a problem
• RTCP– Both data senders AND
receivers send reports
– Reports are periodic
– Period must scale with group size to avoid implosion
– Period is defined to scale linearly
• Each system listens to group and counts number of other users heard from - L(t) is group size estimate
• Period is set to CL(t); C depends on application
• Works fine if group size is static
• $10M question: How does this work when L(t) is dynamic?
Presentation Title704/10/23
Lucent Technologies - Proprietary
Other Applications• Problem is general; applies when
– Some unkown number of users wish to share a bandwidth resource equally
– The users are on a shared medium, and can hear each other, like multicast
– The number of users grows/shrinks over time
• Examples– CBR traffic on Ethernet
– Route updates among BGP peers
– Distributed Sensors
Presentation Title804/10/23
Lucent Technologies - Proprietary
RTCP Intervals• Require RTCP reporting to scale with
multicast group size
• Total RTCP bw should be no more than 5% of session bw
• Divide 5% among participants - 75% senders, 25% receivers
• Algorithm:– At time t, emit packet
– Compute interval T
– Schedule packet for t+T
– Repeat at t+T
• TMIN is 2.5s for first packet, 5s for all others
• Add randomization to avoid synchronization
T TN pktsz
bw
T T
T T
d MIN
d
d
max ,. .
,
,
05 75
1
2
3
2
1 1
Presentation Title904/10/23
Lucent Technologies - Proprietary
Problems• Congestion
– Simultaneous Joins (ER at 10pm)
– RTCP Interval based on group size estimate
– N users, each think there is only ONE group member
– Multicast flood over initial 2.5s interval - all N users send
– Packet losses -> further congestion
• State Storage– To count group size, must store state for each other group member
– Set top boxes - memory size limitations
Presentation Title1004/10/23
Lucent Technologies - Proprietary
Problems• Delay
– Frequency of reports from any user becomes small
– Average loss reports over large intervals useless
– Time to converge on group size exceeds membership times
– DVMRP cache times -> EVERY RTCP packet is flooded!
• Premature Timeouts– Must timeout members if you haven’t heard from them in M transmission
intervals
– Drop in group membership reduces transmission interval
– Sharp drop can cause a user to prematurely timeout other users
• BYE Storms– Many users simultaneously leave and send a BYE message
Presentation Title1104/10/23
Lucent Technologies - Proprietary
Reconsideration• Avoid Congestion
• Last packet send time was tn-1
• About to send at time tn
• Reconsider if number of users changed since tn-1
• Compute interval based on new observation -> T’, add to tn-1. If result is less than current time, send now, else reschedule for tn-1 + T’
• Conditional: As above
• Unconditional: Recompute even if number of users unchanged
• Backwards compatible
• Effectiveness grows with usage
• Safe
tn-1 tn
3 new users observed
tn-1 + T’
Presentation Title1204/10/23
Lucent Technologies - Proprietary
Simulation Assumptions• 28.8 kbps access links downstream
• Infinitely fast upstream links
• Access server buffers = 100kB per user
• Network adds delay uniform between 0 and 600ms
• Full connectivity - multicast join latency not considered
• Step join 10K users at t=0. Each have a homogeneous view of state
Title: rtcp_net_.figCreator: fig2dev Version 3.1 Patchlevel 2CreationDate: Sat Feb 1 16:11:38 1997
Presentation Title1304/10/23
Lucent Technologies - Proprietary
Packets SentTitle: Creator: gnuplotCreationDate:
Presentation Title1404/10/23
Lucent Technologies - Proprietary
Analytical Results• Can derive a differential
equation for L(t) given a step join!
• For small t, solution is linear when there is no network delays
• Can also show that packet rate is roughly derivative of L(t)
• Gives an estimate of network bandwidth
L tt
C
1
Presentation Title1504/10/23
Lucent Technologies - Proprietary
Add Delay• Real network has delay - significant impact on
performance
• Arrival of packets used to increase group size estimate and back-off -> network delays cause initial bunch of senders not to see packets
• Result is a spike of packets
• After packets arrive, causes rapid increase in group size estimates
• Result is a cessation of packet transmission: plateau effect
• What is size of spike?
• Again, analytic solutions exist!
• Let D be fixed network delays, m is modem rate in packets/s
• In limit of N large, distribution of delays doesn’t matter!
• Conditional - first a linear growth in packets, then a growth as t2 - much smaller
NN
TD
mpMIN
2
1
t
Packets S
ent
Np
Presentation Title1604/10/23
Lucent Technologies - Proprietary
Unconditional• Unconditional works much
better - growth in number of packets consists only of small t2 component
• Total packets sent grows with square of delay to TMIN quotient
• Reason - unconditional skews distribution of packet transmissions towards later in time
NN
TD
mpMIN
8
12 2
2
t
Packets S
ent
Np
Presentation Title1704/10/23
Lucent Technologies - Proprietary
State Storage• Problem:
– Must maintain group size estimate
– To distinguish new users from old, must maintain SSRC list for all users
– In a group of thousands, can be problematic
• Solution:– Dynamic sampling
– Given memory size M, store all SSRC until memory full
– Discard all SSRC with MSB=1 (roughly 1/2)
– Sample probability is 2-k
– Group size estimate is Ns 2k
Presentation Title1804/10/23
Lucent Technologies - Proprietary
Delay• May not desire RTCP in current
distributed form– When you know group size is large
apriori
• Central monitor polls stations for reports
• Monitor sends SSRC mask, and interval T
• Stations whose SSRC match mask respond to poller within T seconds
• Must gradually decrease size of mask to avoid flood (Bolot, Wakeman)
• Standard sampling theorems allow group size estimation and confidence computations
• IDEA - Use non-uniform response distribution, and allow poller to cancel poll
• If poller gets lots of responses early, it can expect tons later, so cancel the poll, decrease probability
• Like the beach parking lot problem!
Presentation Title1904/10/23
Lucent Technologies - Proprietary
Premature Timeouts• Subtle problem
– Timeout if no packet seen in MCL(t)
– Rapid leave causes L(t) to drop; timeout interval drops
– Remaining users won’t send packets immediately; change in interval size takes effect after next packet transmission
Original Timeout Window
Timeout Window after Flood
RTCP Packets
BYE Flood
0 1 2 3
Presentation Title2004/10/23
Lucent Technologies - Proprietary
Solution• Two-fold
• Reverse Reconsideration– When BYE arrives, each user
reschedules their next packet a little earlier
– Allows packet transmission rate to closely track group membership
• Group Size filtering– Don’t use L(t) for timeout - use
filtered version!
– How to filter?
– Optimal timeout window size topt is solution to:
pcp
ccp
cnp
ccn
ttn
ntt
ttn
ntt
MCdttL
t
tt opt
1
Presentation Title2104/10/23
Lucent Technologies - Proprietary
BYE Storms• Users simultaneously leave also!
• Usually send a BYE - flood of packets
• Solution: Apply reconsideration to BYE packets– At leave time, schedule BYE packet
– Count number of BYE’s from other users since my leave time
– Use that as “group size estimate” in reconsideration equations
– Works very well
Presentation Title2204/10/23
Lucent Technologies - Proprietary
Conclusions• RTP Scalability to large multicast groups has several
components– Step joins and leaves
– State storage
– Premature timeouts
• Problem is general– Desire equal bandwidth sharing among users
– Number of users is known in a distributed way by observing transmission
– Number of users is very dynamic