Test Plan IPTV Channel Change Testing

26601 W. Agoura Rd.Calabasas, CA 91302

(Toll Free US) 1.877.FOR.IXIA(Int'l) +1.818.871.1800

(Fax) 818.871.1805

www.ixiacom.com

Test Plan

IPTV Channel Change Testing

Copyright © 2006 by Ixia

All rights reserved

Ixia 26601 West Agoura Road, Calabasas, CA 91302 (877) FOR-IXIA

This Test Plan contains a general outline for testing a particular technology. Not all the capabilities of Ixia technology have been exposed in this document. Please feel free to contact us if additional capabilities are required.

© 2006 by Ixia p.3 www.ixiacom.com

IPTV Channel Change Test PlanContentsIPTV Channel Change Test Plan . . . . . . . . . . . . . . . 1

Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Performance Metrics . . . . . . . . . . . . . . . . . . . . . . . 4Basic Setup . . . . . . . . . . . . . . . . . . . . . . . 5Advanced Setup . . . . . . . . . . . . . . . . . . . . 6

1. Test 1 – Maximum Client Capacity . . . . . . . . . . . 9Objective . . . . . . . . . . . . . . . . . . . . . . . . . 9Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9Input Parameters . . . . . . . . . . . . . . . . . . . 10Methodology . . . . . . . . . . . . . . . . . . . . . 11Results . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2. Test 2 – Maximum Transaction Rate . . . . . . . . . 15Objective . . . . . . . . . . . . . . . . . . . . . . . . 15Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . 15Input Parameters . . . . . . . . . . . . . . . . . . . 16Methodology . . . . . . . . . . . . . . . . . . . . . 17Results . . . . . . . . . . . . . . . . . . . . . . . . . . 18

3. Test 3 – Optimal Client Response Time . . . . . . . 20Objective . . . . . . . . . . . . . . . . . . . . . . . . 20Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . 21Input Parameters . . . . . . . . . . . . . . . . . . . 22Methodology . . . . . . . . . . . . . . . . . . . . . 23Results . . . . . . . . . . . . . . . . . . . . . . . . . . 24

4. Test 4 – Server Resiliency . . . . . . . . . . . . . . . . 25Objective . . . . . . . . . . . . . . . . . . . . . . . . 25Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . 25Input Parameters . . . . . . . . . . . . . . . . . . . 26Methodology . . . . . . . . . . . . . . . . . . . . . 27Results . . . . . . . . . . . . . . . . . . . . . . . . . . 28


IPTV Channel Change Test Plan

In the context of Triple Play services delivery, IPTV represents the most significant challenge and often the most complex to test and verify. This channel changing performance test plan assists you in comprehensively qualifying the performance of one or more Devices Under Test (DUT) or Systems Under Test (SUT) in an Internet Protocol Television (IPTV) deployment.

This test plan encompasses several key IPTV performance metrics that are critical in qualifying the DUT/SUT and ensuring a successful deployment. Each of the test contained here provides comprehensive information on stress testing the DUT/SUT to ensure that is it able to perform as specified, and able to handle the typical and extreme load conditions as required. For more information, see the “Performance Metrics” section later in this document.

The recommended testing methodology presented here is also meant to be used as a guideline for creating more case-specific testing scenarios to further characterize the performance limits of the IPTV infrastructure.



BackgroundIn the telecommunications marketplace today, traditional cable and telephone operators are aggressively staging and deploying converged networks that can deliver data, voice, and video services to their rapidly expanding customer base. Such networks are referred to as Triple Play networks.

The effective delivery of the video component of Triple Play services, called IPTV, has a fundamental requirement to operate within a multicast-capable network, delivering broadcast service television over the IP network by ensuring the efficient use of network resources. Using this technology, IPTV services are delivered directly to the home and are intended to replace the traditional video delivery network.

The figure below is a simplified view of a multicast network used to deliver a single channel (using a single multicast address) to a number of group members (or viewers of a particular channel). The flow of data from the Group Source to the Group Members follows the multicast routing tree. A multicast routing protocol such as PIM-SM/DM is used to create an efficient routing path for the delivery of multicast packets between routers. The host to edge-router communication is generally facilitated using IGMPv2/v3.

IP MulticastNetwork

Group Source

GroupMember

GroupMember

Non-Member

GroupMember

Non-Member

Non-Members

Core/AggregationRouter 1

EdgeRouter 1

AggregationRouter 2

EdgeRouter 2

EdgeRouter 3

Figure 1. Simple multicast tree used to deliver single channel multicast traffic to hosts



What is channel changing or channel zapping?

Channel changing or “channel zapping” in the IPTV domain is the equivalent of surfing channels on a television set in the traditional sense. Channel changing, then, always entails leaving one TV channel and joining/watching the next.

Why is it important?

In the traditional cable television network, channels were always “present” on the wire. For this reason, channel changing performance was generally not an issue. Switching from one channel to the next was usually instant, typically was not a source of user dissatisfaction.

For satellite TV providers, on the other hand, the inherent distance that the video signals had travel always introduced a “delay” in switching between channels. The usual approach to reduce the perceived delay was to provide some kind of on-screen feedback when switching channels so that the user knew the channel change request had been received.

Now, with video being delivered over an IP network, one that was not necessarily designed with video transport as a key goal, several optimizations and novel methods are being deployed throughout a providers’ network to improve the perceived “instant” channel change behavior that is expected by traditional television consumers.

IGMP is the industry standard protocol used for host to router communication in a multicast network. Therefore, channel changing (CC) performance measured at an edge router or an aggregation device is intricately tied to inherent implementation and fine tuning of the IGMP protocol. The CC performance is also a function of the emulated end device receiving the multicast stream for final output, such as a set-top box. For this reason, channel changing performance is vital in characterizing performance for devices such as DSLAMS or CMTS and is equally important for an end-to-end IPTV test.

For more information on the specific performance metrics attainable by using channel changing profiles to test the IPTV network, refer to the “Performance Metrics” section later in this document.



What are the channel changing (CC) requirements?

The channel changing performance in an IP network is primarily the result of end-to-end processing of associated multicast protocols, packet switching, and the end-device’s ability to decode or “show” the video with relative consistency.

A deployed system must be able to performance predictably in a sustained mode. A sustained mode of operation refers to a realistic load condition, not a best-performance condition with light use.

In addition to sustained performance, a deployed system must be resilient to fault conditions where peak loads may easily exceed the sustained performance limits. For a system to be resilient, it must be able to perform acceptably well and be able to recover from an overloaded condition.

Channel surfing presents varying load conditions for a multicast network. For example, if there is a sudden power outage in a neighborhood on a typical Sunday “game day,” a mass flood of traffic will be injected into the IP network requesting network configuration information followed by several channel changes to tune back to a desired channel by hundreds, if not thousands, of viewers at almost the same time. Testing such realistic load conditions is critical in ensuring optimal QoS on the network and help tune device settings.

Requirements of good channel performance are characterized using the metrics in the “Performance Metrics” section later in this document.



Performance MetricsTo determine channel changing performance, several performance metrics are used. The following terms are defined and used to provide objective performance. Note that the terms defined here comprise generally accepted industry terms and protocol-specific terms that help characterize the performance.

IGMP Join Latency -– The time between a request to join a multicast group and the receipt of the first byte of data for a multicast group.

IGMP Leave Latency – The time between a request to leave a multicast group and the receipt of the last byte of data for the multicast group

Channel Overlap – The duration of time when data is received for a new joined multicast group and a previously left multicast group. This time is usually zero units.

Channel Switch Delay (STB dependent) – An internal IGMP processing delay between a Leave and Join request. This value is ideally insignificant; however, it can be otherwise.

Channel Change/Zap Delay – The inter-channel change delay, which is the time between a channel Leave request sent and the receipt of the first byte of data from the new multicast channel. It is the IGMP Join Latency + Channel Switch Delay (STB dependent). This value is ideally very close to the IGMP Join Latency; however, the STB can introduce a significant delay.

The following timing diagrams outline the delays for 2 channels.

Join Channel 1IGMP MembershipReport (*, C1)

Leave Channel 1IGMP Leave Group(*, C1)

Multicast Channel 1JoinLatency

Join Channel 2IGMP MembershipReport (*, C2)

CSD – Channel Switch Delay(STB dependent)

Multicast Channel 2JoinLatency

CSD

Leave LatencyChannel Overlap

Figure 2. IGMP Join and Leave latency timing diagram



The specific metrics outlined above must be available on a per-channel/per-subscriber basis. The various metrics make up the parts of a sum that help provide an at-a-glance health check for every subscriber and the channels being watched.

It is not sufficient, though, to use such metrics alone to characterize the performance of an IPTV network/device. To isolate adverse network conditions causing unacceptable channel change performance, network centric measurements must be available to provide an at-a-glance health check of the network on a per-channel/per-subscriber.

Media Delivery Index (MDI) – A scoring mechanism that combines packet delay variation (jitter) and media packet loss to determine the quality of the network to transport good quality video. It is measured in milliseconds.

MDI:DF (Delay Factor) – Defined as cumulative IP jitter. It represents the time it would take to drain an output buffer and ensure good video playback.

MDI:MLR (Media Loss Rate) – Defined as the packet loss rate due to dropped packets, bad/corrupted packets, or out-of-sequence packets.

The Media Delivery Index (MDI) is important because it characterizes the performance of the network and its ability to handle good video streams. The index provides two measures separately so that each IPTV device can be tested with various channel change patterns to help insolate device-specific issues. By identifying problems on an IPTV network device, it becomes easy to troubleshoot and optimize the configuration to provide optimal performance end-to-end.

This approach can be extended to characterize the end-to-end channel change performance of an IPTV deployment.



1. Test 1 – Optimal Channel Change PerformanceObjective

This test setup focuses on determining the optimal channel change performance with IPTV traffic running across one or more DUTs.

Since IPTV subscriber has a different usage pattern, it is necessary to determine the end user’s experience based on realistic channel change patterns. It is also important to determine how the subscriber’s channel change performance changes as the complexity of the subscriber’s behavior (channel change profile) increases and as the number of subscribers increase.

The variability of the CC patterns will have a direct impact on the performance of devices supporting such traffic. The importance of determining the optimal performance of an IPTV DUT is to ensure that it is within operating limits of providing excellent channel change performance experienced by subscribers. For example, is there channel overlap when users are rapidly “surfing” channels? How does poor jitter experienced by one video stream affect the others on the same link? How is the transport of video streams affected by a growing number of subscribers? These are just some of the questions that can be answered by emulating realistic user channel change behavior to determine various acceptable levels of performance.

This test emulates typical load conditions of 1000s of subscribers with multiple channel changing profiles and 100s of video streams. Several iterations of the test will help ensure that the device/network has sufficient raw bandwidth to support the load, reveal at localized congestions, and assist in fine-tuning devices for maximum performance.

The maximum number of users supported by the DUT/network does not necessarily translate to optimal channel change performance. For this reason, there is a trade-off between the best channel change performance with realistic subscriber loads and the maximum performance possible by the device/network. Both metrics have merit and both must be determined.

This test is applicable to an IPTV device and can be extended to characterize optimal end-to-end performance.



Setup

The setup requires at least 2 or more Ixia test ports to generate stateful IPTV traffic. A video server port is used to stream 100s of standard and high definition video streams. The topology presented here is representative of a typical Triple Play deployment network. There are several configurations possible and this test setup is used as a guideline. The video subscribers are connected to an IGMPv3 enabled core router and the video server is connected to a carrier core router. The core multicast network has a 4Gbps backbone and it runs BGP for route determination and PIM-DM as the multicast protocol.

IP Multicast Network

IxiaClientsPort(s)

IxiaServerPort(s)

IxLoadVideoClients

IxLoadVideoServer

MulticastStreams

IGMPControl

1 2

43

65

Trig OutGnd

LM1000GBIC-2

IXIA

SubscriberRouter

CarrierRouter

1 2

43

65

Trig OutGnd

LM1000GBIC-2

IXIA

LACP4GbitTrunk

Figure 3. Topology for Test 1 – Optimal Performance



Input Parameters

Parameter Description

Client Configuration 50 – 1000+ users per test portClient Network One or more VLAN with untagged

and/or 802.1q tagged packets per VLAN

Client Channel Change Profiles

Sequential channel surfing – for 10 – 20 secRandom channel surfing – for 10 sec – 20 minOther random profiles as per tester’s requirements

Client Command-list JOIN channels as per Channel change Profile selected per run

Client IGMP parameters The IGMP options below may be configured as desired. Consider the specific trade-offs for each options before enabling/disabling them.Unsolicited Response ModeGeneral Query/Group Specific Response ModeImmediate ResponseSuppress Reports

Client/Server Test Ports At least one. More ports can be used to scale the test

Video Server At least oneVideo Server Configuration

100+ video streams. Use MPEG2-TS with synthetic payload at 3.75Mbps (for SD). Use higher bitrate video streams to simulate HD streams (9.8Mbps+)Multicast group address – configurable as per network design. Must have sufficient count to ensure all video streams have a multicast group address

Test Objective Iterative method to determine the maximum simulated users

Table 1. Input Parameters for Test 1 – Optimal Performance



Methodology

1. Before starting the test, determine the baseline performance characteristics of the test equipment. This will ensure that the upper limits of the test equipment are known and that the performance per-port or per-system is well defined.

2. Once the baseline is established using the test ports, configure the test setup network or the IPTV DUT.

To support multicast traffic, at least one switch will be required to support IGMPv2 or v3. In addition, a multicast protocol must be running to efficiently deliver multicast traffic.

3. Set up the test, and configure the parameters for the protocols as outlined in Input Parameters for the Client Traffic.

Choose the version of IGMP supported by the switch. Also, ensure that the IGMP parameters are using default values as specified in RFC 2236 (IGMPv2) or RFC 3376 (IGMPv3).

The command-list is where the channel changing profile can be configured. The desirable options here are to run the test iteratively and modify the channel changing profiles to simulate different load conditions on the multicast fabric.

Some channel changing profiles can include:

• A set of subscribers surfing through channels 1-50 for 10-20 seconds each.

• A set of subscribers surfing through channels 51-100 for 1-30 seconds each.

• A set of subscribers emulating a typical habit of watching channel A for 30 minutes with frequent channel surfing within that time for commercial breaks.

The channel-changing profiles must be iteratively set up to test the multicast network and monitor an average subscriber’s channel change performance. Ensure that there are sufficient ports to run the test.

4. Configure the video server traffic to serve 100s of video streams. Refer to the Input Parameters to configure the stream contents appropriately.



5. Configure the Client and Server networks to ensure end-to-end connectivity. Confirm that multicast protocol is running and reachable end-to-end.

6. Set up the test objective to determine the maximum number of sustained users.

7. Run several iterations of the test to determine the maximum performance of the DUT/network (in handling # of subscribers and # of simultaneous channels).

However, consider that the maximum number of subscribers supported by the DUT/network does not necessarily translate to optimal channel change performance. There is a trade-off between the best channel change performance with realistic subscriber loads and the maximum performance possible by the network. Both metrics have merit and both must be determined.

8. Adjust the IGMP parameters to optimize channel change latency.

Vendor implementations of IGMP may vary and specific optimized configurations must be determined from configuration and user guides.



Results

This test was used to characterize various subscriber load conditions and determine the channel change performance experienced by each subscriber. In addition, network flow quality of video streams were examined per stream to determine how the device/system performed when it was incrementally loaded with more subscribers, more channels, and more complex subscriber channel change patterns.

The optimal channel change performance was a trade-off between the maximum performance possible by the DUT/network without packet loss and the acceptable channel change latencies under realistic load conditions. Both metrics have merit. Knowing the peak performance of the DUT/network ensures that extreme load conditions are serviceable during failure or startup times. However, at peak performance a subscriber’s channel changing performance may be less than optimal.

Optimal channel changing performance was measured at an equilibrium point that was below the maximum device performance but still simulated realistic subscriber load conditions.

A brief result of three runs is presented below:

Stat NameStream Bit Rate (bps)

MDI-DF (us) MDI-MLR

MPEG2 TS Loss Jitter (ns)

Inter Pkt Arrival

Time (ns)

Packet Latency

(ns)

Join Latency

(ms)

Leave Latency

(ms)

RUN 1/ User 1 Channel 1

3750007 2835 0 0 280 2805680 38160 69 8479

RUN 2/User 1 Channel 1

3750007 2835 0 0 1780 2807180 38320 108 8460

RUN 3/User Channel 1

3750007 2835 0 0 300 2216420 38540 100 5045

Table 2. Test 1 channel change performance for multiple runs

The first run simulated 300 users watching 100 channels for 10-20 seconds each. The second run increased the number of users to 1200. The jitter increased considerably with the increase in subscriber count. The third run reduced the subscriber count to 800 and modified the channel change behavior for different sets of subscribers.



If the subscribers were increased to more than 1200, there were MPEG2 TS losses seen. The optimal configuration for the network was 800 subscribers – all having different channel changing profiles.

To further characterize the channel change performance, real-time statistics of the Channel Change/Zap Delay were actively monitored. A snippet of these real-time statistics for the third run is presented below.

Figure 4. Real-time channel change/zap delay for test 1, run 3

The average jitter distribution for several channel requests is presented below for the third run, and 99.87% of the packets had jitter below 50 us, which is very good.

Figure 5. Jitter distribution for test 1, run 3



2. Test 2 – Single Subscriber ExperienceThis test is composed of a series of runs with 1000s of subscribers with various channel change patterns. The test will assess the experience of a single subscriber who is watching a few channels while several other subscribers place different load requirements on the multicast network. In this way, the test setup will assess the join/leave latency and the channel change/zap delay, including MDI scores.

Setup

The setup will require at least 2 or more Ixia test ports to generate stateful IPTV traffic. A video server port is used to stream 100s of standard and high definition video streams. The topology presented here is representative of a typical Triple Play deployment network. There are several configurations possible and this test setup is simply used as a guideline. The video subscribers are connected to an IGMPv3 enabled core router and the video server is connected to a carrier core router. The core multicast network has a 4Gbps backbone and it runs BGP for route determination and PIM-DM as the multicast protocol.


IxiaClientsPort(s)

IxiaServerPort(s)

IxLoadVideoClients

IxLoadVideoServer

MulticastStreams

IGMPControl

1 2

43

65

Trig OutGnd

LM1000GBIC-2

IXIA

SubscriberRouter

CarrierRouter

1 2

43

65

Trig OutGnd

LM1000GBIC-2

IXIA

LACP4GbitTrunk

Figure 6. Topology for Test 2 – Single Subscriber Experience



Input Parameters


Client Configuration 50 – 1000+ users per test portClient Network One or more VLAN with untagged

and/or 802.1q tagged packets per VLAN


1 subscriber with fixed channel change patternOther random profiles as per tester’s requirements


Client IGMP parameters The IGMP options below may be configured as desired. Consider the specific trade-offs for each options before enabling/disabling them.Unsolicited Response ModeGeneral Query/Group Specific Response ModeImmediate ResponseSuppress Reports


Video Server At least oneVideo Server Configuration

100+ video streams. Use MPEG2-TS with synthetic payload at 3.75Mbps (for SD). Use higher bitrate video streams to simulate HD streams (9.8Mbps+)Multicast group address – configurable as per network design. Must have sufficient count to ensure all video streams have a multicast group address

Test Objective Iterative method to determine the optimal single subscriber performance (using maximum simulated users)

Table 3. Input Parameters for Test 2 – Single Subscriber Experience



Methodology

1. Configure the test setup network or the IPTV DUT.

To support multicast traffic, at least one switch will be required to support IGMPv2 or v3. In addition, a multicast protocol must be running to efficiently deliver multicast traffic.

2. Set up the test, and configure the parameters for the protocols as outlined in Input Parameters for the Client Traffic.

Choose the version of IGMP supported by the switch. Also ensure that the IGMP parameters are using default values as specified in RFC 2236 (IGMPv2) or RFC 3376 (IGMPv3).

The command-list is where the channel changing profile can be configured. The desirable options here are to run the test iteratively and modify the channel changing profiles to simulate different load conditions on the multicast fabric.

Some channel changing profiles can include:

• One subscriber (pilot) watching a few channels for long durations.

• A set of subscribers emulating many channel change patterns to stress the multicast network differently. The channel-changing profiles must be iteratively set up to test the multicast network to monitor a single subscriber’s channel change performance for the iterations. Depending on how the clients profiles, ensure that there are sufficient ports to run the test.

3. Configure the video server traffic to serve 100s of video streams. Refer to the Input Parameters to configure the stream contents appropriately.

4. Configure the Client and Server networks to ensure end-to-end connectivity. Confirm that multicast protocol is running and reachable end-to-end.

5. Set up the test objective to determine the maximum number of sustained users.

6. Run several iterations of the test to determine the channel change performance for a single pilot subscriber.

Change the multicast load conditions by iteratively changing the other subscribers’ channel change patterns and monitor the pilot user’s Join/Leave and Change Change/Zap delays.



Results

The objective of this test was to assess an single subscriber’s channel change performance under various loads by changing the channel change profiles of the subscribers.

The real-time nature of monitoring a single subscriber’s Join/Leave latencies is essential. An instantaneous view of the real-time information of the single subscriber is presented below.

Table 4. Test 2 channel change performance for multiple runs

The first run included 500 users with varying channel changing patterns. The second run increased the subscriber count to 1000. From the table above, it can be seen that increasing the subscribers and hence the load on the multicast network, increased the subscriber’s perceived Join/Leave latencies by 70% and 65% respectively.

The channel change/zap delay observed for the single subscriber was similarly affected.

The DUT was not tested for raw performance. Therefore, the behavior of a single subscriber being affected by other subscribers is primarily a result of the network having no QoS setup to police any traffic. Generally, the QoS on the network must be set up so that the single subscriber sessions are not affected by other subscribers on the same link.

By monitoring a single subscriber, it is possible to troubleshoot devices that may not be observing the QoS settings or traffic policy properly. Such monitoring, however, may not be possible if the only metric observed is the optimal channel change for all aggregate subscribers.

Stat NameStream Bit Rate (bps)

MDI-DF (us) MDI-MLR

MPEG2 TS Loss Jitter (ns)

Inter Pkt Arrival

Time (ns)

Packet Latency

(ns)

Join Latency

(ms)

Leave Latency

(ms)

RUN 1/ User 1 3750004 2229 0 0 198 2216220 38360 100 4503

RUN 2/User 1 3750004 2233 0 0 422 2217140 38020 170 6901



3. Test 3 – Triple Play TrafficThis test extends the IPTV channel change performance test by provisioning data and voice traffic on the same link as the video streams to assess the performance of this additional traffic.

In a deployed network, the convergence of data, voice, and video traffic creates a realistic scenario where the video streams are sharing resources with data and voice traffic. The traffic types are very different from each other, and each requires a different level of service from the network.

To ensure that each of the devices on the converged network is working properly as part of the IPTV solution, it is necessary to test several Triple Play scenarios where the channel change behavior of several subscribers creates realistic load conditions. This is in addition to the data and possibly voice traffic that must transit the same link, either from the same subscriber or other subscribers sharing an uplink from an aggregate router. By emulating various load conditions on the Triple Play network, optimizations can be made to ensure that the QoS on the devices in the network is working as expected.

Setup

The setup will require at least 3 or more Ixia test ports to generate stateful Triple Play traffic. A video server port will be used to stream 100s of standard and high definition video streams. A second server port will accept data and voice traffic. At least one client test port will generate stateful video, voice, and data traffic based on the conditions in the Input Parameters table. The topology presented here is representative of a typical Triple Play deployment network. There are several configurations possible and this test setup should be used as a guideline. The video subscribers are connected to an IGMPv3 enabled core router and the video server is connected to a carrier core router. The core multicast network has a 4Gbps backbone, and it runs BGP for route determination and PIM-DM as the multicast protocol.


IxiaClientsPort(s)

IxiaServerPort(s)

IxLoadVideoClients

IxLoadVideoServer

MulticastStreams

IGMPControl

1 2

43

65

Trig OutGnd

LM1000GBIC-2

IXIA

SubscriberRouter

CarrierRouter

1 2

43

65

Trig OutGnd

LM1000GBIC-2

IXIA

LACP4GbitTrunk

Figure 7. Topology for Test 3 – Triple Play Traffic



Input Parameters


Client Configuration 50 – 1000+ users per test port

Client Network One or more VLAN with untagged and/or 802.1q tagged packets per VLAN


Sequential channel surfing – for 10 – 20 sec

Other random profiles as per tester’s requirements


FTP traffic – varying bandwidth to simulate typical and adverse conditions

HTTP traffic – varying page retrievals

SIP traffic – to simulate voice-over-IP calls

Client Parameters The IGMP options below may be configured as desired. Consider the specific trade-offs for each options before enabling/disabling them.

Unsolicited Response Mode

General Query/Group Specific Response Mode

Immediate Response

Suppress Reports

FTP client – use Passive mode

SIP – use UDP for signaling with RTP media (codec selectable)

Continued on next page



Traffic Impairment (optional)

Introduce traffic impairment such as latency, jitter, packet drop profiles and duplicate packets (to further stress the edge devices in processing packets with errors or inherent delays/jitter than may cause queuing problems on the interface buffers)


Video Server At least two. A dedicated video server port and a dedicated data and voice traffic port

Video Server Configuration

100+ video streams. Use MPEG2-TS with synthetic payload at 3.75Mbps (for SD). Use higher bitrate video streams to simulate HD streams (9.8Mbps+)

Multicast group address – configurable as per network design. Must have sufficient count to ensure all video streams have a multicast group address

Test Objective Iterative method to determine the maximum simulated users

Table 4. Input Parameters for Test 3 – Triple Play Traffic



Methodology

1. Configure the test setup network or the IPTV DUT.

For multicast traffic, at least one switch is required to support IGMPv2 or v3. In addition, a multicast protocol must be running to efficiently deliver multicast traffic.

2. Set up the test, and configure the parameters for the protocols as outlined in Input Parameters table for the Client Traffic.

Choose the version of IGMP supported by the switch. Also, ensure that the IGMP parameters are using default values as specified in RFC 2236 (IGMPv2) or RFC 3376 (IGMPv3).

The command-list is where the channel changing profile should be configured. The desirable options here are to run the test iteratively, and then modify the channel changing profiles to simulate different load conditions on the multicast fabric. Include data traffic such as FTP and/or HTTP with various page retrievals. Also, enable SIP voice-over-IP traffic (1 call per subscriber).

3. Configure the video server traffic to serve 100s of video streams. Refer to the Input Parameters table to configure the stream contents appropriately. Also configure the SIP callee and FTP server side traffic profile.

4. Configure the Client and Server networks to ensure end-to-end connectivity. Confirm that a multicast protocol is running and reachable end-to-end.

5. Set up the test objective to determine the optimal subscriber channel change performance. The test Objective will vary for each of the activities defined (video, voice and data), depending on the activity and test iteration. The objectives will reflect the load conditions required to effectively determine how background data and voice traffic affect an average subscriber’s channel change performance.

6. To test the capability of the edge routers in handling malformed packets and being able to manage its interface buffer queues properly, introduce impairment such as latency, jitter, duplicate packets or random drop packets.



Results

The objective of this test was to determine the subscriber’s channel change performance on a converged network that is running video and voice traffic.

Several test runs were conducted. Two of the runs are examined here.

The results below show successful SIP calls during one of the test run 2 with 200 subscribers.

Figure 8. Completed SIP calls for test 3



HTTP transactions are presented below for the same test run.

Figure 9. HTTP transactions for test 3

The HTTP transaction latency for two runs is given below. The transaction latency increased on the second run as the FTP throughput was increased from 1.6Mbps to 24Mbps. The FTP traffic directly impacted the HTTP traffic (and the video traffic).

Figure 10. HTTP transaction latency for multiple runs

A similar trend was seen on the voice calls. The per-call jitter increased an average of 18% between the first and second run.



The impact of background data traffic was also apparent on the video streams.

Figure 11. Channel change/zap delay comparison between multiple runs

The channel change/zap delay increased between runs as the data traffic on the link increased. In deployed systems, however, such behavior can be mitigated by designing and implementing strict QoS policies at the edge of the network. In addition, a per-subscriber QoS model must be used to ensure that different traffic originating from the same subscriber is treated differently by the edge routers. This will allow predictable behavior to ensure that data traffic does not impede other more sensitive traffic such as video streams.

About Ixia

Ixia is a leading provider of performance test systems for IP-based infrastructure and services. Its highly scalable solutions generate, capture, characterize, and emulate network and application traffic, establishing definitive performance and conformance metrics of network devices or systems under test. Ixia’s test systems are used by Network and Telephony Equipment Manufacturers, Semiconductor Manufacturers, Service Providers, Governments, and Enterprises to validate the functionality and reliability of complex IP networks, devices, and applications. Ixia’s Triple Play test systems address the growing need to test voice, video, and data services and network capability under real-world conditions. Ixia’s vision is to be the world’s pre-eminent provider of solutions to enable testing of next generation IP Triple Play networks. Ixia’s test systems utilize a wide range of industry-standard interfaces, including Ethernet, SONET, ATM, and wireless connectivity, and are distinguished by their performance, accuracy, reliability, and adaptability to the industry’s constant evolution.

For more information, contact Ixia or visit our Web Site at http://www.ixiacom.com.

Ixia Worldwide Headquarters

Corporate Center 26601 W. Agoura Rd. Calabasas, CA 91302

(Toll Free North America) 1.877.367.4942 (Outside North America) +1.818.871.1800 (Fax) 818.871.1805

www.ixiacom.com

Info: [email protected]

Investors: [email protected]

Renewals: [email protected]

Sales: [email protected]

Support: [email protected]

Training: [email protected]

Ixia USA SalesPhone: 1.866.355.4942 Email: [email protected]

Ixia Canada SalesPhone: 1.877.367.4942 Email: [email protected]

Ixia China SalesPhone: +86.10.84549199 Email: [email protected]

Ixia Europe, Middle East, & Africa SalesPhone: +44.1753.722056 Email: [email protected]

Ixia India SalesPhone: +91.80.25633570 Email: [email protected]

Ixia Japan SalesPhone: +81.3.5365.4690 Email: [email protected]

Ixia Oceania SalesPhone: 1.818.292.1561 Email: [email protected]

Ixia South KoreaPhone: +82.11.897.1326 Email: [email protected]

Ixia Federal SalesPhone: 1.703.822.7527 Email: [email protected]

Contact Ixia

© 1998-2006 Ixia. All rights reserved.

This publication may not be copied, in whole or in part, without Ixia’s consent.

Ixia and its licensors retain all intellectual property rights in all products identified in this publication. Such products may be covered by one or more patents and/or pending patent applications, including but not limited to the following U.S. patents: 6,717,917; 6,408,335; 6,397,359; 6,061,725; 5,937,165; 5,881,237; and 5,838,919. All software and related documentation identified in this publication is licensed, not sold, pursuant to a separate license agreement between Ixia and the recipient. The recipient’s use of such software and documentation is subject to the terms of that agreement.

Restricted Rights Legend

Use, duplication, or disclosure by the U.S. Government is subject to the restrictions set forth in subparagraph (c)(1)(ii) of the Rights in Technical Data and Computer Software clause at DFARS 252.227-7013 and FAR 52.227-19.

THIS PUBLICATION IS PROVIDED “AS IS” AND WITHOUT ANY WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED. IXIA SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NONINFRINGEMENT. THE INFORMATION HEREIN IS FURNISHED FOR INFORMATIONAL USE ONLY, IS SUBJECT TO CHANGE BY IXIA WITHOUT NOTICE, AND SHOULD NOT BE CONSTRUED AS A COMMITMENT BY IXIA. IXIA ASSUMES NO RESPONSIBILITY OR LIABILITY FOR ANY ERRORS OR INACCURACIES CONTAINED IN THIS PUBLICATION.

Ixia, the Ixia four petal logo, and IxLoad are either trademarks or registered trademarks of Ixia in the United States and/or other countries. All other trademarks belong to their respective owners.

Test Plan IPTV Channel Change Testing

Technology

channel zapping

particular channel

multicast network

single channel multicast

test sut

leavingone tv channel

multicastcapable network

iptv services