Spirent: Improving the VoLTE Experience

Improving the VoLTE Experience: Best Practices from Early Launches

An Infonetics Research Webinar

Co-produced with Spirent Communications

#VoLTE

Today’s Speakers

2

JoAnne Emery

Event Director

Infonetics Research

(Moderator)

Rich McNally

Director

Service Experience Programs

Spirent Communications

Stéphane Téral

Principal Analyst,

Mobile Infrastructure and

Carrier Economics

Infonetics Research

Eric Sinclair

Manager

Service Experience Programs

Spirent Communications

#VoLTE

Agenda

3

Market Trends

3

4

Challenges

Best Practices

Case Studies

Sponsor Background

Conclusions

Audience Q&A

1

2

3

4

5

6

7

3

#VoLTE

Mobile Broadband Rules but Voice Stays

‣ Global mobile revenue forecast:

• US$815.8 billion by year end (+2%

over last year)

• US$852.5 billion in 2017

‣ Broadband is the next growth

engine, surpassing SMS/MMS

revenue this year

• We forecast US$290 billion in 2018,

up 75% from 2013

‣ Voice will still account for >48% of

total mobile service revenue in 2018

$0

$200

$400

$600

$800

$1,000

CY12 CY13 CY14 CY15 CY16 CY17 CY18

Reve

nu

e (

US

$ B

illi

on

s)

Voice SMS/MMS Broadband

© Infonetics Research: 2G, 3G, LTE Mobile Services and Subscribers Market Size and Forecasts, June 2014 4

‣ 1.5 billion active users

worldwide of OTT mVoIP

in 2013…

• Mobile operators using OTT for

subscribers travelling out

of country

• Activity continues to expand in

Japan, China, and South Korea

‣ …that will taper as users

whittle choices down and

over-penetration settles out

‣ Proliferation of smartphones

fueling growth

• We expect OTT mVoIP subscribers to more than double from 2013 to 2018, to 3.1 billion

0

500

1,000

1,500

2,000

2,500

3,000

3,500

CY11 CY12 CY13 CY14 CY15 CY16 CY17 CY18

Su

bsc

rib

ers

(M

illi

on

s)

OTT mVoIP VoLTE

5 Infonetics Research: 2G, 3G, LTE Mobile Services and Subscribers Market Size and Forecasts, June 2014

Although OTT Dominates mVoIP…

…the Revenue Model Remains Challenging

‣ In 2013, 1.5 billion mVoIP OTT

subscribers contributed

$10 billion in revenue

• Successful providers are using

the app as a platform to upsell

• Revenue declines as users

eliminate multiple OTT apps

‣ VoLTE is starting to have an

impact but values are small; only

12 operators have commercial

service so far

‣ We forecast combined native and OTT

mobile VoIP to grow to $24.7 billion in 2018

• Based on 3.4 billion mVoIP subscribers, 90% of which are OTT

$0

$5

$10

$15

$20

CY11 CY12 CY13 CY14 CY15 CY16 CY17 CY18

Reve

nu

e

(US

$ B

illi

on

s)

OTT mVoIP Native mVoIP

6 © Infonetics Research: 2G, 3G, LTE Mobile Services and Subscribers Market Size and Forecasts, June 2014

0

20

40

60

80

100

120

140

2009 2010 2011 2012 2013 2014

Cumulative W-AMR HD Voice Networks

HSPA commercial VoLTE commercial

‣ 119 mobile operators offer HD

voice with W-AMR* as key

HD voice enabler

• 95 on HSPA

• 12 on LTE

(VoLTE + IR.94 video calling)

• 8 on GSM/HSPA

• 2 on HSPA/LTE

• 2 on GSM

‣ 92 smartphones (including

carrier and frequency variants)

support VoLTE

• Including products by Asus, Huawei, LG, Pantech, Samsung, and Sony Mobile

• The new Apple iPhone 6 & 6 Plus models support VoLTE

Source: GSA, September 23, 2014

7 *W-AMR: Wideband Adaptive Multi-Rate codec

VoLTE Taking Off, Marketed as HD Voice…

…and SK Telecom Fuels the Momentum

‣ 20 commercial VoLTE networks and 51 million VoLTE subscribers this year

‣ Worldwide revenue of $1.2 billion at the current voice ARPU

• In spite of SK Telecom’s VoLTE adoption (8M+ subscribers in May 2014)…

• …no evidence VoLTE is increasing ARPU (despite SKT charging a premium!)

‣ By 2018 we expect ~325 million VoLTE subscribers and $7 billion in revenue

8

0

50

100

150

200

250

300

350

$0

$2

$4

$6

$8

CY12 CY13 CY14 CY15 CY16 CY17 CY18

Su

bsc

rib

ers

(M

illi

on

s)

Reve

nu

e (

US

$ B

illi

on

s)

VoLTE Market Opportunity

© Infonetics Research: 2G, 3G, LTE Mobile Services and Subscribers Market Size and Forecasts, June 2014

The Feedback Is Extremely Positive so Far

‣ Connection is super fast, ringtone sound quality exceptional

‣ Virtually no background noise; voice is crystal clear

‣ Video call quality is crisp and sound

• Superior to Skype and other OTT apps

9

The ultimate goal is to keep every subscriber on your network

with the carrier-grade service features that beat OTTs

There Is Little Surprise!

‣ VoLTE is reliable

• Offers guaranteed QoS

‣ VoLTE provides high quality

• Uses W-AMR codecs

(12.65kbps or 23.85kpbs)

and QoS Class Identifier

(QCI) SIP signaling

‣ VoLTE is spectrally efficient

• Due to LTE’s all-IP architecture and new features in 3GPP releases —

such as MIMO antenna technology — voice requires less bandwidth in

LTE spectrum than it does in 2G/3G networks

10

Source: Nokia Networks, 2012

Agenda

11

Market Trends

3

4

Challenges

Best Practices

Case Studies

Sponsor Background

Conclusions

Audience Q&A

1

2

3

4

5

6

7

11

#VoLTE

The Experience Matters

13

‣ VoLTE needs to deliver a better user

experience than OTT and legacy services

• There is only one way to truly know if new devices

and services are ready to launch

• You need to measure the experience at the

device, using actual consumer devices

‣ Spirent has been in the trenches, rolling out

VoLTE with leading operators for 2 years

• Following are some best practices for evaluating

and improving VoLTE…

• …to ensure a successful launch of new devices

and services

Agenda

14

Market Trends

3

4

Challenges

Best Practices

Case Studies

Sponsor Background

Conclusions

Audience Q&A

1

2

3

4

5

6

7

14

#VoLTE

Practice #1: Measure What Matters

15

‣ Our experience:

• Functional testing is increasingly insufficient for

assuring user experience

• Combined live network and lab evaluation of user

experience (UX) is a better approach

• If you don’t measure everything that matters to

users, problems will emerge

‣ Best practices:

• Focus on objective assessment of what the

user experiences

• Measure all key factors impacting user experience

• Measure quality and consistency

MOS (wideband

mobile-to-mobile and

narrowband landline)

Mobile Originated and

Terminated Block/Drop

Rates

Conversational Audio

Delay

Video Delivery

Audio / Video Sync

Battery Life

VoLTE UX Metrics

Practice #2: Stress the Device Like a User

16

‣ Our experience:

• Measurements of UX are only valid if they

reflect real user behavior

• Multi-service voice and data usage

stresses the device

• Dropped calls and other UX elements are

impacted by multi-service usage

‣ Best practices:

• Implement multi-service use cases for

pre-launch device evaluation

• For example, set up continuous push

emails during test calls

1.1% 0.9%

1.1%

Manufacturer 1 Manufacturer 2 Manufacturer 3

Voice only Multi-service

Dropped Call Rate by

Device Manufacturer

Voice Only Vs. Multi-Service

1.3% 1.4%

1.9%

Practice #3: Rank Devices (and Services)

17

‣ Our experience:

• There is a wide variance in UX across

device models

• It’s a competitive marketplace - showing

rank by UX category is a powerful motivator

‣ Best practices:

• Consider the relative performance of

devices and not just the absolute

• Rank and compare all pre-launch devices

by UX category

• Set thresholds based on population

performance (and raise over time)

No one wants to be in this range

Speech Quality

Downlink Uplink

Device A

Device A

Device B Device B

Device D

Device D

Device C

Device C

Device E

Device E

Practice #4: Assess from the Top Down

18

‣ Our experience:

• You can spend a lot of time and money

gathering stack-loads of data you don’t use

• Broad statistical assessments with focused

drilldown into problems saves time

• RTP and RF tracing can provide key insight into

root causes of poor experience

‣ Best practices:

• Only collect RTP/IP and RF DM logs at UE

(and IMS) where problems are identified or in

focused pre-planned instances

• Use the data to accelerate triage/isolation of

UX issues to network/device/service

0

10

20

30

40

50

60

200 300 400 500 600

Packets

/s

Received Packets/s Missing Packets/s

HO HO

Isolating the Root Causes of

Poor VoLTE Speech Quality

0

1

2

3

4

5

0 100 200 300 400 500 600 700 800

Do

wn

lin

k M

OS

RTP Metrics Help Identify Cell-Specific Issue

Agenda

19

Market Trends

3

4

Challenges

Best Practices

Case Studies

Sponsor Background

Conclusions

Audience Q&A

1

2

3

4

5

6

7

19

#VoLTE

Case Study: VoLTE Speech Quality

20

‣ Drive test of live US VoLTE

network with varying RF

‣ MOS very flat across good, fair

and poor RF environments

• Average speech quality mean

opinion score (MOS) of 3.5

• In very poor RF conditions, MOS

tends to degrade rapidly

‣ RTP jitter and delay also flat

across varying RF conditions

• In very poor RF, RTP performance

drops off rapidly

22.1

17.2 16.8 16.2 16.6 16.2 15.9

-105 -100 -95 -90 -85 -80 -75

Average RTP Jitter (ms) by RSRP (dBm)

3.55 3.53 3.46 3.53 3.48 3.52

Good Fair Poor

Device 1

Device 2

VoLTE MOS (POLQA) by RF Environment

Case Study: Typical Causes of Poor MOS

21

‣ Week-long drive test of VoLTE in

major US city

‣ RF issues are most common and

fairly easy to detect

• Distant or no dominant server

• Interference / weak server

• Dragging handover

‣ IP (RTP) issues also common

• Configuration / IP connectivity

‣ Other issues

• No apparent RF or IP issues

• Device issue related to buffering,

packet re-assembly or codec

RF 79%

IP 16%

Other 5%

Root cause of VoLTE MOS < 2.5 by category

for week long drive test of major US city

‣ Are problems due to:

device, network, or interaction?

‣ Comparative scorecards help

identify the source of issues

‣ Scorecard on right comes from

tests on a live VoLTE network

• Network RF conditions are good

• Device 2 is less efficient at using

air-interface resources, but…

• RTP performance is okay

• Ideal MOS from RTP stream shows

speech quality should be good

• Device 2 jitter buffer or codec is

not working properly!

Case Study: Isolating Root Causes

22

VoLTE Analytics Scorecard

Overall Rank Device 1 Device 2

User Experience

Speech Quality

Network – RF Conditions

Received Power (RSRP)

Signal Quality (RSRQ)

Interference (CINR)

Device-Network Interaction

Resource Allocation

Modulation Efficiency

RTP – Packet Jitter

RTP – Packet Delay

RTP – Packet Loss

Device Buffering and Decoding

Ideal MOS from RTP

Actual - Ideal MOS

Case Study: Isolating Root Causes

23

‣ Used RTP analysis to isolate issue

with service infrastructure / IP layer

‣ Collected IP sniffer data via device and

server-side logs in area with low MOS

• RF is good

• RTP jitter and delay is okay

• RTP packet loss rate very high

‣ Diagnosis

• The network is working and packets are

getting through but…

• There is a major IP connectivity issue

leading to dropped packets

-86.3

-98.7

Serving Cell Neighbor Cell

RSRP

One location: VoLTE MOS < 2.5 with

good RF but packet loss rate > 9%!

RTP Jitter: 14 ms

RTP Delay: 20 ms

RTP PLR: 9.3%

RF

RTP

RSRQ: -12.3

CINR: 4.6

Case Study: Making Improvements

24

‣ Results from live network testing

of VoLTE and IR.94 video

‣ Key factors that matter to users:

• Speech quality (MOS)

• Frozen or impaired frame rate

‣ Prior to optimization, VoLTE

performs poorly

‣ After optimization VoLTE is 1st

or 2nd best in all UX categories

3.4%

13.2%

19.5%

25.3%

Frozen/Impaired Frames

IR.94 (Optimized)

OTT 2

OTT 1

IR.94 (Pre-optimization)

Frozen/Impaired Frames for IR.94 Video vs. OTT

2.4

3.0

3.3

3.6 3.8

Speech Quality (POLQA MOS)

OTT 1

VoLTE (Pre-optimization)

Circuit-Switched

VoLTE (Optimized)

OTT 2

VoLTE vs. OTT and CS Speech Quality (HD Codec)

Optimization

improvement

Optimization

improvement

Agenda

25

Market Trends

3

4

Challenges

Best Practices

Case Studies

Sponsor Background

Conclusions

Audience Q&A

1

2

3

4

5

6

7

25

#VoLTE

26

Partnering with service providers and device manufacturers to

improve the user experience of devices and services

Over 2 years F4L program

led to decreased variance

between devices, more

consistent speech quality

F4L program also led to

better average speech

quality across portfolio

Effect of Fit4Launch on Portfolio Speech

Quality over first 2 years

Fit4Launch programs for

evaluating user experience

of pre-launch devices are

deployed at 3 US operators

Improving User Experience

27

Inventing systems and methodologies for measuring and analyzing

the user experience of mobile devices and services

Inventing Systems and Methods

Location

2003 2008 2010 2012 2013 2014

Live Lab

Spirent User Experience Analytics timeline

Speech File Transfer &

Web Browsing Call Battery Life

Voice & Video

Calling

2011

Video

Agenda

28

Market Trends

3

4

Challenges

Best Practices

Case Studies

Sponsor Background

Conclusions

Audience Q&A

1

2

3

4

5

6

7

28

#VoLTE

‣ With the RF in order, beat the OTT with voice and video calling

over LTE!

• Video calling over LTE is very challenging because it requires low

latency, low error rate and high bit rate

• With the GSMA toolbox, IR.92 and IR.94, when the LTE network is

congested, VoLTE and video calling services can use LTE QoS

features to prioritize packets to ensure service quality

- Pure OTT can’t

‣ With 71 operators currently investing in VoLTE studies, trials or

deployments, it’s only a matter of time for mobile operators to

retake their destiny

29

The VoLTE Advantage

Thank you for your interest!

30

Watch the full webinar on the link given below:

http://www.spirent.com/OnDemand/Service_Exp

erience/Improving-the-VoLTE-Experience