Key Performance Indicators - ST3 Telkomalfin.dosen.st3telkom.ac.id/.../uploads/sites/8/2015/12/LTE-KPIs.pdf · Introduction 1 Introduction This document describes the Key Performance

Key Performance Indicators

USER DESCRIPTION

37/1553-HSC 105 50/1 Uen G

Copyright

© Ericsson AB 2009–2011. All rights reserved. No part of this document may bereproduced in any form without the written permission of the copyright owner.

Disclaimer

The contents of this document are subject to revision without notice due tocontinued progress in methodology, design and manufacturing. Ericsson shallhave no liability for any error or damage of any kind resulting from the useof this document.

Trademark List

All trademarks mentioned herein are the property of their respective owners.These are shown in the document Trademark Information.

37/1553-HSC 105 50/1 Uen G | 2011-09-12

Contents

Contents

1 Introduction 1

1.1 Concepts 1

1.2 Dependencies and Associated Features 5

2 Key Performance Indicators 7

2.1 Accessibility 7

2.2 Retainability 9

2.3 Integrity 11

2.4 Mobility 15

2.5 Availability 16

37/1553-HSC 105 50/1 Uen G | 2011-09-12


37/1553-HSC 105 50/1 Uen G | 2011-09-12

Introduction

1 Introduction

This document describes the Key Performance Indicators (KPIs) for the LTERadio Access Network (RAN) used to measure the contribution to subscriberperceived quality and system performance.

Monitoring the RAN performance is a very important task for operation andmaintenance personnel, network engineers, and management.

KPIs can be used for the following tasks:

• Monitoring and optimizing the radio network performance to provide bettersubscriber-perceived quality or better use of installed resources

• Rapidly detecting unacceptable performance in the network, enabling theoperator to take immediate actions to preserve the quality of the network

• Providing radio network planners with the detailed information required fordimensioning and configuring the network for optimal use

• Troubleshooting on cell clusters of interest

The information in this document reflects the KPIs at the time of its release.

Note: In the Managed Object Model RBS(MOM) there are a few MO classes,parameters, and counters listed that are there because system designconsiders future aspects. These MOs, parameters, and countersare therefore not relevant nor supported in the current release. Forinformation about unsupported MO classes, parameters, and counters,see Parameter and Counter Limitations.

1.1 Concepts

Concepts related to observability are described in the following sections.

1.1.1 LTE RAN Contribution to End-User Performance

It is important to understand that in the LTE RAN network, KPI definition islimited to the end-user performance of which the network is aware. An LTERAN KPI related to end-user impact is defined to measure the contributionknown to the LTE RAN network.

The performance of end-user applications covers a broader area than the LTERAN part, as illustrated in the following figure:

137/1553-HSC 105 50/1 Uen G | 2011-09-12


L0000266A

Applicationlayer

UE RBS TN TNSGW PGW Server

Figure 1 LTE RAN as Part of Chain to Deliver Packets between Entities on Application Layer

Abbreviations used in the illustration above are defined in the following list:

PGW Packet Data Network Gateway

SGW Serving Gateway

TN Transport Network

UE User Equipment

1.1.2 Observability in Ericsson LTE

Observability covers all functions in LTE RAN that monitor and analyze theperformance and characteristics of the network. This can be done on variouslevels with different target groups and requirements. The figure below illustratesa model for observability in LTE RAN.

L0000269A

End-userperception

SystemCharacteristics

Key Performance Indicator (KPI) level

Performance Indicator (PI) level

Procedure level

Figure 2 Top-Down Approach in LTE RAN Observability

2 37/1553-HSC 105 50/1 Uen G | 2011-09-12

Introduction

The model shows different levels of observability targeting different purposes:

Table 1 Levels of Observability

ObservabilityLevel

Description of Use

KPI The KPI represents the end-user perception of anetwork on a macro level. KPIs are of interest for anoperator's top-level management. KPI statistics aretypically used to benchmark networks against eachother and to detect problem areas. KPIs are calculatedfrom PM(1) counters. The reliability, granularity, andaccuracy of the data are critical, and the data iscollected continuously.

PI(2) The PI normally represents information at the systemlevel that explains the KPI results. Many PIs can bebased on PM counters, for example available PMcounters for Root Cause Analysis, see Managed ObjectModel RBS. The PI can also be in the form of metricsthat show how specific parts of a system perform.PIs do not necessarily have an impact on KPIs. ThePI data can be used for planning and dimensioning.This data, typically PM counters, is normally collectedcontinuously.

Procedure The Procedure level represents in-depthtroubleshooting and measurement systemcharacteristics. It involves measurements on signalling,procedure and function levels to investigate problemsdetected at higher levels.

The amount of data at the Procedure level is enormousand the measurements are generally user-initiated fora specific purpose and area of the network to limit thescope of the measurements. The typical source forthis data is the UE(3) trace and the Cell trace recordingfunctions.

(1) Performance Management(2) Performance Indicator(3) User Equipment

For further information about UE trace and cell trace, see the documentPerformance Management.

1.1.3 ITU-T QoS Model

The International Telecommunications Union - Telecommunications (ITU-T)has described a general model for Quality of Service (QoS) from an end-userperspective to use in mobile networks.

337/1553-HSC 105 50/1 Uen G | 2011-09-12


The QoS categories for Serveability are:

Accessibility The ability of a service to be obtained, within specifiedtolerances and other given conditions, when requestedby the end user.

Retainability The probability that a service, once obtained, continuesto be provided under given conditions for a given timeduration.

Integrity The degree to which a service is provided withoutexcessive impairments, once obtained. Service Integrityrepresents the quality experienced by the end userduring the call or session.

1.1.4 LTE RAN Performance Observability Model

The LTE Radio Access Network (RAN) Performance Observability model usedby Ericsson combines the ITU-T QoS categories of Accessibility, Retainability,and Integrity with Mobility, Availability and Utilization, as described below:

Mobility The ability of the system to allow movement within theLTE RAN.

Availability The ability of an item to be in a state to perform arequired function at a given instant of time within a giventime interval, assuming that the external resources, ifrequired, are provided.

Utilization Describes the network use by measuring traffic level andcapacity resource management, including congestion,admission and load control, and license use. Utilizationinformation is required as input to network planning.For formulas to calculate utilization, see License andResource Use Indicators.

The ITU-T specification focuses on Circuit Switched (CS) calls, that is, one callper end user and one service only. Telecommunications have evolved pastpure CS connections, and in LTE all sessions are Packet Switched (PS). TheLTE network supports multiple simultaneous services per user, so the ITU-Tdefinitions should be used as guidelines rather than rules for performancemeasurement.

KPIs are developed for observing the network performance impact on the enduser, and for observing the performance of the network itself. Each KPI isdefined for observing end-user impact or system performance.

1.1.5 Aggregation of KPIs

For KPIs, the majority of the equations are focused on cell level. The operatormay choose to aggregate the counters over a group of cells and use the same

4 37/1553-HSC 105 50/1 Uen G | 2011-09-12

Introduction

equations to calculate the KPI over a cluster of cells, one RBS, or multipleRBSs.

Example of how a pmXYZ(EUtranCellFDD) counter in an equation can bereplaced to obtain a metric over a larger area:

��

��

��

Equation 1 Formula for Aggregation of Metrics to Clusters of Cells

For most formulas the � is left out. It is only in cases where it serves to clarifythat it is used.

The same aggregation is performed over time. Each counter is reported every15 minutes. The operator may aggregate them over one hour, 24 hours, oranother defined time period. In that case, the operator can replace the MOpmXYZ(EUtranCellFDD) with its time-aggregated sum.

When a formula, involves the sum of fractions, the result of the formula indicatesan invalid result when the results are invalid for all fractions involved. Whenthe result is invalid for less than all fractions involved in the formula, the invalidfractions are ignored. E.g. the result of the formula in Equation 2 presentsan invalid result when the results of pmA/pmB, pmC/pmD and pmE/pmF areinvalid. E.g. the result of the formula in Equation 2 is pmC/pmD + pmE/pmFwhen the result of pmA/pmB is invalid.

��

��

��

��

Equation 2 Formula Example

1.2 Dependencies and Associated Features

There are no specific dependencies and associated features for KPIs.

537/1553-HSC 105 50/1 Uen G | 2011-09-12


6 37/1553-HSC 105 50/1 Uen G | 2011-09-12


2 Key Performance Indicators

The following sections describe the KPIs available, organized by category.

2.1 Accessibility

In providing wireless end-user services, the first step is to get access to thewireless service. After access to the service is performed, the service can beused. The service provided by LTE RAN for accessibility is the Evolved RadioAccess Bearer (E-RAB).

The success rate E-RAB establishment is calculated separately dependingon if the E-RAB is established with the E-RAB setup or Initial Context setupprocedure, the latter depends on the successful establishment of the RRCconnection and S1 signalling between the RBS and the Mobile ManagementEntity (MME).

For additional information about the Initial Context Setup and E-RAB Setupprocedures, see the document Radio Bearer Service.

2.1.1 Initial E-RAB Establishment Success Rate

This KPI measures the impact on the end user.

The following equation gives the accessibility success rate for end-userservices that are carried by E-RABs included in the Initial UE Context setupprocedure. The counters are on cell level.

��

� ��

��

�

� ��

� ��

�

� ��

� ��

Equation 3 Initial E-RAB Establishment Success Rate

Note: The impact of multiple E-RABs in the Initial context setup procedure isignored in the RRC and S1 signalling part contribution.

737/1553-HSC 105 50/1 Uen G | 2011-09-12


2.1.2 Initial E-RAB Establishment Success Rate per QCI


The following equation gives the accessibility success rate for end-userservices that are carried by E-RABs included in the Initial UE Context setupprocedure. The counters are on cell level and per QCI.

��

� ��

��

�

� ��

� ��

�

� ��

� ��

Equation 4 Initial E-RAB Establishment Success Rate per QCI

Note: The impact of multiple E-RABs in the Initial context setup procedure isignored in the RRC and S1 signalling part contribution.

2.1.3 Added E-RAB Establishment Success Rate


Accessibility success rate for end-user services which is carried by E-RABsincluded in the E-RAB setup procedure is given by the following equation.The counters are on cell level.

��

� ��

Equation 5 Added E-RAB Establishment Success Rate

2.1.4 Added E-RAB Establishment Success Rate per QCI


Accessibility success rate for end-user services which is carried by E-RABsincluded in the E-RAB setup procedure is given by the following equation.The counters are on cell level per QCI.

��

� ��

Equation 6 Added E-RAB Establishment Success Rate per QCI

8 37/1553-HSC 105 50/1 Uen G | 2011-09-12


2.2 Retainability

In providing wireless end-user services, it is important that the services arenot interrupted or ended prematurely.

Retainability performance can be divided into two parts, the definition ofabnormal and the normalizing factor.

937/1553-HSC 105 50/1 Uen G | 2011-09-12


Table 2 Retainability Aspects

Aspect ofRetainabilityperformance

Description

Definition of anAbnormal Release

In a packet switched system as LTE it is natural toestablish and release E-RABs. E-RABs don't have tobe released just because they are inactive, they canbe kept to have a fast access once new data arrives.The definition of an Abnormal release shall be that therelease of the E-RAB had a negative impact on theend-user, hence the definition of an abnormal releaseis that there shall be buffered data to be transmitted atthe time of release. The following causes for releaseshall be excluded from the abnormal release definition:

• The release cause was considered "Normal", e.g. CSFallback initiated.

• The E-RAB was released due to successful handover.

Normalizationfactor for theAbnormal Releases

The number of Abnormal releases have to benormalized with something to make sense.

The traditional approach from the speech world havebeen to compare Abnormal releases with the totalnumber of releases. This method is suitable for asystem where the session length is stable, i.e. a call isa call and the only information wanted is to see if thecall could be completed without an abnormal release.

In a packet switched system, where the service mixin a system is unknown and hence the session lengthvaries a lot, the comparison of Abnormal releaseswith the total number of releases does not work welland doesn't reflect the end-user satisfaction as wellas it does for speech. E.g. one user using p2p fordownloading movies during 24 hours. One abnormalrelease during the 24 hours is hardly considered badfrom the end-user perspective, even if the 100% ofthe released E-RABs for this example was consideredabnormal. If we for this case would compare thesession length with the number of Abnormal Releases,it would well reflect the end-user satisfaction.

Considering this it is recommended to keep trackof two Retainability measurements, one where thenormalization is session time and one where thenormalization is the total number releases.

10 37/1553-HSC 105 50/1 Uen G | 2011-09-12


2.2.1 E-RAB Retainability - Session Time normalized

This KPI measures the impact on the end user with the purpose to reflect theprobability of how long a transfer in average can be kept without an abnormalrelease.

The Retainability rate for E-RAB is given by the following equations. Thecounters are on cell level.

� ��

��

Equation 7 E-RAB Retainability - Session Time normalized

Note: Since the KPI measures the impact of the network on the end user, italso includes releases initiated by the MME. To observe the impact ofthe RBS only, remove the pmErabRelMmeAct from the formula.

To achieve the number of minutes per drop, the multiplicative inverse ofthe E-RAB Retainability together with unit transformation from secondsto minutes can be used.

2.2.2 E-RAB Retainability - Percentage

This KPI measures the impact on the end user with the purpose to reflect theprobability if an established E-RAB can be can be kept without an abnormalrelease.

The Retainability rate for E-RAB is given by the following equations. Thecounters are on cell level.

� ��

��

Equation 8 E-RAB Retainability - Percentage

Note: Since the KPI measures the impact of the network on the end user, italso includes releases initiated by the MME. To observe the impact ofthe RBS only, remove the pmErabRelMmeAct from the formula.

2.3 Integrity

In providing wireless end-user services, it is important that the end-userperformance quality meets expectations. The LTE RAN service is deliveryof IP packets.

Integrity performance can be divided into three parts:

1137/1553-HSC 105 50/1 Uen G | 2011-09-12


Table 3 End-User Performance Quality

Aspect ofPerformanceQuality

Description

Latency The time it takes to schedule the first packet on the airinterface, determined from the time it was received inRBS. The uplink (UL) Latency cannot be measured inLTE RAN(1)

Throughput The speed at which packets can be transferred oncethe first packet has been scheduled on the air interface.

Packet Loss Packet Loss Rate can be broken down into:

• the rate of congestion related packet losses (forexample, the packets that get dropped due to activequeue management functionality)

• the rate of non-congestion related packet losses(those are packets that get lost in transmission, forexample, discarded by some link layer receiver dueto CRC failure)

(1) Only the UE has the information when data arrives to an empty UL buffer.

Latency is defined to exclude the impact of the traffic model. The size of the firstpacket shall not make a difference, so the measurement considers when thepacket is scheduled on the air and not when the whole packet is acknowledged.

Throughput is defined to exclude the impact of the traffic model. The file size ofthe transfer and the amount of bursts does not make a difference, so the idletime between transfer bursts is removed, as shown in the following figure:

L0000268A

Session

T_IdleBuffered data

Figure 3 Example of Burst in Transfer

The following figure shows the removal of the contribution from the last TTI indownlink to exclude the impact of the traffic model:

12 37/1553-HSC 105 50/1 Uen G | 2011-09-12


L0000267A

Failed transmission(”block error”)

No transmission, buffer notempty (e.g. due to contention)

Successful transmission,buffer not emptySuccessful transmission,buffer empty

Latencysample

Time used forthroughput calculation

The last TTI with datashall always beremoved. This since thecoding can be selectedbased on size, not radioconditions, hence notend-user impacting.

Data arrives toempty DL buffer

First data is transmittedto the UE

The send buffer isagain empty

Time (ms)

Figure 4 Example of Downlink Throughput and Downlink Latency forCalculating Contribution of One Burst

The following figure shows the method to exclude the impact of the trafficmodel on the uplink throughput metric (to reuse the method that is used indownlink with last TTI does not work well in uplink due to the time difference inscheduling decision by RBS and the transmission time by the UE):

L0000411A

d receptions (“block error”)

Successful receptions, buffer not empty

Receptions excluded from throughput calculations

Successful receptions, buffer empty

No receptions, buffe not empt or example

Faile

r y (f due to contention)

Contribution from thelast TTI and the 4 firstreceptions is removed.This to exclude trafficmodel impact.

Time and volumeused for throughputcalculation

Time (ms)Data arrives toempty UL buffer

Grants sent to UE

First data is transmittedto the RBS

Scheduling requestsent to RBS

The send buffer isagain empty

Figure 5 Example of Uplink Throughput for Calculating Contribution of OneBurst

1337/1553-HSC 105 50/1 Uen G | 2011-09-12


In the downlink (DL), the KPI is defined to look at packet losses which have anegative impact on the end-user performance, that is, the Packet Error Lossrate on non-congestion related packet losses.

In the UL, the lost packets are derived from PDCP sequence numbers from theUE, hence it is not possible for the RBS to distinguish if a packet is lost due tocongestion or non-congestion related reasons. Hence the UL KPI is defined asPacket Loss (and not Packet Error Loss as in DL).

2.3.1 Downlink Latency


DL latency for the UE is given by the following equation. The counters areon cell level.

��

��

Equation 9 Average UE DL Latency

2.3.2 Downlink Latency per QCI


DL latency for the UE is given by the following equation. The counters areon cell level per QCI.

��

��

Equation 10 Average UE DL Latency per QCI

2.3.3 Downlink Throughput


DL throughput for the UE is given by the following equation. The countersare on cell level.

��

��

��

Equation 11 Average UE DL Throughput

14 37/1553-HSC 105 50/1 Uen G | 2011-09-12


2.3.4 Uplink Throughput


UL throughput for the UE is given by the following equation. The countersare on cell level.

��

� ��

Equation 12 Average UE UL Throughput

2.3.5 Downlink Packet Error Loss Rate


DL packet error loss rate for the UE is given by the following equation. Thecounters are all on cell level, except pmPdcpPktDiscDlEth which is on RBSlevel.

��

� ��

� ��

Equation 13 Average UE DL Packet Error Loss Rate, from a cell perspective

where

� � � ��

��

��

2.3.6 Uplink Packet Loss Rate


UL packet loss rate for the UE is given by the following equation. The countersare on cell level.

��

� ��

Equation 14 Average UE UL Packet Loss Rate

2.4 Mobility

This section describes the KPI for mobility success rate.

1537/1553-HSC 105 50/1 Uen G | 2011-09-12


2.4.1 Mobility Success Rate

This KPI measures system performance.

The Mobility Success rate includes both preparation of target cell resourcesand move from the source cell to the target cell, as given by the followingequation. The counters are on the MO EUtranCellRelation orUtranCellRelationlevel:

��

��

��

��

�

�

��

��

�

Equation 15 Mobility Success Rate

2.5 Availability

This section describes the KPI for availability.

2.5.1 Partial Cell Availability (node restarts excluded)

This KPI measures system performance. Since the KPI is measured by theRBS, it does not include time when the RBS is down, i.e. node restart timeis excluded.

The length of time in seconds that a cell is available for service is defined ascell availability. Cell availability for a cluster of L number of cells during Nreporting periods can be calculated using the following formula. The countersare on cell level.

��

��

��

� � ��

Equation 16 Cell Availability

Note: The manual blocking time of a cell is included in this KPI to show theoverall availability of the cell. To remove the manual intervention impacton cell availability, remove the PM counter pmCellDowntimeManfrom the numerator and subtract the value of the PM counterpmCellDowntimeMan from the denominator.

Note: If the files with the PM counters are missing, the time that those filesrepresent in "NxMx900" shall be excluded from Cell Availability result.

16 37/1553-HSC 105 50/1 Uen G | 2011-09-12

Key Performance Indicators - ST3 Telkomalfin.dosen.st3telkom.ac.id/.../uploads/sites/8/2015/12/LTE-KPIs.pdf · Introduction 1 Introduction This document describes the Key Performance

Documents