This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
8/13/2019 LTE E2E Field Network Performance - KPI Definitions RL10, Customer Version
LTE E2E Field Network Performance - Definitions of Key Performance Indicators
Id:0900d80580729625
The information in this document is subject to change without notice and describes only the
product defined in the introduction of this documentation. This documentation is intended for the
use of Nokia Siemens Networks customers only for the purposes of the agreement under whichthe document is submitted, and no part of it may be used, reproduced, modified or transmitted
in any form or means without the prior written permission of Nokia Siemens Networks. The
documentation has been prepared to be used by professional and properly trained personnel,
and the customer assumes full responsibility when using it. Nokia Siemens Networks welcomes
customer comments as part of the process of continuous development and improvement of the
documentation.
The information or statements given in this documentation concerning the suitability, capacity,
or performance of the mentioned hardware or software products are given "as is" and all liability
arising in connection with such hardware or software products shall be defined conclusively and
finally in a separate agreement between Nokia Siemens Networks and the customer. However,
Nokia Siemens Networks has made all reasonable efforts to ensure that the instructions
contained in the document are adequate and free of material errors and omissions. Nokia
Siemens Networks will, if deemed necessary by Nokia Siemens Networks, explain issues which
may not be covered by the document.
Nokia Siemens Networks will correct errors in this documentation as soon as possible. IN NO
EVENT WILL Nokia Siemens Networks BE LIABLE FOR ERRORS IN THIS DOCUMENTA-
TION OR FOR ANY DAMAGES, INCLUDING BUT NOT LIMITED TO SPECIAL, DIRECT, INDI-
RECT, INCIDENTAL OR CONSEQUENTIAL OR ANY LOSSES, SUCH AS BUT NOT LIMITED
TO LOSS OF PROFIT, REVENUE, BUSINESS INTERRUPTION, BUSINESS OPPORTUNITY
OR DATA,THAT MAY ARISE FROM THE USE OF THIS DOCUMENT OR THE INFORMATION
IN IT.
This documentation and the product it describes are considered protected by copyrights and
other intellectual property rights according to the applicable laws.
The wave logo is a trademark of Nokia Siemens Networks Oy. Nokia is a registered trademark
of Nokia Corporation. Siemens is a registered trademark of Siemens AG.
Other product names mentioned in this document may be trademarks of their respectiveowners, and they are mentioned for identification purposes only.
LTE E2E Field Network Performance - Definitions of Key Performance Indicators
Id:0900d8058070a69b
Introduction
support of this service that stands in focus of the first commercial NSN LTE product
release.
In addition, KPIs for the radio link (RB Service, group C - Radio Bearer Service) are also
specified, because the radio link is the most critical component of the E2E transportpath. The other bearers, the E-UTRAN Radio Access Bearer (E-RAB) and Evolved
Packet System (EPS) bearers are composed of the RB Service and of the IP based
bearer services (of group D - IP based LTE Bearer Services: X2, S1, S5/S8 and Exter-
nal).
Monitoring the performance of all bearer services is necessary to fully understand the
E2E transport behavior of the LTE network and to optimize the end-user perceived
quality of application services running on top of it.
Figure 1 LTE Bearer Service Architecture
While the KPI definitions (see 3 Definitions of Key Performance Indicators (KPIs)) are
independent of LTE product releases, chapter 2.3 Measurement Scenarios focuses on
the first product release in order to restrict the number of possible scenario combinations(e.g. by not considering scenarios with Guaranteed Bit Rate (GBR) bearers).
8/13/2019 LTE E2E Field Network Performance - KPI Definitions RL10, Customer Version
LTE E2E Field Network Performance - Definitions ofKey Performance Indicators
Measurement Methods and Reference Conditions
Id:0900d8058072d090
2 Measurement Methods and Reference Condi-
tions
2.1 Introduction
The measurement method to be applied for field performance verification depends on
the specifics of the measured Key Performance Indicator (KPI). End-user perceived
KPIs are measured differently from network performance KPIs, and the conditions for
delay measurements are different from those for accessibility KPIs. For this reason, this
Section discusses measurement methods according to KPI categories.
Since the end-user experience of application services is mostly dependent on through-
put and latency, these aspects together with Handover (HO) scenarios constitute the
core part of field performance verification measurements.
While the scope of KPI definitions follows the performance goals of the LTE system (fordetails see References7. [3GPP25.913]), the selected methods and conditions of field
performance tests have to ensure that typical conditions in operational networks are
covered. Additionally, the measurement conditions also include relevant impairment
factors of the radio link, because the radio link has the most influence on E2E perfor-
mance. Especially, those characteristics of the Orthogonal Frequency-Division Multi-
plexing (OFDM) encoding technique, which have side effects for radio network
performance are considered in the selection of conditions, e.g.:
• Sensitivity to Doppler shift
• Sensitivity to frequency synchronization problems
• Sensitivity to multipath effects and cell area (urban, sub-urban, rural)
• Dependency on used bandwidth and frequency range
• Dependency on used multiple antenna techniques - Multiple Inputs Multiple Outputs
(MIMO) (transmit diversity, spatial multiplexing)
These aspects have a major bearing on the design of measurement scenarios. For
example, investigating the impact of Doppler shift on network performance, e.g. on user
and cell data rates, on delay and packet loss KPIs requires that high speed movement
becomes part of the measurement scenario. Another aspect would be the multipath
reception, which decreases the impact of the Doppler shift. Thus, the measurement
setup should also include space diversity if worst-case conditions are being discussed
measurements with these conditions have to show, how the investigated KPI figure
depends of the velocity and terminal category of the receiving user.
Some of the measurement scenarios might be much too costly to implement in live
network environments, or it may be impossible to ensure the desired radio conditions
during the whole duration of the test. In such cases, lab trials with simulated radio links
are acceptable as replacement of live network tests.
The combination of all different options for reference conditions and measurement sce-
narios would result in a very large number of test cases, which is simply not feasible. It
will be the task of test specifications to select options and scenarios out of the outlined
possibilities for a given LTE performance verification campaign considering the avail-
able resources and time constraints for test execution.
8/13/2019 LTE E2E Field Network Performance - KPI Definitions RL10, Customer Version
LTE E2E Field Network Performance - Definitions of Key Performance Indicators
Id:0900d805807292c5
2.2 Reference Conditions
Performance figures can only be interpreted correctly if the exact conditions under which
they have been acquired are known. This chapter describes the "reference conditions"
under which LTE field performance tests should be executed. They are formulated onan abstract level.
2.2.1 Distinguished Cell Positions
Operators are keen to know the distribution of throughput and latency KPI figures (e.g.
peak user data rate, cell UL/DL throughput, network delay, etc.) across the cells of a "ref-
erence cluster". However, it is impossible or would be associated with prohibitive high
costs to measure these KPIs in a live network environment in so many cell positions that
a Cumulative Distribution Function (CDF) can be calculated for 5%, 50%, or 95% of the
cell area. For this reason, it is proposed to replace the distribution function by KPI figures
measured on three characteristic positions of the cell (in the sequel, the term "cell" will
be used to denote both omni-directional and directional cells):
a) Very Good radio conditions (line of sight close to cell center within 50m, low path
loss)
b) Medium radio conditions (BTS distance of ~1/2 cell radius, channel with medium
noise conditions, medium path losses)
c) Cell Edge conditions (cell overlapping area, receive power from neighbor cells at
about the same level).
The radio conditions are formulated in terms of Signal to Interference and Noise Ratio
(SINR) and receive power levels. The measured KPI figures in Good, Medium and Cell
Edge positions replace their distribution across the LTE cell and will be considered as
their Peak, Average and Minimum values, respectively. The three variants are given bymean and 95% values calculated from all measurement samples, where the number of
samples is to be derived from the required confidence level.
If no suitable cell positions are available in the reference cluster, the tests can be
executed in the laboratory, where the radio conditions are to be simulated. If automated
test execution is possible, even the calculation of CDF function values at 5%, 50% and
95% distribution is an option.
This shows that often the full set of results can only be produced by combined field and
lab trials. In certain cases, even simulations might become necessary to accomplish
(incomplete) measurement results. For this reason, it is best practise to measure all E2E
field network KPIs in vendor lab before going into field tests with the operator.
The description of test conditions (see Table 2, row "Setup") specifies which test
methods are recommended.
8/13/2019 LTE E2E Field Network Performance - KPI Definitions RL10, Customer Version
LTE E2E Field Network Performance - Definitions ofKey Performance Indicators
Id:0900d805807292c5
2.2.2 Channel Models
Several factors of the radio link have influence on the E2E performance of application,
or LTE network services. Their importance is dependent on the category of the KPI to
be measured. The following parameters are checked for each scenario:
• UE category
• Antenna systems on UE and eNB
• Position and distribution of terminals in the cell, terminal movement
• Load conditions in the measured cell (UL/DL, data volume)
• Receive power level, interference and noise conditions, diversity and fading
• Cell area, or clutter type (urban, sub-urban, rural)
In order to reduce the number of possible combinations, mobile terminal speed and
radio propagation conditions for different environments and distances were combined in
"channel models". The relevant channel models for LTE are described in References 30.
Annex B of [3GPP36.521-1]. The following three channel models are defined: • EPA - Extended Pedestrian A
• EVA - Extended Vehicular A
• ETU - Extended Typical Urban
In addition, 3GPP proposes some combinations of channel models and Doppler fre-
quencies, which are complemented in Table 1 with scenarios for
• stationary UEs (EPA0, EVA0 and ETU0), and
• high speed train (ETU250)
The high speed train scenario with app. 250 km/h uses a carrier frequency of 900 MHz
(former GSM band) since it is assumed that initial LTE deployments will use this fre-
quency in rural areas. If the assumption is not true for a given customer trial, the Dopplerfrequency will change proportionally to the applied carrier frequency (500Hz @ 1.8 GHz,
583 Hz @ 2.1 GHz, 722 Hz @ 2.6 GHz). The current document uses only these models
as reference models.
Model Source Maximum Doppler Frequency
EPA0 0 Hz (stationary UE)
EVA0 0 Hz (stationary UE)
ETU0 0 Hz (stationary UE)
EPA5 3GPP36.521-1, Annex B; see 30. 5Hz
EVA5 3GPP36.521-1, Annex B; see 30. 5Hz
EVA70 3GPP36.521-1, Annex B; see 30. 70Hz
ETU70 3GPP36.521-1, Annex B; see 30. 70Hz
ETU250 250 Hz
(with 900 MHz carrier frequency)
ETU300 3GPP36.521-1, Annex B; see 30. 300Hz
Table 1 Channel Models and Doppler Frequencies
8/13/2019 LTE E2E Field Network Performance - KPI Definitions RL10, Customer Version
LTE E2E Field Network Performance - Definitions of Key Performance Indicators
Id:0900d805807292c5
Channel model conditions can be set in lab trials with simulated radio links precisely, but
they cannot be ensured in field network tests. In field trials, radio conditions with best
suit to the channel model are used to calibrate trial lab results.
2.2.3 Template for the Description of Reference Conditions
Table 2 includes the channel model together with other parameters to summarize the
essential conditions for measurement scenarios. The column "Reference Condition"
gives textual descriptions and possible settings as abstract values only.
The meaning of rows is as follows:
• "Antenna" identifies the number, type and modulation of antennas used on the UE
and on the eNB side. The reference UE configuration comprises two receive
antennas in DL and one antenna in UL. (However, it may not be freely selectable in
reference clusters of operational networks.)
• "The "Application" parameter identifies the test application as a real application, oran equivalent replacement (test script), which allows the collection of information
necessary to calculate / extrapolate the KPI figure.
• The "Setup" parameter identifies the environment, where the KPI can best be mea-
sured. It is not always the reference cluster of the live network. Some conditions, e.g.
SINR are easier to set in the lab.
– The "Reference Cluster" denotes a selected area of the operator´s live network
performance verification tests. In the cluster typical load conditions apply, as
generated by real users -stationary or mobile - and the radio conditions are
determined by cell location (rural, or city area, high, or low buildings, etc.). The
conditions are given and cannot be modified during the trial.
– The "Trial Lab" is either a vendor or operator lab used for executing performancetests, where exact measurement conditions can be ensured. The lab allows the
background traffic mix, load levels, and radio conditions to be precisely set. The
simulation of mobile users of high velocity is also possible.
– "Simulation" results are applied where field and even lab trials are expensive or
cannot be executed in the given time or budget.
• "Concurrent Load" describes the load which is to be generated in addition (i.e. back-
ground traffic) to the traffic produced by active test terminals in the measured cell.
Load conditions in neighbor cells are the operational load in field network clusters,
and are set to medium load in the lab.
• "Drive test "Routes" are defined for mobile terminals in live LTE environments. In
trial labs, terminal movement and hand-over are simulated (by varying the channel
conditions).
• "Backhaul Capacity" defines the necessary throughputs of transport links if their
values deviate from default and are important for test execution, e.g. to prevent a
transport link becoming bottleneck in user data rate, or cell throughput measure-
ments. It also defines other parameters, like Packet Delay (PD), Packet Delay Vari-
ation (PDV), Packet Loss Rate (PLR) and their expected values. These parameters
are observed in field trials, and are set to their maximal allowed values in the lab.
8/13/2019 LTE E2E Field Network Performance - KPI Definitions RL10, Customer Version
LTE E2E Field Network Performance - Definitions of Key Performance Indicators
Id:0900d805807292ed
2.3 Measurement Scenarios
Measurement Scenarios describes the measurement methods and recommended sce-
narios according to KPI categories summarized in Table 3. The first column indicates
the respective QoS terminology defined by the ITU [ITU-T.E800], and referenced by3GPP in References 12. [3GPP32.410] and 17. [3GPP32.450] for LTE related KPIs.
The third and fourth columns indicate the selected terminology and explanation for its
usage in the current paper.
Since the KPI category is part of all KPI definitions (see KPI Definition Template ), it is
easy to lookup the corresponding measurement method to each KPI.
2.3.1 Throughput Measurements
Throughput is the primary metric for the characterization of the LTE radio technology
(Orthogonal Frequency Division Multiple Access (OFDMA) in DL, Single-Carrier Fre-
quency Division Multiple Access (SC-FDMA) in UL). It shows the data rate that can be
provided to users among different radio conditions. The throughput of an LTE cell is
expressed as a function of the number of concurrently active users and the amount of
aggregate traffic that the cell can still support. The throughput the LTE system can offer
depends on many factors:
• Channel environment (e.g. stationary or mobile, speed) and fading conditions.
• Reception conditions impaired by traffic load levels, and by interference between the
cells, in short by the user's SINR.
• Network layout, type of antenna.
• Position of users in the cell (implies e.g. path loss and fading).
ITU QoS
Category
3GPPKPI
Category
Terminology
Used
Meaningfor LTE Field Network
Performance
Accessibility Accessibility Accessibility Service accessibility, e.g. Attach, or
EPS Bearer Setup Request success
rates.
Reliabil ity Retainability Reliability Once established, the reliability of theservice, e.g. VoIP call, or EPS bearer
drop rates.
Integrity Throughput,
Latency
Throughput,
Latency
Promise to deliver the service with a
certain quality, level of conformance.
E.g. cell throughput, or user data rates,
packet delay.
Availability Availability not used Relationship between service up and
down times, e.g. cell, or LTE network
availability
not defined Mobility Mobility Different mobility related metrics, e.g.
hand-over success or drop rates, laten-
cies.
Table 3 KPI Categories
8/13/2019 LTE E2E Field Network Performance - KPI Definitions RL10, Customer Version
LTE E2E Field Network Performance - Definitions ofKey Performance Indicators
Id:0900d805807292ed
measured between UE - eNB (LTE Uu), UE - S/P-GW (E-UTRAN Radio Access
Bearer(E-RAB), EPS) and UE - AS (LTE E2E), respectively. The measurement can
(and should) be executed in both directions.
The Ping application sends successive ICMP Echo Requests • either upon reception of the ICMP Echo Reply to the previous request,
• or after a predefined period of time (in the range of ~100ms to 1s)
This is done to ensure that only one Ping request/response is under way at any given
time. This way, the delay time will have no bias caused by queuing, or scheduling delays
along the transport path. The total number of measurements per cycle should be chosen
so that results have the required statistical confidence level. The recommended number
of measurements per IP host is ≥ 100.
If the measurement is executed over the radio link (LTE Uu), the first IP packet might
suffer additional delay caused by the dynamic allocation of radio resources to non-real-
time (non-GBR) bearers. For this reason, the round trip delay of the first packet should
not be considered in statistic calculations. On the other hand, the first ping measurement
can be used to determine UE state transition delays if the UE starts from non-registered
or idle states (from non-registered to registered, or from idle to connected states). The
periodicity of Pings needs to be selected so that the once allocated radio bearer is kept
over the whole duration of the measurement.
The test is to be done for stationary and mobile end users. The measurement with sta-
tionary UE should be performed under average radio link conditions with normal SINR.
For moving UEs, two different positions are to be selected, one in average radio link con-
ditions and normal SINR, and another in the cell overlapping area. It is important to
make sure that handovers between cells are avoided. The reference conditions should
be monitored during the test using UE performance measurements; also, relevant trace
data can be collected from the network.
In lab trials, the measurements need to be executed with and w/o concurrent load.
From the measurement results (samples) mean figures, standard deviation and 95%
delays need to be calculated.
2.3.2.2 Signaling and Media Delay Measurements
If the bandwidths of UL and DL bearers in RTT measurements are identical, the bench-
mark value of UL and DL latencies can be given as 1/2 RTT.
However, latency KPIs of signaling procedures on the network or application level (e.g.
Radio Access Bearer (RAB) service setup, or VoIP call setup flows) are often complexand include the exchange of more than one messages of different sizes. In addition, the
link capacities used in UL and DL are often different, e.g. in case of the application
services data download, gaming or video streaming.
In these cases, the RTT or 1/2 RTT figures are not sufficient to characterize the latencies
incurred by the mobile access network. Here, the time synchronization of network
elements is the central problem, since the start and stop triggers of the measurement lie
on different Network Elements (NE) and interfaces.
In case of measuring latencies over the radio link, the time needed to attach the mobile
to the network and to setup the necessary radio resources is a significant component.
In these cases, the measurement should be made with different initial states of the
mobile, e.g. power off, idle, connected and active states. LTE State Models lists the dif-
8/13/2019 LTE E2E Field Network Performance - KPI Definitions RL10, Customer Version
LTE E2E Field Network Performance - Definitions ofKey Performance Indicators
Id:0900d805807292ed
2.3.4.2 Service Drop Rates
Attach, Service Request, Register, etc. are examples of service requests on different
service levels; service requests establish a relationship between the UE and the network
or application. The relationship is normally terminated with a "Release Request" by theUE. In case of network errors (e.g. handover failures), or insufficient radio resources the
relationship can be terminated unexpectedly. The ratio of such "abnormal" terminations
to the number of all established relationships is given by "drop rate" KPIs.
For certain services, like bearer creation or VoIP call, the service drop rate is also related
to the duration of the session. This is motivated by the fact that the possibility of network
errors, e.g. hand-over failures is higher if the service duration is longer. In such cases,
the service usage time should also be measured, and the drop ratio expressed in [1/s]
units instead of [%]. The current version of this document does not consider session time
for service drop rates. It will be added in the next update of this document.
In order to measure the ratio, the given service request has to be executed in a suffi-
ciently large number of times to assume statistical significance. E.g. each test scenario
should be executed ≥ 100 times. After the relationship has been established, common
operations of the service are to be executed for a typical period of service usage time.
The LTE traffic model can deliver input for this.
Due to the complexity of the test setup and the long time needed to execute the tests,
the measurements are better executed in a lab environment with simulated air links and
automated service execution.
The measurements are to be executed under the following varying conditions:
• Mobile terminal in stationary and mobile use
• The mobile use includes drive routes among good and medium reception conditions,
as well as routes that cross cell overlapping areas. In the latter case, a handover isexecuted. If executed in the trial lab, these conditions are set in simulated radio links,
i.e. the "drive route" is simulated.
• Unloaded and loaded cell conditions. Concurrent load is to be generated up to full
(or target) load level of the cell.
8/13/2019 LTE E2E Field Network Performance - KPI Definitions RL10, Customer Version
LTE E2E Field Network Performance - Definitions ofKey Performance Indicators
Id:0900d805807292ed
In order to measure handover KPIs, the logging of mobility events is necessary in mobile
terminals. A successful handover is marked by an entry in the log, which carries the
identifier of the new cell (destination).
The HO KPIs are measured under the following variations of conditions (Table 10): • Unloaded versus loaded cell conditions for Intra RAT, ideal conditions in target cell
for Inter Radio Access Technology (RAT) (GSM, UMTS) hand-over scenarios.
• Mobile terminals with vehicular speed.
• Drive routes from good reception condition (in the originating cell) over hand-over
area to a good position on the destination cell.
The drive test has to be repeated at least 20 times to get the required level of statistical
confidence. All reasons for handover failures (no resources in target cell, service not
supported, etc.) should be excluded.
Parameter Reference Condition
Terminal
UE Type Smart phone, laptop with LTE card
UE Category UE category tbd. according to [3GPP36.306]; see References
21.
Antenna One antenna: e.g. SISO
Position Good/medium in originating cell directly across hand-over area to
good/medium position in new cell.
Distribution n.a.
Network Environment
Channel Model EVA0, EVA70: stationary, mobile w/ ~50 km/h
Backhaul Capacity n.a.
QoS Configuration n.a.
Concurrent Load Unloaded and loaded destination cell in UL/DL.
Application UDP data stream in DL with hand-offs acc. to reference traffic
model.
Traces MT, Source and Target eNBs, mobility events
Setup Reference Cluster, Trial Lab
Route In reference cluster, drive test route description.
Table 10 Reference Conditions for Handover Measurements
8/13/2019 LTE E2E Field Network Performance - KPI Definitions RL10, Customer Version
LTE E2E Field Network Performance - Definitions ofKey Performance Indicators
Id:0900d805807295be
• WS for post-processing of traces
• Operations, Administrations and Maintenance Performance Management (OAM
PM) for monitoring counters, displaying of statistics
2.4.2 Trial Lab
A description of the Trial Lab should be given with the performance verification test
specifications.
2.4.3 Measurement Data Collection
The following ways of performance data collection are possible in KPI measurements:
• Collection of traces from terminals, network elements and network interfaces
• Collection and evaluation of counters from network elements by OAM PM
The preferred way of collecting performance data in a live network is via PM counters.It does not add additional load to the network and is a permanent source of information
that can be monitored all the times. However, PM counters do not exist for all categories
and for all individual KPIs. Typically, no counters are defined, or can be implemented for
latency KPIs. For this reason, terminal, server and network traces are also collected.
Since tracing puts additional processing load on these elements, traces are activated
only on operator demand and for the time of test execution. On the other hand, traces
include very detailed information on call level, which allows performance analyses of
individual mobiles and operations. With traces, it is possible to go further in monitoring
and optimization operations. For more details on LTE MMS and E-UTRAN traces, see
References 13. [3GPP32.421] and 14. [3GPP32.423].
2.4.3.1 Subscriber and Equipment Traces
Subscriber and UE Traces should be activated on the UE, and on the network elements
MME and S/P-GW of the Reference Cluster, as required by the scenario description
(see 2.2 Reference Conditions) of the given KPI measurement.
Collected measurement data is transferred to an external server for post-processing and
for evaluation. This data is necessary to check if the test run was executed w/o failure
(e.g. dropped call analysis) and if the measured figure can be considered for statistic cal-
culations and statements. Moreover, it helps to verify if the reference conditions were
met (e.g. RF coverage and capacity) during the whole measurement. Traces deliver
additional information for tuning and optimizing the Reference Cluster itself.
The capability to log data on any interface at call level for a specific user (e.g. Interna-
tional Mobile Subscriber Identity (IMSI)) or mobile type (e.g. International Mobile Equip-
ment Identity (IMEI)), or service initiated by a user makes it possible to get information
which cannot be deduced from Performance Measurements such as the end-user per-
ception of Quality of Service (QoS) during a call (e.g. requested QoS vs. provided QoS),
and correlation between protocol messages and RF measurements. Moreover, as
opposed to OAM Performance Measurements, which provide values aggregated on an
observation period (i.e. interval statistics), Subscriber and UE traces give instantaneous
values for a specific event (e.g. call, location update, etc.).
8/13/2019 LTE E2E Field Network Performance - KPI Definitions RL10, Customer Version
LTE E2E Field Network Performance - Definitions of Key Performance Indicators
Id:0900d805807297e9
3.3 Application Services
3.3.1 Data Download / Upload (FTP) Service KPIs
Download of reference files with different sizes from the Internet, e.g. SW images, pic-
tures, or video, as well as upload of the same files by FTP. For FTP protocol details, see
References 33. [RFC979].
3.3.1.1 (FTP) Service Accessibility Ratio
(FTP) Service Accessibility Ratio
KPI Category, Object, Type, Unit M Category: Accessibility, Object: AS (UE, FTP server),
Type: Ratio, Unit: [%]
Definition M The service accessibility ratio denotes the probability that the user canestablish the necessary bearer (EPS) and access the FTP service suc-
cessfully (see References 39. [ETSI102.250-2]).
Service access covers starting the FTP client on the UE, setting up
mobile access and creating a TCP connection to the FTP server.
Measurement method O Accessibility measurement, see 2.3.3 Service Accessibility Measure-
ments. Field and lab trial.
Assumptions, pre-conditions O Conditions for accessibility measurements see 2.3.3 Service Accessi-
bility Measurements
Formula (logical) M
Related KPIs O (FTP) Completed Session Ratio
Related PM counters O No PM counters exist for this KPI
Message flow, trigger points O Trigger points:
• Start: ftp get / put command issued on the UE
• Stop: first data byte sent / received by the UE, or ftp indicates
LTE E2E Field Network Performance - Definitions of Key Performance Indicators
Id:0900d805807297e9
be issued, a TCP connection to the FTP server is created and the data transfer phase
begins.
Trigger events of the Service Access Time KPI cannot be seen on the terminal UI. In
order to find them, it is necessary to analyze the message flows on the R interface level.The Session (and the IP Service Access phase, see Figure 4) begins with setting up the
TCP connection to the FTP server and terminates with the transmission (send, or
receive) of the last IP packet with file content payload. The phases Service Access and
IP Service Access end, and the Data Transfer phase begins with the transmission (send,
or receive) of the first IP packet with file content payload.
Figure 4 KPIs for Packet Switched Data Services
8/13/2019 LTE E2E Field Network Performance - KPI Definitions RL10, Customer Version
LTE E2E Field Network Performance - Definitions of Key Performance Indicators
Id:0900d805807297e9
It is to be noted that the measurement of the User Data Rate (FTP) KPI (named after
[ETSI102.250-2]; see References 39.) is executed as a single user measurement
without concurrent load in the cell, since the goal is to find the maximum available FTP
data rate for one user.
Measurement method O Throughput measurement, see 2.3.1.2 Peak User Data Rate. Single user
measurement both in field and lab trials.
In order to eliminate the impact of service startup times, the measurement
begins with the transmission of first IP packets with file content payload.Since the data rates can vary strongly from session to session, a sufficiently
large number of measurement executions is needed (>10) to get statistical
confidence.
The size of the large reference file should be selected such that the data
transfer phase does not take less then thirty seconds. Optionally, a small ref-
erence file can be measured in addition, but the number of executions should
be the same.
The measured User Data Rates should be displayed in diagrams as a
function of time to show the impact of FTP slow start.
Assumptions, pre-conditions O Conditions for capacity measurements see 2.3.1.2 Peak User Data Rate.
The FTP server will be connected directly to the SGi interface. Additionaldelays or bottlenecks between the S/P-GW and the server should be
excluded.
The initial status of UE before starting the ftp application:
EMM-REGISTERED, ECM-CONNECTED.
The FTP application will be executed in binary mode, and the file to be trans-
ferred will also be binary.
Formula (logical) M
Related KPIs O (FTP) Service Access Time, (FTP) Session Time
Related PM counters O No PM counters exist for this KPI.
Message flow, trigger points O The average throughput is measured from the opening of the data connec-
tion to the end of the successful transfer of the content (file, e-mail or web
page).
The trigger points for the Data Transfer phase can be seen in Figure 4. For
more details see References 39. [ETSI102.250-2], chapters 4.6.1.7 and
Related KPIs O a) Attach Time, Service Request (ESP) Success Rate.
b) E-RAB Accessibility" KPI in [3GPP32.450], see References17. chapter 6.1.1. Since an
SRB and S1 connection, as well as initial EPS bearer(s) have to be created during the
Attach procedure (see Figure 5), its success rate is dependent of the success rates ofthose component bearers. The success rate of initial EPS bearers is given by the follow-
ing expression (for “additional” EPS bearers, refer to the 3.4.1.5 Service Request (EPS)
Success Rate):
c) See "Attach Failure Ratio" KPI in References39. [ETSI102.250-2], Chapter 4.6.3.1
defined as the inverse probability of successful network attachment.
LTE E2E Field Network Performance - Definitions of Key Performance Indicators
Id:0900d8058072981f
3.4.1.5 Service Request (EPS) Success Rate
Service Request (EPS) Success Rate, Control Plane
KPI Category,Object,
Type, Unit
M Category: Accessibility, Object: EPS (UE R, eNB Uu),
Type: Ratio, Unit: [%]
Definition M This KPI is defined as the ratio between successfully established EPS bearers
compared to the overall number of EPS bearer establishment attempts. It corre-
sponds to the probability that a user or the LTE network can establish an EPS bearer
at any moment in time.
Requests that are terminated by timer expiry (due to the unaccessibility of some LTE
resource) are considered as unsuccessful attempts. Authentication errors (requests
rejected by the MME) are included in the total number of failures.
Only the first RRC CONNECTION REQUEST is to be considered, since counting
retries of the same message would increase the overall number of bearer establish-
ment attempts, and thus reduce the success ratio.
Measurement
method
O Accessibility measurement, see 2.3.3 Service Accessibility Measurements. Field
and lab trial. Stationary users only. For calculating the success ratio, the same series
of "Service Request (EPS) Time" measurements can be used.
Assumptions,
pre-conditions
O Conditions for accessibility measurements see 2.3.3 Service Accessibility Measure-
ments.
See additional conditions in 3.4.1.4 Service Request (EPS) Time, Network Initiated.
Formula (logical) M
Message flow,trigger points
O The message flow diagram is shown in Figure 6 and Figure 7. The trigger points forUE initiated EPS bearer setup success ratio on the R interface are:
LTE E2E Field Network Performance - Definitions ofKey Performance Indicators
Id:0900d8058072981f
(~minutes). Of equal interest are peak values observed for a short time period only
(~seconds).
Please note that data rate KPIs have also been defined for application services, e.g.
User Data Rate KPI of the Data Download / Upload service, but these do not character-
ize the capability of the LTE network alone, but also depend on the application service
and its protocols.
3.4.1.13 (LTE) Packet Loss Rate
(LTE) Packet Loss Rate UL / DL, User Plane
KPI Category, Object, Type, Unit M Category: Availability, Object: LTE (UE R interface, IP peer entity)
Type: Ratio, Min (CDF 5%), Mean (CDF 50%) Max (CDF 95%) values
Unit: [%]
Definition M This is the ratio between the numbers of lost or corrupted IP packets,and of all IP packets sent. Corrupted IP packets are those that contain
bit errors in their headers or in their payload.Packets with "residual", i.e.
undetected errors are not counted as lost.
Measurement method O Availability measurement, see 2.3.3 Service Accessibility Measure-
ments.
The EPS bearer is to be configured such that corrupted IP packets are
not delivered to the application.
Measured in loaded and unloaded network, under different radio condi-
tions. Stationary and mobile users.
The packet loss rate should be measured in test scenarios, where han-
dovers occur. The number of handovers should be oriented toward theNSN reference traffic model, or, in operator trials, toward the operator´s
traffic model.
Assumptions, pre-conditions O Conditions for availability measurements, see 2.3.3 Service Accessibil-
ity Measurements.
Measured in loaded and unloaded network, under different radio condi-
tions. Stationary and mobile user including HO scenarios.
The ratio can be calculated on a series of User Data Rate (see
2.3.1.2 Peak User Data Rate) measurements.
Formula (logical) M
Message flow, trigger points O UL / DL:
• Total: packages sent at UE R if / IP peer entity
• Lost: Total minus packets received at IP peer entity / UE R if.
Related KPIs O a) (RB) Initial Radio Block Error Rate (BLER)
b) No PLR KPI (LTE level) is defined in 3GPP and ETSI documents.
Related PM counters O Measurements of packet loss rate on LTE level (R <-> Gi) are neither