2G Test Methodology and Definition V1.0

8/13/2019 2G Test Methodology and Definition V1.0

1/37

2G Test Methodology and Definition

I 2014ZTE CORPORATION. All rights reserved. ZTE Confidential Proprietary

2G Test Methodology andDefinition


2/37

2G Test Methodology and Definiton


Version Date Author Approved By Remarks

V1.00 2011-04-27 ZTE Not open to the Third Party

2014 ZTE Corporation. All rights reserved.

ZTE CONFIDENTIAL:This document contains proprietary information of ZTE and is not to bedisclosed or used without the prior written permission of ZTE.

Due to update and improvement of ZTE products and technologies, information in this documentis subjected to change without notice.


3/37


III 2014ZTE CORPORATION. All rights reserved. ZTE Confidential Proprietary

TABLE OF CONTENTS

2G RAN Radio Parameter Mapping Proposal .............................................................................. II

TABLE OF CONTENTS .................................................................................................................. III

FIGURES V

TABLES V

1 Introduction ................................................................................................................ 1

2 DiGi Field Test and KPI Requirements..................................................................... 12.1 KPI Requirements ........................................................................................................ 12.2 Test ItemsRequirements .............................................................................................. 2

3 2G Field Test Definition for SSV, Cluster and Area ................................................ 33.1 Test methodology ........................................................................................................ 33.2 Main Test Items Description ........................................................................................ 63.2.1 RF Scanner Test .......................................................................................................... 63.2.2 Short Call ..................................................................................................................... 73.2.3 Long Call ...................................................................................................................... 73.2.4 MOS ............................................................................................................................. 83.2.5 Attach and PDP activation & FTP UL/DL .................................................................... 83.3 Acceptance Requirements for the Drive Test .............................................................. 93.3.1 Single site performance measurement ........................................................................ 93.3.2 Cluster network wide performance measurement ....................................................... 93.3.3 Area network PAC measurement .............................................................................. 11

3.3.4 Area FAC requirements ............................................................................................. 113.3.5 Ericsson and Siemens Reference ............................................................................. 11

4 Field Test Equipment and Specifications .............................................................. 124.1 MS/UE ........................................................................................................................ 124.2 Scanners .................................................................................................................... 124.3 Positioning Systems / GPS ........................................................................................ 134.4 Data Collection Tools (SW and HW) ......................................................................... 144.4.1 PC Hardware and Software Requirements ............................................................... 144.4.2 OS Requirements ...................................................................................................... 144.4.3 Introduction to TEMS Investigation 11.0 Data Collection .......................................... 144.4.4 Applications................................................................................................................ 154.5 Internal / External Antennas ...................................................................................... 16

4.6 Network Servers ........................................................................................................ 164.7 Laptop ........................................................................................................................ 16

5 Test Kit Setup and Methods .................................................................................... 175.1 Laptop/Data Collection Gear Setup ........................................................................... 175.2 Trace and Data Test .................................................................................................. 175.2.1 Scanner Test ............................................................................................................. 185.2.2 Multiple Test............................................................................................................... 185.3 Tools Kit Configuration .............................................................................................. 195.3.1 TEMS Configuration Flow .......................................................................................... 195.3.2 TEMS Configuration Procedures ............................................................................... 205.4 Voice Configuration Parameter ................................................................................. 205.5 Data Configuration Parameter ................................................................................... 20


4/37


5.6 Measurement and PS/CS Configuration Specifications ............................................ 215.6.1 Voice call.................................................................................................................... 215.6.2 Short call .................................................................................................................... 235.6.3 Long call..................................................................................................................... 24

5.6.4 FTP test ..................................................................................................................... 245.6.5 Scanning .................................................................................................................... 25

6 Acceptance Reports Contents ................................................................................ 286.1 SSV/SSOA ................................................................................................................. 286.1.1 Single Site Verification (SSV) .................................................................................... 286.1.2 Single Site Optimisation Acceptance (SSOA) ........................................................... 286.2 Cluster Acceptance Report ........................................................................................ 286.3 Area Network Acceptance Report ............................................................................. 30


5/37


V 2014ZTE CORPORATION. All rights reserved. ZTE Confidential Proprietary

FIGURES

Figure 1 Trace and Data Test Connection ................................................................................. 17

Figure 2 Scanner Test ................................................................................................................ 18

Figure 3 MultipleTest .................................................................................................................. 18

Figure 4 TEMS Configuration Flow ............................................................................................ 19

TABLES

Table 1 Test KPI Requirements .................................................................................................. 1

Table 2 Test Item Requirements................................................................................................. 2

Table 3 2G KPI Measurement Methodology ............................................................................... 3


6/37


7/37

Copyright DiGi 2011

1 Introduction

Key performance indicators (KPI) are being used to measure and evaluate theperformance of the networks. The quality of the network is ultimately determined by thesatisfaction of the users of the network. Field tests such as drive-test gives the 'feel' ofthe designed network as it is experienced in the field. The testing process starts withselection of the 'live' region of the network where the tests need to be performed, andthe drive-testing path.

This document describes the 2G RAN field test KPIs and requirements for future ZTEsolution based DiGi network. As we describe field test KPIs, the test definitions for 2GRAN network implementation, the testing equipments, its set-up and specifications, thecollection and processing will be introduced.

2 Field Test and KPI Requirements

Because contract files are the criterion of the final acceptance, in this chapter, wedescribe the requirements of field test KPI in contract.

2.1 KPI Requirements

Based on the final acceptance requirement KPI in the contract, the required field testKPIs for the cluster, Area acceptance are the same. But they have different workingtargets for ZTE to achieve in different project implementation stages.

Table 1 Test KPI Requirements

No. KPI Parameter

Voice Network AccessibilityGT_01 Call Set up Success Rate

GT_02 Location Update Success Rate

Voice Retainability

GT_03 TCH Drop Call Rate

GT_04 Handover Success Rate

Coverage & QualityGT_05 RxLevSub (dBm)

GT_06 Call Setup Time

GT_07 Voice Quality (MOS)

GT_08 % RXQUAL between 0-4

Packet DataGT_09 Attach success rate

GT_10 PDP context activation success rate

GT_11 Attach Time (setup time)


8/37


GT_12 PDP context activation time

GT_13 EDGE DL throughput

GT_14 TBF Drop Rate

GT_15 EDGE UL throughput

2.2 Test ItemsRequirements

In this section, we will define the test items according to the KPI items in contract For

example, the voice CSSRCall Setup Success Rate will be tested by Short Call.

The following table gives the test items according to the field test KPI requirements.

Table 2 Test Item Requirements

No. KPI Parameter Related Test ItemVoice Network AccessibilityGT_01 Call Set up Success Rate Short Call

GT_02 Location Update Success Rate Short Call

Voice RetainabilityGT_03 TCH Drop Call Rate Long Call

GT_04 Handover Success Rate Long Call

Coverage & QualityGT_05 RxLevSub (dBm) Long Call

GT_06 Call Setup Time Short CallGT_07 Voice Quality (MOS) MOSGT_08 % RXQUAL between 0-4 Long Call

Packet DataGT_09 Attach success rate FTP UL/DL

GT_10 PDP context activation success rate FTP UL/DL

GT_11 Attach Time (setup time) FTP UL/DL

GT_12 PDP context activation time FTP UL/DL

GT_13 EDGE DL throughput FTP DL

GT_14 TBF Drop Rate FTP UL/DL

GT_15 EDGE UL throughput FTP UL

For cluster swap benchmarking, in order to evaluate the Coverage Equivalencethe

RF scanner test is required for cluster benchmarking test before swap and after swap.

Category KPI Item Test description

Coverage Coverage Equivalence RF Scanner Test

The summary of the basic 2G KPI measurement methodology is contained as below


9/37

Copyright DiGi 2011

Table 3 2G KPI Measurement Methodology

End-User Services

Measurement Method

Drive testfor outdoor

Walk test forindoor

Stationary testOSS StatisticsMeasurement

Voice YES YES YES YES

Packet-Service YES YES YES YES

During SSV, the chief measure for single site is the drive test. SSV check list and drivetest will performed during SSV/SSOA. For cluster and Area acceptance, the drive testKPI and OSS statistics KPI will both considered.

3 Field Test Definition for SSV, Cluster andArea

3.1 Test Methodology

The test methodology for each test case defined in this Section will be used for ClusterAcceptance, Area Acceptance, and Network Performance Stability Acceptance inaccordance to Annex(Acceptance Procedures).

The table below is a list of different test methods from the contract

Methodreference MOC MTC Method desctription RemarksT01 Voice Call

GSM

UE1 SUE1 Originate call repeatedly with holding

time of 120s and waiting time of 5sAlternating MOC/MTC.

T02 Voice CallDualMode

UE2 SUE2 Originate call at the start of the testand held on for the entire test duration

Alternating MOC/MTC.

T03 VideoShort Call

UE3 UE9 Originate call repeatedly with holdingtime of 60s and waiting time of 10s

If abnormal call release, a new call ofholding time of 60s is setupimmediately after waiting time of 10s

T04 Video

Long Call

UE4 SUE3 Originate call at the start of the test

and held on for the entire test durationIf abnormal call release occurs, a newcall will be set up immediately

T05 UL PS-64Short Call

UE5 FTPserver

Set up a PDP context andimmediately follow by FTP session toupload a 450kBytes file

The procedure is repeated uponcompletion of upload after a waitingtime of 10s

For stationary test,the test is repeatedfor 5 timesFor in building test,the test isconducted onceper every static testpoint.

T06 DL PS-384 UE6 FTP Set up a PDP context and


10/37


Methodreference MOC MTC Method desctription Remarks

Long Call server immediately follow by FTP session todownload a 250MBytes file

The procedure is repeated uponcompletion of download after a waitingtime of 5s

T07 HSUPACall

UE7 FTPserver

Set up a PDP context andimmediately follow by FTP session toupload a 5MBytes file

The procedure is repeated uponcompletion of upload after a waitingtime of 10s

For stationary test,the test is repeatedfor 5 timesFor in building test,the test isconducted onceper every static testpoint.

T08 HSDPA 5

codes 16QAM LongCall

UE8 FTP

server

Set up a PDP context and

immediately follow by FTP session todownload a 650MBytes file


T09 StaticVoice Call

UE1 UE2 Originate call with holding time of 60sand waiting time of 10s

Repeat for 5 times

T10 StaticVideo Call

UE1 UE2 Originate call with holding time of 60sand waiting time of 10s

Repeat for 5 times

T11 DL PS-384Short Call

UE1 FTPserver

Set up a PDP context andimmediately follow by FTP session todownload a 2500kBytes file

Repeat for 5 times


T12 DL PS-128Call

UE1 FTPserver


Repeat for 5 times


T13 DL PS-64Call

UE1 FTPserver


Repeat for 5 times


T14 HSDPA 5codes 16

UE3 FTPserver

Set up a PDP context andimmediately follow by FTP session to


11/37

Copyright DiGi 2011

Methodreference MOC MTC Method desctription Remarks

QAMShort Call

download a 25MBytes file

Repeat for 5 times


T15 HSDPA 10codes 16QAM Call

UE3 FTPserver

Set up a PDP context andimmediately follow by FTP session todownload a 50MBytes file

Repeat for 5 times


T16 HSDPA 15

codes 16QAMShort Call

UE3 FTP

server

Set up a PDP context and

immediately follow by FTP session todownload a 100MBytes file

Repeat for 5 times


T21 Ping TestR99

UE1 FTPserver

Perform a ping with 100 byte packetsize to FTP server

Repeat 5 times

T22 Ping TestHSDPA

UE3 FTPserver

Perform a ping with 100 byte packetsize to FTP server

Repeat 5 timesT23 Voice with

PDPcontext

UE1 UE2 Setup PDP context and immediatelyoriginate a call with holding time of60s

Repeat for 5 times

Based on the above list, the 2G test methodology used for the drive test is suggested asfollowing

Method reference MOC MTC Method desctriptionT01 Voice Short Call

GSMUE1 UE2 Originate call repeatedly with holding time of

60s and waiting time of 10s

If abnormal call release, a new call of holdingtime of 60s is setup immediately after waitingtime of 10s

T02 Voice Long CallGSM

UE3 SUE1(orcall thespspecifictestnumberprovidedby DiGi)

Originate call at the start of the test and heldon for the entire test duration

If abnormal call release occurs, a new callwill be set up immediately after waiting timeof 10s


12/37


Method reference MOC MTC Method desctriptionT03 FTP UL UE4 FTP

serverAttach and set up a PDP context andimmediately follow by FTP session to uploada 450kBytes file

The procedure is repeated upon completionof upload after a waiting time of 10s

T04 FTP DL UE5 FTPserver

Attach and set up a PDP context andimmediately follow by FTP session todownload a 450kBytes file

Repeat for 5 times

The procedure is repeated upon completionof download after a waiting time of 5s

T05 MOS UE6 SUE2 Originate call at the start of the test and heldon for the entire test duration

If abnormal call release occurs, a new callwill be set up immediately after waiting timeof 10s

3.2 Main Test Items Description

3.2.1 RF Scanner Test

Test Description

RF Scanner Test is used for measuring the coverage of particular clusters/regions.Through RF scanner test, we can get the Rxlev comparison between before and afterswap in cluster, and to keep the same coverage after sites cutover.

For coverage comparison, the RF scanner will be chosen to survey the coverage ofcluster regions. In Cluster coverage benchmarking, we will use one scanner to scanboth 900M and 1800M frequencies with same test route before and after swap.

Before frequency scan, connect the RF scanner to the notebook; choose the correctchannel frequencies of DiGi used to be surveyed. It s best to scan are all BCCHs usedin the cluster.

For outdoor cluster benchmarking test, DT routes should include all the available roadsof test region, including border. Walk-test will be chosen to survey the indoor coveragewhen necessary.

Output KPIs

Before Swap and After Swap mean Rxlev and distribution Plot

Before Swap and After Swap Rxlev CDF and PDF


13/37

Copyright DiGi 2011

3.2.2 Short Call

Test Description

Short Call test is used for evaluating network access performance. Call details such asthe call setup success rate, call setup time and so on will be measured by short call.Network access related problems can be found through short call test.

For outdoor short call test, spot-test is selected in SSV. DT is chosen for Clusterverification.

For indoor short call test, spot-test is also selected in SSV. For cluster, if those areascannot drive-test, walk-test will be used.

For short call during SSV, we will lock to the frequency of the particular cell under test.

Short call will be performed fully follow the SSV checklist, and repeat throughout theentire drive test in cluster test.

For SSV spot-test, the principle of spot selection is that 1 point in each sector, for OMNIsite, 1 point is selected.

Output KPIs

Call Setup Success Rate

Call Setup Time

Location update Success Rate

3.2.3 Long Call

Test Description

Long Call test is used for testing the consistency during a call. Problems such ashandover and the voice quality related can be discovered by long call testing.

For outdoor test, DT is preferred. For those areas cannot drive-test, like indoor case,walk-test will be selected.

Test equipment is located in the car.MS will be connected with a test notebook, GPS isalso connected. The MO MS will dial the MT phone number for long call test. There isno limitation on the call duration.

Long call test will be realized automatically on the test equipment. If drop-call happens,it will make a new call with 10s delays before the redial.

For outdoor test, DT routes should include all the available roads of test region,including the border. Walk-test will be chosen to survey the whole indoor area.

Output KPIs

Rxlev and Rxqual Plots


14/37


Handover Success Rate

Call Drop Rate

Rxqual 0~4 share

3.2.4 MOS

Test Description

MOS test is used for testing the overall performance in a particular region or network.

MOS Test will be executed by 2 mobile phones. The MOS with Mobile-to-Mobile testwith have the calling mobile located in the moving vehicle, whereas the called mobilewill be located in a fixed location of good RF environment. MO MS and MT MS will

connect with an audio box and also GPS is connected with the test notebook. The MOMS will make long call to MT MS. The call time is no limitation and test will be realizedautomatically on the test equipment. If drop-call happens, the MS will redialautomatically. DT is chosen to carry out the MOS test. Test routes should include all theavailable roads of test region and across the cluster border.

Output KPIs

Mean value of MOS

3.2.5 Attach and PDP activation & FTP UL/DL

Test Description

Attach and PDP activation test is used to analyze the access performance of PS service.

File Transfer Protocol (FTP) is a technology used to connect two network elementstogether to transfer files. FTP test is used to analyze the ftp upload and downloadperformance of PS service.

The test will be executed by sequence of Attach, PDP Activation, FTP UL/DL andDetach (Suggested there will be 1s interval between 2 round of test).

The FTP UL/DL test will perform separately due to the throughput will affect both mobilephones if they are doing the test simultaneously.

Output KPIs

Attach Success Rate and attach time

PDP Activation Success Rate and time

FTP DL/UL Average Throughput(EDGE)

Throughput DL/UL EGDE Plot

TBF Drop Rate


15/37

Copyright DiGi 2011

3.3 Acceptance Requirements for the Drive Test

3.3.1 Single Site Performance Measurement

Single Site Verification (SSV)

The single site verification inclusive drive test for outdoor site(s) or walk test for in-building site(s). The test cases below are generally described at a high level whichincludes:

Site Data Verification

Cell Data Verification

Check Points Test

Accessibility Test Which Includes of Voice Call

Data Accessibility

Throughput Test

Handover Test

Coverage Plot (Drive Test or Walk Test)

Quality Plot (Drive Test or Walk Test)

Single Site Optimization Acceptance (SSOA)SSOA is required when a new site is being brought on air to an accepted cluster. Thissite is required to be optimised and accepted as part of Provisional and/or Final

Acceptance.

Drive test/ Walk test and OSS statistic KPIs are required to be measured for the newsite and its 1st tier neighbour.

Drive test/ Walk test route will be defined by DiGi and this will be discussed andmutually agreed between DiGi and ZTE.

The SSV check result and SSOA result will contain in SSV report after the single sitepassed all the required items.

3.3.2 Cluster Network Wide Performance Measurement

If a swap or new rollout is done Cluster Acceptance is required for ProvisionalAcceptance.

Cluster Acceptance Test Requirements

The drive test measurement shall be conducted continuously:


16/37


Speech traffic: normal working day from 9am to 7pm

Normal speed regulation for the area shall be followed

The measurement shall be done using the tools specified in Section 5

The measurement shall contain and get all the samples per cell.

Antenna of drive test tools has to be inside car (use of external antenna out sidecaris not allowed).

Cluster Size and Measurement Routes

A cluster typically consists of 10 to 15 sites.

DiGi will select the sites and drive test measurement routes. This will be discussed

and mutually agreed between the Supplier and Purchaser.

The site clusters and routes shall be documented with electronic maps

The drive test routes must include

Serving areas for all cells in the cluster

Handover between cells

Most highways, major roads, secondary roads and VIP area will be included

Cluster Acceptance Procedures

Before a swap, a benchmarking needs to be done for targeted cell clusters. Thebenchmark KPIs will be based on drive test result. The purpose of cluster acceptance isthe quality and reliability after swap shall be at least on the level it was before swap.

Cluster Acceptance Test Cases

The table below summarizes the tests to be done for Cluster Acceptance from contract.

No Test CaseMethodology Referencein Section 7

Cluster Drive Test

1 Voice CallGSM T01 Yes

2 Voice CallDual Mode T02 Yes

3 Video Short Call T03 Yes

4 Video Long Call T04 Yes

5 UL PS-64 Short Call T05 Yes

6 DL PS-384 Long Call T06 Yes

7 HSUPA Call T07 Yes

8 HSDPA Long Call T08 Yes

9 Voice Walk-In test T17 No


17/37

Copyright DiGi 2011

10 Voice Walk-Out Test T18 No

11 Voice Drive-In Test T19 No

12 Voice Drive-Out Test T20 No

Based on the above, the 2G drive test will perform as following

No Test CaseCluster DriveTest

1 Video Short CallGSM Yes

2 Video Long CallGSM Yes

3 FTP UL Yes

4 FTP DL Yes

5 MOS Yes

6 RF Scanner Test Yes

3.3.3 Area Network PAC Measurement

The drive test for the area acceptance will perform mainly for the cross boarder ofdifferent cluster after the cluster in one area were all passed acceptance. The drive testroutes will be planned to allow handover between cells. Most highways, major roads,secondary roads and VIP area will be included.

The Area Network will consist of the cluster(s) that have been presented to DiGi forProvisional Acceptance. The area size can be defined same as the pilot network which

follow the Scope of the Contract DiGi will propose the borders, measurement drive testroutes (planned and/or randomly routes/optimization activities) to ZTE.

The KPI for the Area acceptance will focus on the overall performance of particularregion / area; most of the region/area cell KPI should meet the KPI requirement incontract.

3.3.4 Area FAC requirements

To be defined by DiGi and ZTE in near future.

3.3.5 Ericsson and Siemens Reference

The same testing methods and items were used to collect Ericsson and Siemens clusterand network performance before swap.


18/37


4 Field Test Equipment and Specifications

4.1 MS/UE Handsets

The mobiles are used for data collection during the field test.

Handset Model Support Test Software

Sony Ericsson W995/Z750TEMS Investigation 11.0 DataCollection

The above mobile sets are the exact model that used for field test in this project.

Specification

Size 105.0 x 47.0 x 22.0 mm

Weight 115.0 g

Networks UMTS 2100

GSM/GPRS 900/1800/1900

Connectivity Bluetooth technology

Infrared port

Modem

Synchronization PC

USB mass storage

USB support

For more detail about the mobile phone, please see the user manual of Sony EricssonW995/Z750, or go tohttp://www.sonyericsson.com for more information.

4.2 Scanners

Scanners are used for scanning the frequency distribution when doing coveragecomparison in GSM testing. BCCH and BSIC decoding are from the scanner test.

Scanner Model Support Test SoftwarePCTEL EX WCDMA/GSM

Scanning ReceiverTEMS Investigation 9.1 DataCollection

Support Network Standards:

WCDMA/HSDPA (2100, 900, 1900, 850, AWS) MHz

GSM (1800, 900, 1900, 850) MHz

Feature

Concurrent Measurements of Dual Technologies
http://www.sonyericsson.com/http://www.sonyericsson.com/


19/37

Copyright DiGi 2011

RSSI Scan across Multiple Bandwidths

Simultaneous Testing of All Modes

Built-in GPS

Available Measurements

WCDMA/HSDPA Top N BCH

Ec/Io (CPICH, PSCH and SSCH)

Io and Peak Ec/Io (CPICH)

Signal to Interference Ratio (SIR)

Rake Finger Count

Delay Measurements

GSM/EDGE/GPRS BSIC Decoding

Carrier to Interference (C/I) Option

BCCH Decoding Option

The scanner can make fast scanning than use mobile scanning, can provides highdensity measurements and the result from scanner is much accurate in frequencydetection.

4.3 Positioning Systems / GPS

GPS is used for locating the position when doing DT test and scanner test in outdoor.

GPS Model Support Test SoftwareGlobalSat BU-353 TEMS Investigation 11.0 Data

Collection

The above GPS or the hardware support for 2G test is used for the position system forthe outdoor test.

Specification

Datum WGS-84

Dimension 2.08''diameter*0.75''(53mmdia.*19.2mm)

USB Cable Length 60''(152cm)Voltage 4.5-5.5V DC input

Current 80mA typical

OperatingTemperature

40---85

If you want more information about this GPS, please visitwww.usglobalsat.com .

The USB GPS receivers requires a Windows laptop computer, with a CDROM*, USBport, and NMEA compatible navigation software. The 3rd party mapping/navigationsoftware may have its own system requirements, please check with software vendor fordetails.
http://www.usglobalsat.com/http://www.usglobalsat.com/


20/37


21/37

Copyright DiGi 2011

Supports multi-mode test: WCDMA and GSM/GPRS Networks.

Supports multiple phones test simultaneously, which includes ten terminals at most(six UE, three Scanner and one GPS), each phone can be controlled independently

Supports test of indoor radio signals, and supports test with or without pre-definedpaths.

Supports all GPS that comply with NMEA and communicate through the RS-232port, and can test and configure GPS rapidly and intelligently.

Supports Scanner measurement including pilot frequency of tow methods forTOPN and User List, continue wave (CW) and spectrum scan.

Can display the process of 2/3G handover directly and provide the contrast ofserving cell between before handover and after handover.

Supports link fault detection and automatic recovery of COM ports, supportsalarming for insufficient battery capacity and storage space, GPS location failure,link loss of handset and PNSCanner, and serial port CRC errors, as well asalarming for parameter definition of test items.

Can collect GPS location data accurately and rapidly, features powerful geographicdisplay function (for real-timely displaying parameters in different colors), and callevent display.

Supports real-time capturing and graphical analysis of L1, L2, L3 and layer NASdata, and powerful browsing, real-time decoding, filtering, and categorized displayof L3 messages.

Supports elaborate real time acquisition and parameters display, including view ofphysical channel parameters called layer 1, transport channel named layer 2,logical channel, namely MAC layer and RLC related parameters.

Powerful voice test, supporting test plan customization, automatic test based on theplan customized, supporting re-connection upon link disconnection, intelligentanalysis for causes of call failure, rapid problem location, and real-timely recordingthe statistics of the voice service dialing test.

Powerful data service test, with multiple protocols such as PPP, FTP, and PINGintegrated, supporting PPP re-dialing at link disconnection, PPP Call by Call test,supporting FTP and PING test. All the test processes can be displayed on the

interface as graphics or messages.

Can display the multiple paths of each pilot frequency, Ec/Io, and other parameters.

Supports simultaneous test of multiple handsets and multi-Scanner.

4.4.4 Applications

TEMS can be used to test air interface of WCDMA/GSM and HSDPA. It enablescollection of the signal transmitted and received by UE, and capturing the layer3messages.


22/37


TEMS can make automatic call test according to the relevant configuration, includingconfiguration of different call types and different duration.

TEMS is easy to operate and the user interface is friendly.

TEMS can smoothly upgrade.

TEMS can support HSUPA drive-test data collection.

4.5 Internal / External Antennas

Only the scanner will use external antenna. The antennas are also provided by thescanner equipment vendor and in accord with the scanner.

4.6 Network Servers

Data FTP upload / download server

The FTP upload and download server is provided by DiGi and the entire FTP DL/UL testwill use the same FTP address.

4.7 Laptop

The laptops are used for data collection and also data processing. The Minimum dataprocessing requirement is able to support UEs/MSs, data collection application andtesting scenarios for longer time. The laptop cannot run any of applications which arenot related to the test, it may slow down the processing power and affect the upload &

download throughput.

The least requirements for the test laptops are as following:

Operating System:

Windows2000/XP

Lowest Configuration:

CPU: 1GHz Memory: 1024MB

Video card: SVGA, 16-bit color display Monitor resolution: 800*600

Recommend Configuration:

CPU: Duo CPU with 2.0 GHz Memory: 2.00 GB

Video card: SVGA 32-bit color display Monitor resolution: 1280*800

Hard disk: 20GB remaining space


23/37

Copyright DiGi 2011

5 Test Kit Setup and Methods

5.1 Laptop/Data Collection Gear Setup

5.2 Trace and Data Test

Figure 1 Trace and Data Test Connection

Install and configure test USB port of the adapter box(USB Hub) if some notebook hasnot enough usb port for testing.


24/37


5.2.1 Scanner Test

Figure 2 Scanner Test

The Scanning Receiver can be connected directly to the serial port of your laptop ifapplicable. The direct serial port connection provides RF measurement information andGPS coordinate results.

5.2.2 Multiple Test

Figure 3 MultipleTest

Install and configure test USB port of the adapter box. Refer to manufacturersinstructions for more information.

Use the RS- 232 cable connects laptop and scanning receiver. Connect the RFinput port of the scanning receiver communications cable to the RF antenna.

Connect USB port of the adapter box to the Laptop USB port.


25/37

Copyright DiGi 2011

Connect the GPS antenna to the GPS ANT port of the adapter box. And usedate/trace cable connect test phone to the adapter box.

The test engineer will define the equipment based on the actual configuration of his

hardware.

5.3 Tools Kit Configuration

The main equipment for the test includes 2 mobile phones, 1 scanner, a notebook, aGPS, sometimes it will need a USB hub (in case the USB port is not enough), a harddog for TEMS.

We need configure several parameters in TEMS before test and to check whether allthe equipment status is normal.

5.3.1 TEMS Configuration Flow

In this section, we will introduce the configuration flow of TEMS, as this is importantbefore the test.

Figure 4 TEMS Configuration Flow


26/37


5.3.2 TEMS Configuration Procedures

Install TEMS Investigation on the portable computer. The installation is automatic.Just insert the CD and follow the on-screen instructions.

Install drives of the test hardware, such as drives for GPS, Scanner, and Mobilephone. USB drivers need to be installed in order to use the hardware for test.Whether for TEMS measurements or data service testing. These drivers arespecific to each equipment (hardware) supplier.

Connect the hardware for testing. Ensure the equipment is in good connection andthe test mobile phones and GPS can be identified and assigned ports.

Equipment Configuration for different tests. Most of the configuration is the samefor both packet-switched and circuit-switched service. Where differences exist, theyare pointed out on each occasion.

Command sequence and parameters setting for test. Command sequencedetermines the procedures of the test and parameters are used to control the testwhen the test starts.

After the configurations of the above steps, the test investigation can be start.

5.4 Voice Configuration Parameter

Parameters DescriptionShort CallValue

Long Call Value

Call Number The call number MT number MT number/

specific Number

Continuous Call A call with unlimited times No Yes

Call Count The number of calls unlimited times N/A

Call Time(S) The duration of the call 60s unlimited times

Idle Time(S) The idle time 10s 10s

5.5 Data Configuration Parameter

Item Parameter explanation Value

Attach PDP APN The APN in the PDP Activate Request DiGi to provided

FTPdownloadtest

Dial-Up Networking

During the test, the test software will dial-up the network automatically when thisoption is selected; otherwise system willindicate the user to make the dial-upmanually.

DiGi to provide

The proper APN should be choose andactivated if DiGi has more than one APN.

Server IP DiGi to provide

User name DiGi to provide


27/37

Copyright DiGi 2011

Password DiGi to provide

The local path to save thedownload file

The local path to save the download file Defined by the tester

FTP uploadtest

Dial-Up Networking

During the test, the test software will dial-up the network automatically when thisoption is selected; otherwise system willindicate the user to make the dial-upmanually.

DiGi to provide

The proper APN should be choose andactivated if DiGi has more than one APN.

IP address of the server DiGi to provide

User name DiGi to provide

Password DiGi to provide

Path of the file serverThe path and name of upload file used for

test in the FTP server

Defined by the tester on

PC

5.6 Measurement and PS/CS ConfigurationSpecifications

This section will describe each test step of all features in detailed, for example: Voice all,Ping test, FTP test and so on. ZTE will use TEMS to test at overlay network.

TEMS test setup Description

5.6.1 Voice call

Connect among the test phone GPS soft dog and laptop.

Configure the COM port of all equipment, and connect with laptop.


28/37


Set up long call and short call parameters


29/37

Copyright DiGi 2011

5.6.2 Short Call

After navigating to Voice Call Test, Call Monitoring window appears as shown in, InCall Monitoring window user can configure voice call parameters.

Software is used for automatic dial-up, at least 100 times.


30/37


Set hold-on time with 10s; set idle time with 10s; set call setup time with 10s.Record number of call success times and total number of origination times.

In the test process, the drive speed is according to the road traffic environment. .

5.6.3 Long Call

You can dial the special service number (provided by DiGi), so that you can use onlyone MS to do the long call test .Or tester can dial the MT phone number then 2 mobilephones will be used for testing.The duration you should set as large as possible.Besides, if you set redial options, when call-drop happens, the MS will redial.

5.6.4 FTP Test

The purpose of the FTP test is to verify the stability of data transfer rate, one MS is usedto originate the PS data service through the system. FTP download and upload functiontest will be performed. Following is the testing procedures:

On the test route, use MS to originate the PS data service through the system


31/37

Copyright DiGi 2011

The MS uses the FTP software to download a data file (450K), meanwhile the UEuses the ftp software to upload a data (450K) to the server, continuous downloadand upload in the test route.

In the test process, the drive speed is according to the road traffic environment.

Ftp server setting is following:

APN and IP Address will provided by DiGi.

5.6.5 Scanning

The purpose of the scanner test is to verify the coverage of the network, Following is thetesting procedures:

Select appropriate test route.

On the test route, connect the scanner to the laptop, set the frequency and recordthe signal data.

In the test process, the drive speed is according to the road traffic environment.


32/37


For the most part we don t choice decode the BSIC. It will consume lots of systemresource and will lead to inaccurate results. So if it is not really necessary we don tchoice decode BSIC.


33/37

Copyright DiGi 2011


34/37


6 Acceptance Reports Contents

The reports for 2G test for SSV, Cluster and Area will include the following content.

6.1 SSV/SSOA

6.1.1 Single Site Verification (SSV)

The Supplier shall commence and perform the site verification inclusive drive test foroutdoor site(s) or walk test for in-building site(s). The test cases below are generallydescribed at a high level which includes:

Site Data Verification

Cell Data Verification

Check Points Test

Accessibility Test Which Includes of Voice and Video Call

Data Accessibility

Throughput Test

Handover Test

Coverage Plot (Drive Test or Walk Test)

Quality Plot (Drive Test or Walk Test)

6.1.2 Single Site Optimisation Acceptance (SSOA)

Drive test/ Walk test and OSS statistic KPIs are required to be measured for thenew site and its 1st tier neighbour.

Drive test/ Walk test route will be defined by DiGi and this will be discussed and

mutually agreed between the Supplier and Purchaser

A report which consists of the SSV and SSOA will be submitted to DiGi after all the testitems passed.

6.2 Cluster Acceptance Report

Table of Contents

Executive Summary

KPI Results


35/37

Copyright DiGi 2011

Cluster Drive Test KPI results

Relevant GERAN sites KPI

OSS statistics

Cluster Information

Cluster Map Location

Cluster Site Information(1) Table containing Site ID, Site Name, Latitude, Longitude, Height, No. of antenna, Antenna

type, Antenna Tilt, Scrambling Code, BTS Type, RNC BSC(2) Attachment of the RRM parameter settings

Defined Measurement Test Routes

Cluster Drive Test Plots

Cluster Drive Test RSCP Plot

Cluster Drive Test Ec/Io Plot

Cluster Drive Test Voice BLER Plot

Cluster Drive Test Voice MOS Plot

Cluster Drive Test Video BLER Plot

Cluster Drive Test PS384 DL Throughput Plot

Cluster Drive Test Failed Events Plot (fail events include call setup fail, drop call,handover failure, location update failure)

Site Link Budget Calculation

Coverage Prediction Plots from planning tool

Issues and action/recommendation

Analysis of Failed Events and actions taken

List of other optimisation activities

List of outstanding items

Monthly Performance Reports

Table of Contents

Executive Summary

KPI Results


36/37


Summary table of current and previous round(s) of Cluster Drive Test KPI results

Summary table of current and previous round(s) of Cluster Stationary Test KPIresults

Summary table of current and previous round(s) of Cluster OSS Statistics KPIresults

Area Drive Test Plots











6.3 Area Network Acceptance Report

Table of Contents

Executive Summary

KPI Results

Summary table of Cluster Drive Test KPI

Summary table of monthly stationary tests KPI (optional)

Summary table of monthly Cluster OSS Statistics KPI

Area Network Information

Updated Node B cell design data

Table containing Site ID, Site Name, Latitude, Longitude, Height, No. of antenna,Antenna type, Antenna Tilt, Scrambling Code, BTS Type, RNC BSC


37/37

Attachment of the up to date RRM parameter settings

RNC configuration settings and Performance

Key Core Network and Service Network settings that may affect the biasness of theFinal Acceptance test

Defined Measurement Test Routes

Summary of Drive Test Plots







All 6 rounds of monthly Cluster Stationary Test (optional)

Table of results for every stationary test spots

Area Network OSS Statistic KPI trend for each KPI





2G Test Methodology and Definition V1.0

Documents