Verification & Acceptance of LTE Nodes

Slide titleIn CAPITALS

44 pt

Slide subtitle 20 pt

Verification and Acceptance of LTE Nodes


44 pt


Introduction

Slide title 30 pt

Text 18 pt

Bullets level 2-516 pt

›!"# $%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~¡¢£¤¥¦§¨©ª«¬®¯°±²³´¶·¸¹º»¼½ÀÁÂÃÄÅÆÇÈËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿĀāĂăąĆćĊċČĎďĐđĒĖėĘęĚěĞğĠġĢģĪīĮįİıĶķĹĺĻļĽľŁłŃńŅņŇňŌŐőŒœŔŕŖŗŘřŚśŞşŠšŢţŤťŪūŮůŰűŲųŴŵŶŷŸŹźŻżŽžƒȘșˆˇ˘˙˚˛˜˝ẀẁẃẄẅỲỳ–—‘’‚“”„†‡•…‰‹›⁄€™−≤≥fifl

Why learn about Acceptance

› Triggers Important legal events– Right to invoice and receive payments– Avoid penalties and liquidated damages– Transfer of Title to the customer– Start of warranty period

› Acceptance is extremely vital for Ericsson’s cashflow and margins

› Product Verification– Ensuring the product works as expected

› Quality Assurance to the Customer

› Dynamic Acceptance Criteria

Slide title 30 pt

Text 18 pt



Scope and objectives

Upon completion of this course, you will be able to:

› Understand the requirements of planning for acceptance

› Explain the process of LTE Node Acceptance

› Understand method of Acceptance Execution

Objectives

Scope

› LTE RAN Overview

› Acceptance Processes

› Acceptance Planning & Execution


44 pt


> Overview

Slide title 30 pt

Text 18 pt



Target Group

› Ericsson personnel working in LTE RAN domain,

including:–Services Engineers

– Test Leaders and Coordinators

–Solution Architects

–Consultants

–Customer Project Managers

–Customer Solution Responsible

–Sales Support Engineers

Slide title 30 pt

Text 18 pt



LTE Acceptance

› Vital step in contractual closure

› Covers all aspects of network deployment and service delivery

› Can be a costly and time consuming process

› Assures product and service quality to Customers

› Criteria defined during contact negotiations–Network is designed and implemented to meet this criteria


44 pt


LTE RAN Overview

Slide title 30 pt

Text 18 pt



LTE RAN OverviewOverview

› Evolution

› Nodes and Interfaces

› Architecture

HSUPAMBMS

Rel 6

MIMOHOMCPC

Rel 7Rel 4R99

HSDPA

Rel 5

4G4G

Further enhancements

WCDMA/HSPAWCDMA/HSPAWCDMAWCDMA HSPA EvolutionHSPA EvolutionRel 8

LTELTELTE AdvancedLTE Advanced

Slide title 30 pt

Text 18 pt



Mobile Evolution

CDMA Track (3GPP2)

GSM Track (3GPP)

2001 2005 2008 2010

LTEFDD and TDD

GSM WCDMA HSPA

TD-SCDMA

CDMA One EVDO Rev A

LTE - FDD- TDD

1G 2G 2.5G 2.75G 3G 3.5G 3.9G 4G

Technology FDMA GSM GPRS EDGE WCDMA HSPA LTE IMT Advanced

Voice Analogue CS CS CS CS CS VoIP only

Data - 9.6Kbps 50Kbps 150Kbps 384Kbps 42Mbps 100Mbps 1Gbps

Slide title 30 pt

Text 18 pt



LTE Nodes and Interfaces

Other Management

Systems

Other Management

Systems

ERBS

Core Network

ERBS

ERBS

Radio AccessNetwork

X2

S1

S1

Uu

OSS-RCOSS-RC

Mul

Mul

MunNetwork

ManagementEnvironment

UE

Uu

X2

S1

SAE GWSAE GW MMSMMS

ERBSERBS

Slide title 30 pt

Text 18 pt



LTE Architecture

eNB eNB

eNB

S1

X2

X2

X2

SAE (System ArchitectureEvolution)

LTE (Long Term Evolution)

EPC (Evolved Packet Core)

E-UTRAN

EPS (Evolved Packet System)

UE

Uu

MME MME

HSS

P/S-GW P/S-GW

S6a


44 pt


Acceptance Processes

Slide title 30 pt

Text 18 pt



Acceptance ProcessesProcesses

› Product/Supply/Service Acceptance

› Network Rollout Steps

› Acceptance Timelines & Milestones

Slide title 30 pt

Text 18 pt



LTE Acceptance

Product Product AcceptanceAcceptance

Product Product Functions and Functions and CharacteristicsCharacteristics

Supply Supply AcceptanceAcceptance

Hardware and Hardware and Software Software DeliveryDelivery

Service Service AcceptanceAcceptance

Service Service PerformancePerformance

Slide title 30 pt

Text 18 pt



Product Acceptance

Product Acceptance

› Optional activity

› Feature Demonstrations only

› Covers product related aspects demanded by the customer

› Field validation of some product features, eg. network configuration

› Should only be performed once, at the first instance of the product

Slide title 30 pt

Text 18 pt



Product Acceptance cntd.

Product Acceptance

Feature Demonstration

› Focus on new functionality

› Minimise demonstration of legacy features

› Before test execution:– Scope/criteria agreed with the customer– Ready for Validation (RFV) is reached– Demo execution begins

› → LTE Toolbox [Acceptance Gateway] - DOL

Slide title 30 pt

Text 18 pt



Product Acceptance cntd.

Product AcceptanceInter-Operability Test (IOT)

› Assigned by BNET IOT Program

› Limited to BNET wireless products

› Run based on bilateral agreements with strategic vendors

› All the IOT information is protected by NDAs, Non Disclosure Agreements, and demands other parties permission prior disclosure

Slide title 30 pt

Text 18 pt



Supply Acceptance

Supply Acceptance

› Delivery of Hardware and site material

› Delivery of Hardware Activation Code and Software Licenses

› Delivery of Software, license documents and keys

Slide title 30 pt

Text 18 pt



Supply Acceptance

Supply AcceptanceHardware

› Before shipment, all nodes are loaded with software, configured and fully tested in the factory

› No faulty equipment is ever shipped

› Factory Test Report is delivered, showing results of factory tests

› Installation and Integration Services perform onsite tests, confirming that the node starts and runs correctly

Slide title 30 pt

Text 18 pt



Supply Acceptance

Supply AcceptanceHW AC/SW LK

› Accepted when the licenses are made available to the customer in ELIS

› Difficult and expensive to demonstrate capacity in the field – avoid capacity tests

› HW AC and SW LK are limited by the installed hardware, and restricted by the codes ordered

› Customers may request on site activities for full acceptance

Slide title 30 pt

Text 18 pt



Supply Acceptance

Supply AcceptanceHW AC/SW LK

Capacity Licenses

CapacityChannelBandwidth=10MHzCapacityChannelBandwidth=15MHzCapacityChannelBandwidth=1p4MhzCapacityChannelBandwidth=20MHzCapacityChannelBandwidth=3MHzCapacityChannelBandwidth=5MHz

CapacityConnectedUsersCapacityOutputPower=40WattCapacityOutputPower=60Watt

DlBasebandCapacityUlBasebandCapacity

Optional Features

AnrCdma2000SessionContinuity

Dl64QamDualAntDlPerfPkg

GsmSessionContinuityIntraLTEHandover

IntraLTEHODataFwdMultiErabsPerUser

PmInitiatedUeMeasurementsRetCascading

RetConfigurationMgtRetSupportTmaSupport

Ul16QamVswrSupervision

WcdmaSessionContinuity

Slide title 30 pt

Text 18 pt



Supply Acceptance

Supply AcceptanceSoftware

› Basic SW is not normally subject to acceptance activities

› Features are normally demonstrated as a part of Product Acceptance if requested

› Remaining payments may be triggered by startup of the equipment (installation or integration service)

› Customers may request on site activities for full acceptance

Slide title 30 pt

Text 18 pt



Service Acceptance

Service AcceptanceRecommended KPI’s

› Session Setup Success Rate

› Session Abnormal Release RateONLY used for drive tests with short calls

or› Minutes per Abnormal Release

Used for stats based KPI & drive tests with long calls

› DL User Throughput

› UL User Throughput

Slide title 30 pt

Text 18 pt



Network Rollout steps

AcceptanceExecution

AcceptancePlanning

Slide title 30 pt

Text 18 pt



Acceptance Types

Hardware Acceptance

Normally accepted on Delivery

Integration Services

Normally accepted on Integration

HW Activation Codes/SW License Keys

Normally accepted on Demonstration

Software Acceptance

Normally accepted on Demonstration

Installation Services

Normally accepted on Installation

Tuning Services

Normally accepted on Cluster Test

KPI Based Acceptance

Slide title 30 pt

Text 18 pt



Site Acceptance timeline

Hardware Acceptance

Software Acceptance

Integration Services Acceptance


Acceptance

Installation Services Acceptance

Tuning Services


Slide title 30 pt

Text 18 pt



Milestones

Hardware Acceptance

Software Acceptance

Integration Services Acceptance

Installation Services Acceptance


Acceptance

Tuning Services Acceptance


Lab Demonstratio

n

Single Site Integration & Acceptance

Cluster / Network

Acceptance

Slide title 30 pt

Text 18 pt



Hardware Acceptance

› Accepted on delivery to a warehouse (or similar)

› Accepted on a per node basis– based on factory test reports that are provided with the node

› Independent on the availability or readiness of sites.

› AVOID: tying to the completion of other NRO services:– like integration or tuning

Hardware Acceptance

Slide title 30 pt

Text 18 pt



HW Activation Codes /SW License Keys

› The HW Activation Codes and SW License Keys fullyaccepted when the codes are made available to thecustomer in ELIS (the license management system)

› Demonstration in a lab environment (lab acceptance)

› Upgrade of a first network node (First Node Implementation, FNI)

› Licensing mechanism accepted after completion of the lab demonstration, and latest after the FNI


Acceptance

Slide title 30 pt

Text 18 pt



Software Acceptance

› New software releases, typical procedure:1. Demonstration in a lab environment (lab acceptance)

2. Upgrade of a first network node (FNI)

3. Network Upgrade

› SW Acceptance should be done early, during lab demonstration.– Should be based on generic acceptance documentation provided with the new SW

release (DOL, DI etc)

› AVOID: tying to the completion of NRO or upgrade services

› New features:– should be done in a test environment with Ericsson nodes in all parts of the

network, especially core network– Only terminals that have been part of Ericsson’s IODT or IOT testing should be

accepted– In case the customer wants to use Core networks or terminals from other vendors,

this activity should be offered as a separate IOT service

Software Acceptance

Slide title 30 pt

Text 18 pt




› Network Performance is dependant on outsidefactors:– Radio tuning/optimisation– Terminal performance

› AVOID: KPI based acceptance as part of product acceptance criteria

› The following should be considered before committing:– Small, well-defined cluster– Ericsson takes full responsibility for the design and tuning of the cluster– Only terminals that have passed Ericsson IODT or IOT are used– Cluster and KPI targets are assessed by Ericsson NRO– Ericsson should perform the execution of acceptance tests



44 pt


Acceptance Planning & Execution

Slide title 30 pt

Text 18 pt



Acceptance Planning and Execution CasesCases

› General Equipment & Software

› Test Cases

› UE / Network Evidences

› Generic Lab Setup

› Lab Demonstrations

› Single Site Acceptance

› KPI Based Acceptance

Slide title 30 pt

Text 18 pt



General Equipment

› UE terminals– Only use terminals that are verified by IODT

› Lab Demonstration– RF Cables & Connectors (for clean RF conditions)– OFDM Spectrum Analyser– Fixed and Variable Attenuators (Signal power manipulation)– OFDM Noise Generator (Signal noise manipulation)– OFDM Fading Emulator (mobility fading emulation) –

optional & expensive– MIMO Box – optional– Test UEs and laptops

› Field Demonstration– Drive Test Vehicle (preferably with external roof-mounted

antennas)– Test UEs and laptops

Slide title 30 pt

Text 18 pt



General Software

› UE Reporting Software– TEMS Investigation

› UE Performance Measurement and Demonstration– iperf/eperf (TDP/UDP Performance)– Filezilla (FTP Client Software)– NetPerSec (Throughput Demonstration)

› Network monitoring– ping (measuring roundtrip times)– Wireshark (Packet analyser)

Slide title 30 pt

Text 18 pt



Test Cases

› Starting point → Demo Object List

› The following should be defined for each test case:– Test Case Details – description of what the TC does– Objective – description of what the TC means to prove– Pass/Fail Criteria – definition of what evidence needs to be presented to pass the

test case– Parameter Setting – details of which non-generic (TC specific) parameters need to

be set– Demo Execution Steps – step-by-step instruction for executing the TC, describing

Actions and expected Results

› How can you prove the correct outcome?– What evidences do you need?

Slide title 30 pt

Text 18 pt



UE Evidences

› Layer 3 Signalling– RRC (Radio Resource Control)– NAS (Non-Access Stratum)

› UE Measurements– RSRP (Reference Signal Received Power)– RSRQ (Reference Signal Received Quality)– SINR (Signal to Interference + Noise Ratio)– BLER (Block Error Rate)– RI (Rank Indicator)

› General Measurements– Throughput (iperf / NetPerSec)– Latency / Round trip times (ping)

Slide title 30 pt

Text 18 pt



Network Evidences

› eNodeB – Layer 3 Signalling– RRC (Radio Resource Control)– S1 (eNodeB to MME)– X2 (eNodeB to eNodeB)

› eNodeB Radio Measurements

› KPI based Acceptance– Performance Counters– UE/Cell Trace

› End to End Performance– Latency (ping RTT)– Dedicated Bearer (PCRF)– TCP/UDP Performance (Transmission

Bottlenecks)

Slide title 30 pt

Text 18 pt



Example Lab Demo

A

UE Configuration

Test UE +TEMS

Load UE

Loading UEs

Test UE

Test Cell

A

Same-cell load UE

Test UE

Test Cell

Other-Cell

B

Same-cell load UE

Single Cell Environment Multiple Cell Environment

Slide title 30 pt

Text 18 pt



Example Radio Conditions

Uplink

Downlink

Slide title 30 pt

Text 18 pt



Example Generic Lab Setup

Slide title 30 pt

Text 18 pt



Single Site Acceptance

› Follow Site Integration Checklist– CPI: Integrating RBSs On Site

› Additional tests may be required by the customer

› Generally covers:– Hardware Acceptance– HW Activation Codes & SW License Keys Acceptance– Installation Services Acceptance– Integration Services Acceptance

Item Description Yes No N/A

1 Site basic and site equipment configuration complete?

2 Configuration finished indication received?

3 External alarms OK?

4 Each cell have expected PCI (or CGI)?

5 Traffic session OK?

6 Integration report stored on the client?

7 Integration report stored on an FTP server?

8 General concluding routines are performed?

Slide title 30 pt

Text 18 pt




Progress & Cluster KPI Achieved Payments Progress and Cluster/Network KPI Achieved Payments

1) Refer to NDI Tendering and Contracting Guidelines

2) Which ever comes first 3) Delayed deployed sites or expansion sites

Launch Readiness

& Network

Acceptance

Post Launch Optimisation

Post Launch Optimisation Acceptance

RadioNetwork Design

OR

RadioNetwork Design Review

Initial Tuning (Golden Cluster)

Radio Network Design

Accepted

Golden Cluster

Acceptance

Golden Cluster

Accepted

Network Acceptance2

System Launch2

Post Launch Optimisation

Accepted

NDI Milestones1

Clusters Accepted3

Initial Tuning

Cluster Acceptance

Type 2

Initial Tuning

Cluster Acceptance

Type 2

Initial Tuning

Cluster Acceptance

Type 2

Initial Tuning

Cluster Acceptance

Post-Launch3Initial Tuning

Cluster Acceptance

Type 1

Initial Tuning

Cluster Acceptance

Type 1

Initial Tuning

Cluster Acceptance

Type 1

Initial Tuning

Cluster Acceptance

Pre-Launch – Area X

Area XClusters Accepted

Network Design

Site Integration

(NDI)

Initial (Cluster) Tuning

System Launch

Post Launch Optimization

RadioNetwork Design

Acceptance

Milestones

Contract Final Acceptance4

Final Acceptance4

4) Optional overall contract status review


44 pt


exercises Apply the learnings

Slide title 30 pt

Text 18 pt



ExerciseS

› Requirements:– Access to Ericsson Intranet– Understanding of Ericsson LTE products

› Exercises:– Read the DOL and try executing test cases

› Plan execution, proof, procedure› Execute the TC

– Read KPI based acceptance– Browse the LTE Toolbox, Acceptance Gateway

› Read Business Critical Terms and Conditions› LTE Acceptance Priciples› LTE Acceptance Approach› LTE KPIs


44 pt


Summary

Slide title 30 pt

Text 18 pt



Summary

› Acceptance is a crucial step in contractual closure

› Provides quality assurance

› Criteria is often dynamic and must be negotiated before

contractual commitments

› Requires careful planning

› Acceptance can be costly and time consuming– Focus on minimising efforts

–Avoid proving already proven things

–Plan in advanced, especially with KPI based acceptance


44 pt


QUIZ

Slide title 30 pt

Text 18 pt



Quiz

› Acceptance can trigger the following legal event:– Right to invoice and receive payments– Avoid penalties and liquidated damages– Transfer of Title to the customer– Start of warranty period– Any of the above

› Single Site Acceptance does not cover:– Integration Services– KPI Based Acceptance– Installation Services

› Acceptance criteria is defined after Single Site Integration:– True– False

Slide title 30 pt

Text 18 pt



Quiz – Answers

› Acceptance can trigger the following legal event:– Right to invoice and receive payments– Avoid penalties and liquidated damages– Transfer of Title to the customer– Start of warranty period– Any of the above

› Single Site Acceptance does not cover:– Integration Services– KPI Based Acceptance– Installation Services

› Acceptance criteria is defined after Single Site Integration:– True– False

Slide title 30 pt

Text 18 pt



More Information

› Acceptance Gateway [link]

› CPI L11A [link] – Preliminary

– Integrating RBSs On-Site [link]

› Demo Object List L11A – DOL [link]

› Related Centra Sessions – LTE RAN GCC [link]

› KPI-Based Acceptance [link]

Verification & Acceptance of LTE Nodes

Documents

liquidated

radio resource

integration

cell load

product acceptance

single site

single site

kpi based