219905365 lte-kp is-and-acceptance

Slide title

In CAPITALS

44 pt

Slide subtitle

20 pt

LTE KPI’s and Acceptance

Slide title

In CAPITALS

44 pt

Slide subtitle

20 pt

Introduction

Slide title

30 pt

Text

18 pt

Bullets level 2-5

16 pt ›!"# $%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~¡¢£¤¥¦§¨©ª«¬®¯°±²³´¶·¸¹º»¼½ÀÁÂÃÄÅÆÇÈËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿĀāĂăąģĤĊċĥĎďĐđČĖėđĒēĔĞğĠġĢģĪīĮįĠıĶķĕĖĻļėĘŁłŃńĽľŅņŌŇň․ŖŊŉŗŋŌōŎġĢŠšŏŐőŒŪūœŔŕŖŲųŴŵŶ

ŷ ŗŘřŚŽžƒȘșˆˇ˘˙˚˛˜˝ẀẁẃẄẅẆẇ

‡†„‧‚…‥…•‣⁄€−≤≥fifl

Why learn about ACCEPTANCE & KPI…s

› Acceptance is part of Business Critical Terms & Conditions

› Control of KPI commitments is control of cash flow and profitability

› Ericsson had 80BSEK locked in Acceptances in 2008

› If all projects finished one day earlier = 500 MSEK/day for Ericsson

Slide title

30 pt

Text

18 pt

Bullets level 2-5



‡†„‧‚…‥…•‣⁄€−≤≥fifl

Scope and objectives

› Explain how Acceptance & KPI’s are

impacting business

› Describe Ericsson view on LTE

Acceptance principles and KPI’s

› List LTE KPI approach and point to

further information

Objectives

Scope

› Customer and Ericsson view on

Acceptance and KPI’s -> business Impact

› KPI Based Acceptance

› Understanding the KPI’s

› LTE KPI’s

Slide title

In CAPITALS

44 pt

Slide subtitle

20 pt

Overview

Slide title

30 pt

Text

18 pt

Bullets level 2-5



‡†„‧‚…‥…•‣⁄€−≤≥fifl

LTE ACCeptance principles › HW should be accepted on delivery. The LTE product HW is tested at the factory

according to Ericsson quality procedure. Normal claims and warranty naturally

apply.

› HW Activation Codes should be accepted when the codes are made available to

the customer in ELIS, the license management system.

› SW should typically be accepted after successful demonstration in the customer

lab, following the Demo Object List (DOL). This should include the mechanisms for

both HW Activation Codes, and for SW Feature Activation Licenses.

› Installation services should be accepted when the node has been installed and

stand-alone tests have been successfully completed.

› Integration services should be accepted when the node has been configured and

integrated into the network, and stand-alone tests have been successfully

completed.

› Tuning services should be accepted when tuning service has been completed

and documented, ideally on a cluster per cluster basis.

Separate payments for HW, SW and Services !

Slide title

30 pt

Text

18 pt

Bullets level 2-5



‡†„‧‚…‥…•‣⁄€−≤≥fifl

LTE Acceptance tips

Not recommended:

› Contractual formulations including delay of fraction of payment of

HW/SW coupled to fulfillment of service

› Contractual formulations including liquidated damages (referring to

payment of HW/SW) coupled to fulfillment of service

Wrong acceptance approach has:

› Effect of tied capital –increases our Employed capital/Net sales ratio

› Effect of lowered margins – increased costs

If we tie up KPI…s with the HW and SW sales we might end up like this...

Slide title

30 pt

Text

18 pt

Bullets level 2-5



‡†„‧‚…‥…•‣⁄€−≤≥fifl

CASH FLOW Scenario

Upfront 15%

HW

75%

Delivery

SW

75%

Delivery

Services

75%

Monthly

Progress

Final Acceptance

10%

Recommended Terms

10%

75%

15%

Penalties

Discounts

! High risk

Bad Cash flow

Cost blow out

KPI Based Acceptance Terms

Upfront 15%

HW

x%

Delivery

SW

y%

Delivery

Services

z%

Progress

Final Acceptance

10%

KPI

Targets

25% plus

50% or less

KPI linked

payment risk

Slide title

30 pt

Text

18 pt

Bullets level 2-5



‡†„‧‚…‥…•‣⁄€−≤≥fifl

Execution Acceptance Invoice/Collection

CAPITAL

EMPLOYED

TIME

Unbilled

Sales Billed Receivables

Inventory

ITO days: 83 days Account Receivables, DSO: 115 days

Overdue

Revenue

recognition

First invoice sent

Total: >80BSEK

Contract terms & conditions, project closure, and billing, key areas to address

“CASH OUT!”

“CASH IN!”

Operators are using complex acceptance

procedure with unrealistic KPIs ...

Slide title

In CAPITALS

44 pt

Slide subtitle

20 pt

KPI BASED ACCEPTANCE

Slide title

30 pt

Text

18 pt

Bullets level 2-5



‡†„‧‚…‥…•‣⁄€−≤≥fifl

Why ARE CUSTOMERS DEMANDING KPIS ?

It…s not just about performance IT…S CONTROL AND COST!

TO CONTROL...

RISKS COSTS

› Internal pressure

› Don’t understand the

technology -> LTE !

› Force Ericsson to fix their

performance issues

› Avoid risk, shift risk to

Ericsson

› Own cash flow

issues

› Extra services at no

cost

› Penalties, discounts

or things for free

› Avoid paying

Ericsson

Slide title

30 pt

Text

18 pt

Bullets level 2-5



‡†„‧‚…‥…•‣⁄€−≤≥fifl

Ericsson Driving Forces

› Profitability, ability to fulfill contract

› Cash flow, timely invoicing

› Separate payment terms for HW, SW, and Services

› Close contract with clear responsibility, scope and acceptance

criteria

› Control of contract fulfillment

› Additional sales

Slide title

30 pt

Text

18 pt

Bullets level 2-5



‡†„‧‚…‥…•‣⁄€−≤≥fifl

Contractual Acceptance directive

1. Acceptance should have a defined scope, procedure and

timetable agreed in advance,

2. To be based upon objective and measurable criteria

3. KPI based acceptance should be limited to network and

technology consulting part of rollout contracts and limited to KPIs

defined in the KPI library at the Acceptance Gateway.

4. If products or parts are put into operation, automatic final

acceptance of such parts

Reference: Business Critical Terms&Conditions documents

http://anon.ericsson.se/eridoc/component/eriurl?docno=03402-3096/5Uen&objectId=09004cff82670e79&action=current&format=pdf







Slide title

30 pt

Text

18 pt

Bullets level 2-5



‡†„‧‚…‥…•‣⁄€−≤≥fifl

ContracTual considerations and mu…s

› Contractual Considerations: The MU shall always strive to follow the

Acceptance Principles (Acceptance Approach and the Business Critical Terms

and Conditions).

› Restriction: If the MU deviates from the Acceptance Principles, it should

clearly state the deviations and estimate the risks and associated impact on

the business case for the contract.

Deviations from the Acceptance Principles must be presented to the LTE Deal

Strategy meeting, pre-SDP2.

The MU is responsible to secure required input, Price Governance, material (PPT,

Xls )

Set up of this meeting is done by contacting Christer Gustafsson or Linda Lindberg

(manager EAB/FRP).

http://cpi.al.sw.ericsson.se/alexserv?ID=17018&DB=23677-lzn7840900_1r4a.alx&FN=1_10056-HSC10550_1Uen.E.html


http://internal.ericsson.com/wps/portal/yasper/!ut/p/c1/04_SB8K8xLLM9MSSzPy8xBz9CP0os3hjN8MQD3MPIwMD30BnAyNLRzOPsDBDQ3cXM6B8JE55A0djArrDQfbhVmFqgFcebD4-eW9jfT-P_NxU_YLcCIPMgHRFAErM0NQ!/dl2/d1/L0lDU0NTS0tVZ2tra0tDbEVLQ2xFS0NsRS9ZRVVFQUFBSUlDSUFBTU1FSUtJRUFH



http://internal.ericsson.com/wps/portal/yasper/!ut/p/c1/hZDNjoIwAISfxQfYtNAKcoSyUMCi_AiUCwEkivwmuIo8_bp72ZNu5vjNTDIDUvBUn9_qU36thz5vAQepnCFDCKlMRbjZUwmKCNs7DBmCWAZRl891Vy_VEYQggTgLLnBk26lni7Lbh5_NNSgX5hw5JedDYscV_qhaanflIfXzZNQKbaYPlX9J7RzEdl4mhkcE7xApmZEh




Slide title

30 pt

Text

18 pt

Bullets level 2-5



‡†„‧‚…‥…•‣⁄€−≤≥fifl

Acceptance activities (typical)

Qualify

Sales

Opportunities

Create

Proposal Close Deal

Contract

Management

Contract

Handshake

Risk

Full Track

SDP 4 SDP 1 Risk SDP 2 Risk SDP 3

Critical Phase

Work out

your strategy

Prepare

offer/

contract/

Strong

wanted

position

Negotiate

Refine

Agree

Work out

the details –

Keep it simple –

Keep costs

down

Execute –

Look for ways

to save and

close out +

add on sales!

Ability to influence

Correction Cost

Slide title

30 pt

Text

18 pt

Bullets level 2-5



‡†„‧‚…‥…•‣⁄€−≤≥fifl

ACR

Strategy

Payments

Satisfaction

KAM

CSR

System

Solution

Service

Solution

CFR

Cash Flow

Delivery

Execution

Ensure link between

Terms of Payment

and Acceptance

Procedure.

(Particularly Cashflow)

Align Services (SoW)

and Solution with

Acceptance

Commitment incl. KPIs

Ensuring KPI Based

Acceptance is

integrated in total

Acceptance Procedure

Core 3 Responsibility KPI Based Acceptance

• Overall Decision

• Ensure Acceptance risks are

understood by team and push

for flow between solution and

Acceptance

• Promote use of KPI Champion

KPI Based Acceptance Champion

../AppData/Local/Microsoft/Windows/Temporary%20Internet%20Files/AppData/Documents/Acceptance/Assignments/Presentation%20in%20C:/Documents%20and%20Settings/epagrah/Master%20Tender/China/Checklists/Decisionform%20Core%20Review%20-%20Finalization%20Proposal.xls

http://bmob.ericsson.se/af/acceptance/acceptance_info.shtml

http://bmob.ericsson.se/af/acceptance/acceptance_info.shtml

Slide title

In CAPITALS

44 pt

Slide subtitle

20 pt

Understanding the KPI…s

Slide title

30 pt

Text

18 pt

Bullets level 2-5



‡†„‧‚…‥…•‣⁄€−≤≥fifl

Basic facts KPI…s

› Operator thinking: More KPI’s, Operator

is getting more control

› Ericsson thinking: More KPI’s, higher

costs, longer procedure, risk for delays in

payment and penalties

› LTE is new, we can expect a lot of KPI’s

since operators want to protect

themselves

› We can be active and suggest KPI’s,

rather than wait for them - We know what

we are doing

› Right support is provided from the LTE

start – look at the Acceptance gateway

What KPI’s and what values to discuss

Acceptance method

Slide title

30 pt

Text

18 pt

Bullets level 2-5



‡†„‧‚…‥…•‣⁄€−≤≥fifl

WHAT IS IMPACTING KPI VALUES ?

TARGETS

PRE-CONDITIONS HOW TO

MEASURE

CONDITIONS

EXCLUSIONS

PROCESSES

Slide title

30 pt

Text

18 pt

Bullets level 2-5



‡†„‧‚…‥…•‣⁄€−≤≥fifl

Common mistake Reverse engineering

Often only KPI targets are stated in the contract

TARGETS

PRE-CONDITIONS HOW TO

MEASURE

CONDITIONS

EXCLUSIONS

PROCESSES

Slide title

30 pt

Text

18 pt

Bullets level 2-5



‡†„‧‚…‥…•‣⁄€−≤≥fifl

Kpi…S AND PROJECT TYPES

› In Large Rollout Projects where we have a lot of

risks, unknowns and dependancies and where

services are small part - Minimise KPI risks and

costs

› In Standalone projects where we have mostly

services, and we have control - Use KPIs as

business driver

KPIs can result in good business !

Slide title

In CAPITALS

44 pt

Slide subtitle

20 pt

LTE KPI…s

Slide title

30 pt

Text

18 pt

Bullets level 2-5



‡†„‧‚…‥…•‣⁄€−≤≥fifl

LTE kpi Acceptance Types of Acceptance

› The scope of a contract will determine the type of acceptance used

› Four different types of acceptance are proposed in this document

Product (Supply) Acceptance

Proof of Concept Acceptance

Network (or Cluster) Acceptance

Stability Acceptance

› A network contract may stipulate more than one of these types of

acceptance

› It is desirable to minimise the acceptance testing – it can be

consuming and costly

KPI’s handling will depend on the type of Acceptance

Slide title

30 pt

Text

18 pt

Bullets level 2-5



‡†„‧‚…‥…•‣⁄€−≤≥fifl

There are 4 KPI recommended for use in LTE acceptance

Session Setup Success Rate

Session Abnormal Release Rate ONLY used for drive tests with short calls

or

Minutes per Abnormal Release Used for stats based KPI & drive tests with long calls

DL User Throughput

UL User Throughput

Recommended KPI For use in LTE Acceptance

1

2

3

4

Slide title

30 pt

Text

18 pt

Bullets level 2-5



‡†„‧‚…‥…•‣⁄€−≤≥fifl

Recommended KPI Recommended Location for KPI Measurement

No Recommended KPI Controlled

Environment Golden Cluster

Network

(Cluster)

Drive Test

Counters /

Statistics

1 Session Setup Success Rate X X X X

2a Session Abnormal Release Rate X X X

2b Minutes per Abnormal Release X X

3 DL User Throughput X X X

4 UL User Throughput X X X

Product (Supply) /

Proof of Concept

Acceptance

Network

Rollout

Acceptance

Network

Stability

Acceptance

Slide title

30 pt

Text

18 pt

Bullets level 2-5



‡†„‧‚…‥…•‣⁄€−≤≥fifl

Recommended KPI example

Session Setup Success Rate

98.00%

99.0%

99.3%

99.50%

97.50%

98.00%

98.25%

98.50%

96.00%

96.50%

97.00%

97.50%

98.00%

98.50%

99.00%

99.50%

100.00%

L10A L10B L11 L12 =>

Lab / Controlled

Field / Drive test

Comments and Recommendations for Acceptance:

› SSSR should be tested using terminals which have undergone IODT with Ericsson

› Short calls are recommended for field test acceptance

› Lab and Controlled environment testing is to be undertaken in good radio conditions

(RSRP > -85dBm, SINR > 6dB)

› Exclusions should be included for drive testing where the setup failure is caused by

a factor beyond Ericsson’s control (eg. No coverage due to radio network design

constraints)

Slide title

30 pt

Text

18 pt

Bullets level 2-5



‡†„‧‚…‥…•‣⁄€−≤≥fifl

Throughput KPI High Level Summary: Acceptance Process

DL Throughput (EPA 5Hz) vs SNR 10MHz

0

10000

20000

30000

40000

50000

60000

70000

80000

-5 0 5 10 15 20 25 30 35 40

SNR (dB)

Th

rou

gh

pu

t (k

bp

s)

Simulation value

Maximum Acceptance Value

Note: Graph is only an example, real values should be

calculated using Coverage and Dimensioning

Guideline

› Special methodology for the

KPI User throughputs

› Applications, measurement

methods, exclusions

explained

› Theoretical max value

calculations (based on NW

configuration) vs. Practical

value

› Throughput KPI’s are well

documented

› Safe way for Ericsson to

take lead in the KPI

negotiations

Slide title

30 pt

Text

18 pt

Bullets level 2-5



‡†„‧‚…‥…•‣⁄€−≤≥fifl

Optional KPI For use in LTE Acceptance

There are 6 optional KPI for possible use in LTE acceptance:

Simultaneous Connected Users per RBS

Intra LTE HO Success rate

Round Trip Time (RTT)

Terminating Session Success Rate (Paging)

Control Plane Latency (Idle to Active)

Cell Availability

9

8

7

6

5

10

Slide title

30 pt

Text

18 pt

Bullets level 2-5



‡†„‧‚…‥…•‣⁄€−≤≥fifl

quiz

LTE Acceptance Strategy

Quiz

(continue to the next slide to start the Quiz)

PROPERTIES

On passing, 'Finish' button: Goes to Next Slide

On failing, 'Finish' button: Goes to Next Slide

Allow user to leave quiz: At any time

User may view slides after quiz: At any time

User may attempt quiz: Unlimited times

Slide title

In CAPITALS

44 pt

Slide subtitle

20 pt

Summary

Slide title

30 pt

Text

18 pt

Bullets level 2-5



‡†„‧‚…‥…•‣⁄€−≤≥fifl

Summary LTE Acceptance & KPI…s

› KPI’s are often financial driver, rather than technical

› Keep it simple, minimize number of KPI’s

› Consult "LTE Acceptance Toolbox" for overall

procedure

› For KPI’s listing (Recommended and Optional), and

their definitions consult "Acceptance KPIs - methods

and values"

› KPI Target values are last in chain - consequence of

pre-conditions

› Special care is given to the throughput related KPI’s

– systematic and structured approach

› Negotiate with customer

› For more information about KPI Based Acceptance

please contact Graham Allen or Marius Sirbu

Slide title

In CAPITALS

44 pt

Slide subtitle

20 pt

references

Slide title

30 pt

Text

18 pt

Bullets level 2-5



‡†„‧‚…‥…•‣⁄€−≤≥fifl

REFERENCES

› LTE Acceptance toolbox

› Business Critical Terms & Conditions

› LTE Acceptance approach – HW, SW, Services Acceptance

› KPI Based Acceptance.

› LTE KPI recommendations and values for rollout projects Acceptance KPIs - methods and values

› Network Rollout Projects - KPI Tendering&Contracting

› Price Governance material (PPT, Xls )

http://internal.ericsson.com/page/hub_inside/products/productmanagement/acceptance/lte/index.jsp














http://anon.ericsson.se/eridoc/component/eriurl?docno=EAB-09:049437Uen&objectId=09004cff834deafd&action=current&format=msw8















http://internal.ericsson.com/page/hub_inside/products/productmanagement/acceptance/kpi_acceptance/index.jsp





http://anon.ericsson.se/eridoc/component/eriurl?docno=EAB-09:059392Uen&objectId=09004cff8364864b&action=current&format=ppt8












http://internal.ericsson.com/page/hub_inside/products/productmanagement/acceptance/kpi_acceptance/Tendering_and_Contracting.jsp













Slide title

30 pt

Text

18 pt

Bullets level 2-5



‡†„‧‚…‥…•‣⁄€−≤≥fifl

219905365 lte-kp is-and-acceptance

Technology