SAP Metrics that Matter Stephen Missett

SAP METR

ICS T

HAT

MATTE

R

MEA

SU

RIN

G A

SA

P C

HA

NG

E CO

NTR

OL I

MPLEM

ENTAT IO

N

DES IGNED BY:STEPHEN MISSETT

CONTENTS

1 First Things First

2 Basic Points

3 First 100 days

4 Pilot group ROI

5 ROI Data and SAP Change Control

6 Go Live

FIRST THINGS FIRSTWe still see “big bang” rollouts in simpler environments but phased implementations are increasingly preferred. There are various approaches to phased deployments. A geographically segmented approach rolls out to individual regions or countries. Deployment by business unit or by increasingly business-critical landscape might make sense, depending on your organisation.

With phased deployments, you can launch a combined program of user education and system remediation. If you run a pilot program first, the pilot team members can become invaluable “initial experts” since they’ll be familiar with the planned implementation and can work with your users.

BASIC POINTS

Here are some basic points to bear in mind during implementation:

• Have you monitored ROI data points before, during and after implementation?

• Have you used pilot or early implementation data to “tune” for better ROI?

• Have you used ROI data points to help continually tune the implementation during a phased deployment?

• Have you structured your data so it can serve double-duty with SLAs and KPIs (Key Performance Indicators) when implementation is complete?

Keep the above points in mind and you’ll finish the implementation with solid ROI data points, hard data that will keep growing as your ROI increases over time.

FIRST 100 DAYS INITIAL IMPLEMENTATION AND TRAININGThe first 100 days are an intuitive first assessment time. Use them to make your case. Tangible measurements we have seen at different organisations during implementations can often be phrased as questions. If your answer to any of these is yes, estimate a count or other metric to use for an ROI baseline:

• Have do-overs and back-outs increased with the complexity of your landscapes? Estimate an average incident count per day, week or month. If you can estimate time lost per incident, include that figure with the incident count

• Have bottlenecks, such as approvals, delayed tasks? By how long?

• Do you need to bring down average time to project completion? Has a high volume of parallel object development caused conflicts during migration? Estimate an incident count

• Do rekeying activities chew up developer resources or delay project release dates? Estimate man days and document delays

• Is there an unacceptable post go-live ‘warranty’ period during which errors and fixes seem to take more time than they should?

• Do you hope to see fewer or shorter consultant engagements? Generate an average pre-implementation engagement length

• Has down-time increased due to system failures, cutover times, etc.? Estimate incident counts or loss of productive time in affected systems

• Do you need better business-side support, measured as quicker responses, faster fixes, and fewer problems? Gather pre-installation figures for response times, problem counts

These are just examples. The point is, if you can measure, count, or simply estimate a baseline value that applies to your organisation, you can use it later to assess initial implementation ROIThe result will be to document how implementation outcomes matched up to expectations. You can establish that you have satisfied requirements and can identify where further action may be needed

PILOT GROUP VALIDATION ROI AND THE SAP CHANGE CONTROL PROCESS

Pilots (or POCs) let you test the waters. By putting changes through QA before moving them into production

Any big project gets better results if you run a pilot before committing to an overall roll-out

Pilot programs can validate strategies and give early feedback on emergency processes, documentation, security, migration, approvals, special object management, validation rules, and object collision management issues (among others)

Not every change control solution allows for piloting, so make sure you don’t get locked into a “big bang” style, all-or-nothing rollout

Here are a few points to bear in mind as you plan how to use the pilot test to help measure project success:

Who are your internal change-team champions? Have you established a team of “initial experts” to assist in the final SAP change control solution rollout? What additional data points can they suggest for ROI metrics? What is their feedback on potential trouble points when scaling up to the full enterprise implementation? Do potential trouble points reflect pre-implementation problems? Is there a way to use trouble points as “before data” during later ROI measurement? Look for ways to record incident counts,

total or per-incident time lost, or make other valuations that you can use later to establish project ROI

ROI DATA AND SAP CHANGE CONTROL CONFIGURATION EFFECTS

Hard data based on actual, measurable variables in your system will provide metrics that cannot be easily challenged or dismissed. But when you look a little closer, those metrics might not measure the intangible benefits you thought they did. System reliability, user satisfaction, smoother business processes - things like that are intangibles, so there can be surprises

Take a close look before you draw conclusions about your project’s ROI. Does your dependable ROI data mean what you think?

How you configure your implementation can leave room for a lot of “gotcha!” factors:

Look closely for disconnects and accidental effects that might distort your findings by changing the intangibles you’re actually looking at

Look for recent system configuration changes that might affect your data. For example, credit policy tightening might, in retail, lead you to think your project’s faster processes led to fewer transaction failures when actually more credit was declined, resulting in fewer transactions

Have input forms changed and raised barriers to user interaction? You could draw mistaken conclusions about why user complaints dropped off

Review all configuration changes between your initial baseline data and your first data collection, typically about 100 days after implementation – look for such accidental effects

ROI DATA AND SAP CHANGE CONTROL CONFIGURATION EFFECTS ( CONTIN..)

If you take into account any possible configuration side effects that could distort your conclusions regarding ROI, you won’t find yourself trying to defend an error, or making wrong assumptions about your project’s performance. It is far better to know the truth and to work with reality than to suddenly find yourself out of touch with your own IT environments and wondering how it happened

GO-LIVE AND PRODUCTIVE USE At this point the set of hard data points you established for your ROI baseline can contribute to monitoring your system’s KPIs. In terms of ROI, using this data in a larger set of system KPIs will be doubly useful as it ensures your ROI measurement won’t slip

Some examples of possible ROI data that can become KPI data after implementation include:

Number of Severity 1 emergency changes in a given period before and after implementation Mean time to migrate normal changes from development to production Mean time to migrate emergency changes from development to production, number of

emergency incidents Time by required resource level – senior Basis, SME team members QA testing team member, release management team, etc. Taking into account the additional cost of higher staff levels required before implementation

compared to post installation

Your ROI data will feed into KPIs and SLAs, enabling you to monitor and measure whole-system performance over a period of time. Following standard ALM methodology for continuous process improvement, measure/monitor your change control processes. Utilise your pre-defined KPIs, analyse the results, make adjustments to improve performance, and repeat.

IN CONCLUSION

The flexibility of running SAP “like a factory” will let you stay nimble and responsive to day-to-day business requests that are ultimately driven by competitive market forces far outside your control

The data will demonstrate that your SAP change control strategy can evolve with the business as it adapts, changes, and moves into the future

Thank you