Krzysztof pulkiewicz kpi

How to measure your business resiliency Define the KPI’s/KRI’s and scorecards to control your security and business continuity capabilities?

Krzysztof Pulkiewicz | BCMLogic

Platforma Zarządzania Ciągłością Działania BCMLogic | www.bcmlogic.com

Abstract

Business Continuity Management is the process, not just a one-time

project activity. In order to control the alignment between the BC plans

and business as usual as well as synchronize the changes, it is required to

setup the scorecard based measurement process.

The set of KPI's and KRI's is aimed to visualize the maturity of BCM, risk vs.

lost metrics and level of protection mechanisms against the business

requirement.

I will present the business resiliency scorecard framework with special

focus on the methods of data gathering and integration with IT

infrastructure landscape.


Agenda

Why to measure?

What to measure?

How to gather data?

How to present results?

How to do it in practical way?

Key takeaways


KPI/KRI fundamentals

Key performance indicator(KPI) is a measure of performance, commonly used to help an organization define and evaluate how successful it is, typically in terms of making progress towards its long-term organizational goals.

Key Risk Indicators Measures are used to indicate how risky an activity is. KRI give us an early warning to identify potential event that may harm continuity of the process

KPI primer

From row data to metrics

• Details raw information

• Metrics are refined data

• KPIs are metrics with business-context

• Business context makes security relevant.

A Key Performance Indicator

• Must be something that can be measured and continued to be measured

• Must be precise, meaningful and understandable

• Must be relevant to the business

• May be required by legislation and/or Regulations

• Must have a measurement index that has meaning

• Should be tied to the organization’s vision and strategy


• You can control only the things you really cannot measure

• To understand the overall readiness level of your company

• To justify investment

• To plan and assess the risk based on the statistic and past experience

• Executives love metrics and dashboards. Always time-constrained, they ask for

metrics that can be reviewed at a glance

• Money talks - especially when you speak with your CFO

• C-level managers are used to percept from KPI’s- give it to them

• Justify your security investments based on the measurable objectives

• The KPIs can be used to help comply with legislative or regulatory requirements

Why to measure?


What to measure?

Recoverability Planning

Compliance

Technology

BC project

Can our organization be

recovered within our

tolerance for downtime?

Status and results of planning activities

Regulatory and audit point of view

Project based reporting

• Incidents statistic

• How did we react?

• BIA overview

• Risk assessment

results

• Processes covered

by BCP

• Minimum

operational teams

• BCM Maturity

• Compliancy level

• Risk assessment

results

• IT infrastructure

• IT services SLA

• Service Desk

• Business RTO/RPO vs. technology capabilities

• Milestones

• Financial spent • FTE effort


BIA overview

Tactical view on the BIA requirements

• # of Department- Business Impact Analyses vs. Total Expected • # of Department- Table Top Exercises Completed vs. Total Expected • # of Supplier Business Continuity Assessments Completed vs. Total Number of “Critical” Suppliers • Ten top processes (based on criticality score) • Most critical assets • RTO/RPO distribution

Example BIA dashboard


BIA per business unit

• Financial impact over time

• Time wise

• One day stop

• Reputation impact

• Formal and Legal impact

• Number of scenarios affecting the business unit

• Required assets (MAC)

• Minimum operational team vs. total unit

headcount

• Critical processes vs. all processes handled by

unit

Tactical view on the BIA requirements defined at the level of each business unit

Tłumaczenie radar chart

Radar charts allows to visualize benchmarking

Criticality Business Process BIA Updated Plan Updated Tested

1 Call Center customer support Yes Yes Yes

2 Accounts Payable Yes No No

1 Liquidity managment No No No

Business Unit X

Criticality Business Process Recovery Objective Recovery Capability

(based on most recent test)

Gap

1 Call Center customer support4 hours 8 hours 4 hours

2 Accounts Payable 2 hours 1 hour 1 hour

1 Liquidity managment 2 days 2 days 0

Business Unit X


BIA per business process

• Prioritize the BC process list based on: – Business impact when interrupted

– Vulnerability of underlying assets

– Risk level

• Benchmark criticality among different business units/ entities

Criticality level defined for each process


Risk assessment Defined scenarios are depicted based on the probability and impact

Tłumaczenie + oryginalny wykres

Probability

Imp

act

TOLERANCE

Przykładowe scenariusze: • Chwilowa przerwa w zasilaniu • Okresowa absencja pracowników

PREVENTION

Przykładowe scenariusze: • Niedostępność budynku Centrali relokacja • Awaria IT- Disaster Recovery

ELIMINATION

Przykładowe scenariusze/rozwiązania: • Awaria zasilania/agregaty • Awaria łączy/redundantne linie telekomunikacyjne

MONITORING

Przykładowe scenariusze • Przeciążenia zasobów IT/ monitorujemy

elementy systemów

High

Hig

h

Low


Readiness level

To report the progress of BCP project:

• How many process have contingency measures

• How many scenarios are planned

• How many solutions tested

No. BCM management objective Related example KPI

1 Proper crisis situation management (incident management, start-up and implementation of the recovery tasks)

Number of reported incidents

The ratio of the risk response plans for scenarios of potential threats

2 Eliminating the potential effects of process interruption

Number of incidents that were not closed before crossing the processes RTO that are related to. Incidents can be divided into:

- incidents that concerned the processes associated with financial impact

- incidents that concerned the processes associated with reputation impact

- incidents that concerned the processes associated with law impact

3 Providing processes resume after the crisis situation

The ratio of recovery tasks completed successfully for all recovery tasks

4 Continuous development and improvement of BCM

Number of performed BCM tests

The ratio of the number of BCM plan tests completed successfully for all BCM plan tests at this time

Number of risk which probability or potential impact was reduced after implementation BCM corrective tasks


IT services management Monitor and visualize critical service conditions


Typical examples of BC KPIs that can be included in a BC Policy Document:

• Level of disruption response/recovery time

• Time to detect disruptions

• Time to trigger action to disruption events

• Time to complete recovery action

• Time to declare `business as usual'

• Level of business continuity testing/exercising/audit

• Level of service delivery and quality acknowledgement by clients?

• Level of knowledge of business continuity awareness/acceptance/culture

• Level of availability and/or knowledge of alternative fall-back to critical resources

(human/ technical/ location)

• Level of effectiveness of Service level agreements

Example KPIs


• KPIs can be used to measure the Effectiveness of Investment (EOI)

• A Return on Investment (ROI) for business continuity is difficult to measure since risk, and especially risk reduction, is challenging to quantify in terms of money

• The Effectiveness of Investment (EOI) could be the comparison of the effectiveness of the resiliency measures with the value of the investment

• Proper KPI/KRI reporting may also provide a financial institution the ability to reduce the

percentage of reserve required to offset operational risk defined by the Basel II, Solvency

Effectiveness of Investment

Cost

Risk

Equilibrium

Protection

Loss

High

Hig

h

Low


Methods

• Retrieve information from IT systems

– Data base interface (direct or ETL)

– API

• Gather information from people

– Automated forms workflow (reporting)

Sources

• Service Desk system (incidents, time to resolve)

• IT infrastructure monitoring (alerts, up/downtime, service level)

• BPM (process effectiveness)

• PMO (project reporting)

• Call Center

• People (line managers)

How to gather data?


How to visualize? The reporting mechanism must support three purposes • Highlight or alert whether business expected targets are being not met • Provide trending and an overview of performance indicators • Provide details that pinpoint which areas within each performance indicator require

actions


KPI reporting audience

IT infrastructure failures MTTR RTO /RPO DR testing Critical services incidents

IT service availability E2E process SLA Customer service level Customer service downtime

C-level

Managers

Business units

IT

Value at Risk Complikancy level BC scope and cost

Each audience may require different information and different presentation

Business continuity events

• The level of aggregation and

or abstraction required may vary considerably

• You may not want to talk about number of backup site workstations to the Chairman of the Board!

• Don’t assume that the higher the level, the simpler the presentation


• Define 10-15 (smart) KPI

• Identify the KPI stakeholders (RACI)

• Understand where information resist

• Leverage the available data to link the KPIs to other • key operational metrics that include both technology • and process metrics

• Integrate with other systems and applications

• Use existing reporting capabilities to establish periodical reporting

• You can use MS Excel or one of the specialized tools

• Share the information across the organization

• Make the KPIs actionable

How to do it in practical way?


KPI reporting mistakes

• Lack of management commitment

• Measuring too much, too soon

• Measuring too little, too late

• Measuring the wrong things

• Imprecise KPI definitions

• Using KPI data to evaluate individuals

• Using KPI to motivate, rather than to understand

• Collecting data that is not used

• Lack of communication and training


1. Define measurable objectives of the security process

2. Utilize existing data sources

3. Learn from the past

4. Align the results presentation to the audience

5. KPIs can be used to help comply with legislative or regulatory requirements

Key takeaways


Questions

Krzysztof pulkiewicz kpi

Documents

business unit financial

bc plansand business

business unit tactical

measure of performance

measurement process

risk vs

bia requirements

krzysztof pulkiewicz