1 Metrics and Maturity Cartagena de Indias © ISM3 Consortium 2009
1
Metrics and Maturity
Cartagena de Indias© ISM3 Consortium 2009
2
Managing is achieving results with the resources available for it.
There are specific activities for management that we will call
“Management Practices”.
Management
3
Testing: Assessment of whether process outputs are as expected when test data is put in.
Management Practices
4
Monitoring: Checking whether the outputs of the process and the
resources used are within normal range.
Management Practices
5
Improving: Making changes in the process to make it more suitable for the purpose, or to reduce usage of resources.
Management Practices
6
Planning. Organizing and forecasting the amount, assignment and milestones of tasks, resources, budget, deliverables and performance of a process.
Management Practices
7
Assessment. How well the process matches the organization's needs and compliance goals.
Management Practices
8
Audit. Whether the process inputs, activities and results match their documentation.
Management Practices
9
Certify: Whether the process inputs, process documentation, activities and results comply with a pre-defined standard, law or regulation.
Management Practices
10
Benefits realization: Show how achieving security
objectives contributes to achieving business
objectives.
Management Practices
11
The more sophisticated your management practices, the higher your capability.
Management and Capability
12
Therefore, there is a strong link between the metrics used and capability.
Management
You can perform few management practices without metrics.
13
Types of Process Metrics
A quantitative measurement that can be interpreted in the context of a series of previous or equivalent measurements
It is possible to audit the capability of a process checking the metrics used to manage it.
14
Types of Process Metrics
Activity: Number of outputs produced and their mean age.
15
Types of Process Metrics
Scope: Percentage of all inputs producers covered by this process.
16
Types of Process Metrics
Unavailability: Number, frequency and duration of interruptions in the normal operation of the process.
17
Types of Process Metrics
Effectiveness: Number, mean time between inputs and percentage of Inputs that produce an Output.
18
Types of Process Metrics
Efficiency: Ratio between the number of outputs submitted and the available resources for this process in actual use.
19
Types of Process Metrics Load:
Percentage of resources reserved for the process in actual use.
20
Types of Process Metrics
Quality: Measure of the fitness for purpose of the outputs.
21
Description of what is measuredHow is the metric measuredHow often is the measurement takenHow are the thresholds calculatedCurrent range of values considered
normal for the metricBest possible value of the metricUnits of measurement
Metrics Specification
22
What are metrics good for?
Enable performing management practices. Determine whether security objectives are met (test
success); Show how security objectives contribute to business
objectives; Measure how changes in a process improve (or not) the
ISM system; Inform decisions to fix or improve the ISM processes.
23
What are metrics good for?
Detect significant anomalies (tell normal from abnormal, saving investigation efforts);
Diagnosis Business Decision
Fault in Plan-Do-Check-Act cycle leading to repetitive failures in a process
Fix the process
Weakness resulting from lack of transparency, partitioning, supervision, rotation or separation of responsibilities (TPSRSR)
Fix the assignment of responsibilities
Technology failure to perform as expected.
Change / adapt technology.
Inadequate resources . Increase resources or adjust security targets.
Security target too high. Revise the security target if the effect on the business would be acceptable.
Incompetence, dereliction of duty. Take disciplinary action.
Inadequate training. Institute immediate and/or long-term training of personnel
24
Security Investment, Maturity Level & Risk
Security Investment
Risk
Risk Reduction/Additional SecurityInvestment
ISM3 Maturity Levels
(Qualitative Graphic. Risk Reduction / Extra Security Investment, scaled x40 for readability)
25
ISM3 Maturity Levels (examples)
ISM3 Basic Level - Significant risk reduction from technical threats, for a minimum investment in essential ISM processes.
For organizations with low Information Security Targets in low risk environments.
ISM3 SMEs Level - Highest risk reduction from technical threats, for a significant investment in Information Security processes.
For organizations with high Information Security Targets in normal or high-risk environments.
ISM3 Military Level - Highest risk reduction from technical and internal threats, for a high and optimized investment in Information Security processes.
For organizations affected by specific requirements (such as utilities, and financial institutions) with high Information Security Targets in normal or high-risk environments.
26
3 – Definición Objetiva de Madurez
Inde
finid
o
Defin
ido
Ge
stio
nad
o
Con
trola
do
Op
timiz
ado
Prácticasde Gestión D
ocu
men
taci
ón
Act
ivid
ad
Alc
ance
Dis
po
nib
ilida
d
Efic
aci
a
Carg
a
Cob
ert
ura
Calid
ad
Efic
ienci
a
OptimizaciónEvaluaciónMejora de CalidadPlanificaciónRacionalizaciónMonitorizaciónPruebasCertificaciónAuditoria
27
ISM3 Capability Levels
CapabilityLevel
Metrics Requirements Enabled Managed Practices
Basic Documentation Audit and Certify.
Defined Basic, plus Activity, Scope, Unavailability and Effectiveness
Basic, plus Test
Managed Defined, plus Load Defined, plus Monitor, Benefits Realization, Planning and removing weaknesses before they produce incidents, and getting feedback on the result of changes.
Controlled Managed, plus Quality Managed, plus Assessment and removing bottlenecks that hamper performance.
Optimized Controlled, plus Efficiency Controlled, plus finding points of diminishing return and making trade-offs.
28
Learn to implement High Performance Security Management Processes http://cli.gs/ism3
Web www.inovement.esVideo Blog youtube.com/user/vaceitunoBlog ism3.comTwitter twitter.com/vaceitunoPresentationsslideshare.net/vaceituno/presentations
Articles slideshare.net/vaceituno/documents
29
THANK YOU