InfoSec Measurement and Quantitative vs Qualitative Methods
Recorded Webinar Here: https://www3.gotomeeting.com/register/604059902
Aliado and Risk Centric Security would like to introduce you to the world of quantitative risk and decision analysis.
Our webinars will provide you with a glimpse of the power and credibility that quantitative methods can bring to the problems that Information Security Professionals face every day
Topics covered include:
What is risk? Possibility and Probability What is a measurement and what is it for? Qualitative vs. Quantitative methods Static modeling vs. Monte Carlo simulation Calibration and the power of a calibrated estimate Modeling Expert Opinion and the RCS BetaPERT calculator
A. Definitions 1. Risk 2. Risk and Opportunity 3. Possibility vs. probability 4. Measurement 5. Precision vs. accuracy 6. Qualitative vs. quantitative methods
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Measurement,Qualitative vs. Quantitative Analysis,
Heather Goodnight is an accomplished Global Sales and Business Development Consultant. Over the years, her unique, practical insight into problems of risk and opportunity have provided important guidance for organizations both large and small. She is a cofounder of Risk Centric Security and currently serves as President of the Corporation.
Patrick Florer has worked in information technology for 30 years. In addition, he worked a parallel track in medical outcomes research, analysis, and the creation of evidence-based guidelines for medical treatment. His roles have included IT operations, programming, and systems analysis. From 1986 until now, he has worked as an independent consultant, helping customers with strategic development, analytics, risk analysis, and decision analysis. He is a cofounder of Risk Centric Security and currently serves as Chief Technology Officer.
Houston, we have a problem …
When speaking with our customers, we recognized:
Information Security Professionals are comfortable speaking the technical language of firewalls, logs, threats, vulnerabilities, and exploits.
Business managers are comfortable speaking the language of return on investment, discounted cash flows, and risk as financial impact.
Mutual misunderstanding can occur, and it is often a source of frustration for everyone.
By learning to speak about risk in business terms, Information Security Professionals can reach out and bridge the language gap.
The technical details of sql injection attacks may be important to you, but your business counterparts may not understand, and they usually don’t care.
Instead of talking about threats, vulnerabilities, and controls, talk about risk in terms of financial impact. Tell the business people what a sql injection attack could cost.
They will understand that!
(They may not believe you, but they will understand what you are saying!)
RiskRisk and OpportunityPossibility vs. probabilityMeasurementPrecision vs. accuracyQualitative vs. quantitative methodsThe “not enough data” syndromeMonte Carlo simulationModeling expert opinion and the PERT distribution
The possibility of suffering harm or loss; danger.A factor, thing, element, or course involving uncertain
danger; a hazard.The danger or probability of loss to an insurer.The amount that an insurance company stands to lose.The variability of returns from an investment.The chance of nonpayment of a debt.
NOTE 1 An effect is a deviation from the expected —positive and/or negative.
NOTE 2 Objectives can have different aspects (such as financial, health and safety, and environmental goals) and can apply at different levels (such as strategic, organization-wide, project, product and process).
NOTE 3 Risk is often characterized by reference to potential events (3.5.1.3) and consequences (3.6.1.3), or a combination of these.
NOTE 4 Risk is often expressed in terms of a combination of the consequences of an event (including changes in circumstances) and the associated likelihood (3.6.1.1) of occurrence. NOTE 5 Uncertainty is the state, even partial, of deficiency of information related to, understanding or knowledge of, an event, its consequence, or likelihood
In the USA, NIST, Special Publication 800-30 describes risk in the following way:
Risk is:
“the net mission impact considering the probability that a particular threat-source will exercise (accidentally trigger or intentionally exploit) a particular information system vulnerability, and the resulting impact if this should occur.”
NIST (The National Institute of Standards and Technology), provides an additional definition of risk in Special Publication 800-39:
Risk
A measure of the extent to which an entity is threatened by a potential circumstance or event, and typically a function of: (i) the adverse impacts that would arise if the circumstance or event occurs; and (ii) the likelihood of occurrence. Information system-related security risks are those risks that arise from the loss of confidentiality, integrity, or availability of information or information systems and reflect the potential adverse impacts to organizational operations (including mission, functions, image, or reputation), organizational assets, individuals, other organizations, and the Nation.
NIST, The National Institute of Standards and Technology, Special Publication 800-39, Appendix B, Page B-7.
A possibility is something that is “capable of happening, existing, or being true without contradicting proven facts, laws, or circumstances known to be true. *”
A probability is "the likelihood that a given event will occur.”*
In statistics, a probability is “a number expressing the likelihood that a specific event will occur, expressed as the ratio of the number of actual occurrences to the number of possible occurrences.“
Probability is calculated after tossing the coin many times.
Probability is always a number between 0 and 1, sometimes expressed as:
How can we use this in information security risk analysis?
The fact that something can happen (possibility) doesn't tell us how likely it is to happen (probability), or how much impact it might have if it does happen (probability).
Estimating these values helps us prioritize our activities in a rational way.
Precision: a machine can produce the same part to within 1/1000th mm all day long. This is no guarantee that the part is the correct length, however. Accuracy: a machine can produce the same part to within +/- 2/1000th mm of the correct length. Although some parts are a bit shorter and some are a bit longer, every part is within spec.
Benefits of qualitative methods? They are useful in certain scenarios, and can be quick and good enough.
Problems with qualitative methods?Variability between assessorsInconsistency of a single assessorArithmetic and statistical operations not possibleProblems near the boundaries of categoriesLoss of information
Minimum: What is the least or lowest (best or worst) numerical estimate that you believe to be reasonable? This will be the smallest number you come up with.
Most Likely:What is the most likely or most probable numerical estimate in your opinion? This number must fall between the minimum and maximum. It may equal either the minimum or the maximum, but should not equal both
What is the greatest or highest (best or worst) numerical estimate that you believe to be reasonable? Note that “best” or “worst” case estimates could be either minimum or maximum values, depending upon the scenario.
In a risk / loss exposure scenario, lower is better, so the minimum represents the lowest loss, or best outcome. The maximum represents the highest loss, or worst outcome.
In a sales or opportunity scenario, it’s the reverse: lower is not better, so the minimum represents the worst case. Higher is better, so the maximum represents the best case.
On a scale that includes “Very Low”, “Low”, “Average”, “High”, and “Very High”, how confident are you in the accuracy of your estimates?
This parameter controls the sampling around the most likely value, and thereby also controls the height of the histogram or slope of the cumulative plot.
For most analyses, using “Average” for the confidence parameter works well. In this instance, “Average” really means having no strong feeling about the matter – being evenly divided between under-confidence and over-confidence.
1% of values are <= 10,044 and 99% are > 10,04410% of values are <= 11,120 and 90% are > 11,12020% of values are <= 11,658 and 80% are > 11,65850% of values are <= 13,025 and 50% are > 13,025
The 50th percentile has another name - it’s called the Median. The Median is the mid-point in a list of values - half of the values in the list are less and half are greater than the Median.