Top Banner

Click here to load reader

RSK80016 - Phil McKenzie (6900593) - Research Oral Presentation

Jan 24, 2017

ReportDownload

Documents

  • Analytical Variance in Qualitative

    Bowtie Risk Analysis

    Phillip McKenzie

    Research oral presentation submitted in fulfilment of the

    Degree of Master of Risk Management by Research

    1

  • Research Part 1

  • Research problem

    Research objectives

    What is bowtie analysis

    What is analytical variance

    Analytical variance sources and types

    System based model of analytical variance

    3

    Research part 1 agenda

  • 4

    Companies routinely apply risk assessment tools

    and methodologies in their risk management

    systems. One methodology that is growing in use

    is qualitative bowtie analysis.

    It has been observed that qualitative bowtie

    analysis often produces inconsistent analytical

    results (analytical variance).

    This is concerning as it calls into question the

    reliability and validity of the methodology.

    Research problem

  • Qualitative bowtie analysisISO 2000

    5

  • Quantitative bowtie analysisISO 2000

    6

  • Analytical

    Process

    Analytical

    Subject

    Analytical

    Results

    Simplified model of the analytical process7

    The research premise is that given a common

    analytical subject; and a common analytical

    process; you would expect consistent analytical

    results across multiple analyses.

    However, in practice the analytical results appear to

    be highly at variance from each other.

  • Research objectives8

    Objective 1:

    To identify and describe the

    antecedent factors inherent in the

    qualitative bowtie analysis process

    which cause the observed analytical

    variance.

  • Literature review

    A literature review was performed on qualitative

    bowtie analysis and analytical variance.

    This explored the factors occurring throughout the

    risk analysis process which may be sources of the

    observed analytical variance.

    The findings of the literature review has produced a

    typology of analytical variance which demonstrates

    the sources of analytical variance and the related

    types of variance factors.

    9

  • No. Analytical Element Common Analytical Element Synonyms

    1 Hazard threat energy

    2 Top Event hazardous event

    3 Causes mechanisms, threats

    4 Outcomes consequences

    5 Controls barriers, safeguards, defences, mitigations

    6 Defeating factors escalation factors, preconditions, active failures

    7 Defeating factor controls escalation factor controls

    10

    Typical qualitative bowtie analysis in practice

  • 11

    Evolution of accident modelsHollnagel & Goteman 2004

    Model type Search principle Analysis goals Example

    SequentialSpecific causes and well-

    defined links

    Eliminate or contain

    causes

    Linear chain of events

    domino, Trees / networks

    EpidemiologicalCarriers, barriers, and

    latent conditions

    Make defences and

    barriers stronger

    Latent conditions,

    Carrier-barriers,

    Pathological systems

    SystemicTight couplings and

    complex interactions

    Monitor and control

    performance variability

    Control theory models,

    Chaos models,

    Stochastic resonance

    Qualitative bowtie analysis employs an

    epidemiological modelling approach. This

    approach is associated with high information

    complexity arising from barrier control analysis.

  • 12

    Analytical variance

    Variance: the fact or quality of being different,

    divergent, or inconsistent Oxford University Press 2014

    Variance is the actual state of difference between

    two or more things. The term analytical variance

    therefore refers to the inconsistent results of

    multiple comparative analyses.

    In the case of this research, analytical variance is

    the inconsistency observed in the analytical results

    of qualitative bowtie analysis.

  • Omissions of relevant analytical elements

    Inclusions of irrelevant analytical elements

    Differences in characterisations of the same analytical elements

    Differences in classifications of the same analytical elements

    Differences in relationships between the analytical elements

    13

    Observed analytical variance manifestation in

    qualitative bowtie analysis

    cause

    outcome

    top

    event

    hazard

    defeating

    factor

    control

  • 14

    Variance typologies ANS and IEEE 1983; Ferson & Ginzburg 1996; Regan, Colyvan & Burgman 2002; Carey &

    Burgman 2008; Markowski, Mannan & Bigoszewska 2009; Ferdous et al. 2012; Shahriar, Sadiq &

    Tesfamariam 2012; Ferdous et al. 2013

    Analytical variance is discussed in the literature as

    resulting from either uncertainty or variability; with

    uncertainty being the most prevalent term used.

    A review of the literature on the concepts of

    uncertainty and variability typologies within the

    domain of risk analysis was undertaken.

    This revealed a very wide spectrum of typologies

    which use divergent terminology and describe

    many different types of uncertainty and variability.

  • Variability(Aleatory Uncertainty)

    Analytical Subject Analytical Methodology Human Analysts

    Data (parameter) uncertainty

    Amount of data, Diversity of data

    sources, Accuracy of data sources

    Completeness uncertainty

    List of initiating events, system failure

    contributors, accident sequence,

    definition of system damage states, list

    of system interactions, accounting of

    human factors

    Model uncertainty

    Limitations of binary logic models

    Model uncertainty

    Skill and accuracy of analyst,

    Misapplication of method rules

    ANS and IEEE 1983

    Variability (objective uncertainty)

    Heterogeneity, stochasticity

    Ignorance (epistemic uncertainty)

    Systematic measurement error,

    incomplete information

    Ferson & Ginzburg

    1996

    Epistemic uncertainty

    Measurement error, Systematic error,

    Natural variation, Inherent randomness

    Epistemic uncertainty

    Model uncertainty

    Linguistic uncertainty

    Vagueness, Context dependence,

    Ambiguity, Underspecificity,

    Indeterminacy of theoretical terms

    Epistemic uncertainty

    Subjective judgement

    Regan, Colyvan &

    Burgman 2002

    Variability

    Naturally occurring, unpredictable

    change

    Incertitude

    Lack of model parameter knowledge,

    Lack of model relationship knowledge

    Linguistic uncertainty

    Ambiguity, Vagueness, Underspecificity,

    Context dependenceCarey & Burgman

    2008

    Objective uncertainty

    Variability, Random behaviour

    Subjective uncertainty

    Lack of knowledge

    Parameter uncertainty

    Imprecise data, Vague data, Missing

    data, Inadequate data

    Completeness uncertainty

    Have all significant phenomena and

    relationships been considered

    Modelling uncertainty

    Inadequacies and deficiencies in

    formulation of accident scenario

    structure

    Subjective uncertainty

    Vagueness in interpretation

    Markowski, Mannan &

    Bigoszewska 2009

    Aleatory uncertainty (variation)

    Stochastic, Objective, Irreducible,

    Random

    Epistemic uncertainty (knowledge)

    Imprecise, Incomplete, Ambiguous,

    Ignorance, Inconsistent, Vague

    Ferdous et al. 2012

    Data uncertainty (epistemic)

    Impreciseness, Vagueness, Lack of

    knowledge, Incompleteness

    Model uncertainty

    Interdependency of event relationshipsShahriar, Sadiq &

    Tesfamariam 2012

    Aleatory uncertainty

    Natural variation, Random behaviour of

    a system

    Epistemic uncertainty

    Lack of knowledge, Incompleteness

    Data uncertainty

    Incomplete, Inconsistent or imprecise

    data, Missing or unavailable data, Multi-

    source data, Vagueness or inadequacy

    in input data

    Quality uncertainty

    Knowledge deficiency about a system

    Model uncertainty

    Model adequacy, Mathematical and

    numerical approximations in the model,

    Assumptions or validation of the model

    Quality uncertainty

    Error in hazard identification,

    Incorrectness in identification of

    consequences and their interactions

    Ferdous et al. 2013

    Uncertainty(Epistemic Uncertainty)

    Variability(Aleatory Uncertainty)

    Analytical Subject Analytical Methodology Human Analysts

    Data (parameter) uncertainty

    Amount of data, Diversity of data

    sources, Accuracy of data sources

    Completeness uncertainty

    List of initiating events, system failure

    contributors, accident sequence,

    definition of system damage states, list

    of system interactions, accounting of

    human factors

    Model uncertainty

    Limitations of binary logic models

    Model uncertainty

    Skill and accuracy of analyst,

    Misapplication of method rules

    ANS and IEEE 1983

    Variability (objective uncertainty)

    Heterogeneity, stochasticity

    Ignorance (epistemic uncertainty)

    Systematic measurement error,

    incomplete information

    Ferson & Ginzburg

    1996

    Epistemic uncertainty

    Measurement error, Systematic error,

    Natural variation, Inherent randomness

    Epistemic uncertainty

    Model uncertainty

    L