V&V of models - ppt

8/3/2019 V&V of models - ppt

1/18

IEEE Paper by:- Robert G. Sargent

Presented by:-

Pranav Jain..106Neil Mahaseth..108


2/18

Introduction

Concern of rightness of results

Model Validationy Computer program

y Implementation

Model Verificationy Computerized model

y Domain & Accuracy

Model Credibilityy Develop confidence

Model Accreditation

Cost & Time


3/18

Basic Approaches

Development team makes decision onvalidityDevelopment team

When development team is small

Users of model involved with its developersDev. Team with

Users

Third party responsible

Conduct concurrently or afterwards

Independent

Validation &Verification

Determine scores subjectively for categories

Valid if, overall score > passing some scoreScoring Models


4/18

Paradigms

Relate Verification and Validation to

Development process

2 common ways:y Simple

y Complex


5/18

Simplified View

Problem Entity

(system)

Computerized

Model

Conceptual

Model

Data

Validity

Experimentation Analysis &

Modeling

Computer Prog &

implementation

Computerized Model

Verification

Conceptual Model

ValidationOperational

Validation


6/18

Complex View

Real World and Simulation World Relationships with Verification and Validation


7/18

Validation Techniques

AnimationComparison

to othermodels

DegenerateTests

EventValidity

ExtremeCondition

testsFace Validity

HistoricalData

HistoricalMethods

Internal

Validation

Multistage

Validation

Operational

Graphics

Parameter

Variability

PredictiveValidation

Traces Turing Test


8/18

Data Validity

Not considered a part of model validation

Need for data:-

-Building conceptual model

-Validating model-Using the model

Concerns for data:-

-Appropriate-Accurate

-Sufficient


9/18

Conceptual Model Validation

Theories and Assumption

Reasonable representation

Uses the following techniques:-

-Face validation

-Traces-Flowchart

-Graphical model


10/18

Computerized Model Verification

Simulation Languages

Higher Level Programming Languages

Static Testing-Structured walkthroughs

-Correctness proofs

Dynamic Testing

-Input-Output validation

-Internal consistency checks

-Reprogramming


11/18

Operational Validity

Observable Systems

Non-Observable Systems


12/18

Comparisons ofOutput Behavior

Comparison of Simulation Output with

System output or Other Model

Three Methods :-

-Graphical Comparison of Data

-Confidence Interval

-Hypothesis Testing

Use of Graphs most common


13/18

Graphical Comparison ofData

Histograms

Scatter Plots

Box Plots

Used for Making

Subjective

Judgment

Turning Test Face Validity


14/18

Graphical Comparison ofData


15/18

Hypothesis Testing

Acceptable range of accuracy

Two types of error

-Type I (Model Builder's Risk)-Type II(Model Users Risk)

The amount of agreement

The probability of acceptance of a modelbeing valid Pa


16/18

Hypothesis Testing


17/18

Documentation

Convincing Users of correctness of a

model and its results.

Detailed And Summarized


18/18

Recommended Procedures

Agreement be made prior to modeling

Amount of accuracy required

Tests Assumption and Theory

Face Validity on the conceptual model

Explore the simulation models behavior

In the last model iteration make

comparisons Develop validation documentation

Schedule for periodic review

V&V of models - ppt

Documents