CassBeth Inc. Specification Analysis Tool (SAT) Overview October 2005.

CassBeth Inc. www.cassbeth.com 1

Specification Analysis Tool(SAT) Overview

www.cassbeth.com/sat

October 2005


Importance of Good Req’s

• Make Sure You Design and Implement What You Thought You Needed– Req’s Driven Development

• Show That You Actually Implemented What You Needed– This May Include Certifications

• ROI Justifications Go Back 25+ Years


Problem Statement

How Can you Create a Clean Useful Specification?

Buzz Words are:

Clear, Complete, Consistent, Testable, etc


Different Views on Specs

• A NASA View– Automated Req’s Measurement (ARM) tool

• An FAA View– Prepared by the Plain English Network (PEN)

(www.plainlanguage.gov)

– Adapted for FAA, February 2000 (http://www.faa.gov/language/)

– Edited for AUA Plain Language Month, April 2003

• Walt's View


Observations

• No one approach is better or worse– Optimized for each organization

• An approach is appropriate if it works– Only organization can make determination

• Appears to be need for User defined analysis rules – Can allow each approach to be optimized


Levels of Consciousness

surface key req’s that terrify all

No Politics - Truth

did you get all leavesCompleteness

eliminate similar, duplicates, conflicts

Consistency

source, sink, nouns, action verbs

Clear Complete Testable Imperatives

shall, must, will, should ...Consistent

Imperative Use


Alternatives

• Do Nothing• Use Manual Techniques• Create Your Own Tool• Use Other “Specification Analysis” Tools

– QuARS, ARM (NASA), TIGER

• Use SRDB (DOORS)– Filtering and View Mechanisms

• Use SAT with SRDB Parent Child Reports


What Does SAT Do?

• Performs automated specification analysis– Very fast but Humans make final decisions

• Looks for bad requirement text– Uses rules spread throughout your organization

• Looks for complex spec problems– Missing capabilities & key requirements

• Makes recommendation for each problem– Shows newly minted staff spec creation

• Gathers Metrics


Where Does SAT Fit

AnalysisIdentify

Key Req’sPrelim

Spec DocExpand

Key Req’s

DOORS Import

Clean Spec Doc

Req’s Clean Up

DOORS Parent Child

Reports

Req’s Clean Up

Final Spec Doc


SAT Services• Predefined Services

– Requirement Text Assessment (RTA)– Generic Structure Analysis (GSA)– Domain Structure Analysis (DSA)– Generic Capabilities Analysis (GCM)– Domain Capabilities Analysis (DCA)– Key Requirements Analysis (KRA)

• User Defined Services

• Metrics


SAT Operations

SAT LeadStake Holders

Updated Specs

Reports

Official Reports

1.

2.


SAT Portal - Demo


SAT Portal - Demo


How Does SAT Work

• User Uploads File– From Local Computer or Network

• SAT Examines Extension– If .doc SAT Attempts to Parse Into Objects

• Looks for PUI– If None Matches PUI MASK, creates PUI

• Converts Each Line Into Object– DOORS Excel Export

– MsWord Saved as .txt


How Does SAT Work

• Subjects Each Object To Each SAT Rule– Based on Display Filter Setting

– Tabulates Metrics For Each Rule

• Subjects Each Object to Duplicate Check– If Enabled

• Outputs On-The-Fly Object Results• Outputs Metrics at End of Last Object• User Modifies Display Filters and Rules


SAT Control Panel


SAT Control Panel


RTA Rule Templates• SAT Rules

– Untestable, Unsure, Vague 1, Vague 2, Vague 3, Vague 4, Compound Req, Internal Reference, Not Standalone, Fragment, Declarative, Duplicates

• CMSEI Rules– vague, Subjective, Choice or Option, Implicit Expressions,

Weakness, Under Specification, Multiplicity, Duplicates

• INCOSE Rules– Standard Constructs, Words to Avoid - Vague, Completeness,

Duplicates

• NASA Rules– Completeness, Continuances, Directives, Options, Weak Phrases

CMSEI, INCOSE, NASA as interpreted by SAT staff


SAT Report


SAT Report


CMSEI Report


CMSEI Report


CMSEI Report


INCOSE Report


INCOSE Report


INCOSE Report


NASA Report


NASA Report


NASA Report


SAT Report All Services


SAT Report All Services


Find Duplicates


Find Duplicates


Generic Structure Analysis


Domain Structure Analysis


Generic Capabilities Analysis


Generic Capabilities Analysis


Domain Capabilities Analysis


Key Reqs Analysis


Key Reqs Analysis


Key Reqs Analysis


Other Domains


Other Domains


Other Domains


Other Domains


Other Domains


SAT Big Picture

3. TEMPLATES

1. RULES

2. SERVICES

5. DOMAINS

COMMUNICATIONS

DEFENCE

ATC

6. PRECISE LANGUAGE INDUSTRIES

ENGINEERING

GOV REGULATION

LEGAL4. APPLICATION

.


SAT Features

• Its Simple and Fast– Automates Tedious Tasks With Greatest Benefit

• Dynamic Display Filters – Change with User Defined Rules

• Dynamic Metrics– Change with User Defined Rules

• Report Format Contains Everything– Offending Object Text

– Analysis Findings and Recommendations

– Metrics


SAT Features

• Ubiquitous– Uses Internet Technologies

• Highly Scalable– Uses Internet Technologies

• User Defined Rules– Allows Build Up of Templates

• Its New– You Can Influence Its Engine, Services, Rules, etc


Why is SAT Different

• User defined rules with template examples– Allows users to grow their own

• Structured in terms of services that span simple to highly abstract consciousness– User decides what level makes sense for them

• Realizes that simple services based on keyword searches remove significant “distractions / noise” from problem set– So hard stuff can begin to be addressed


What I Would Like To See Happen

• Community Buys SAT

• Community Creates and Publishes Rules

• Community Suggests Engine Attributes

• Community Suggests Engine Features

• Other Domains Adopt SAT– Legal (e.g. contracts, laws, constitutions)


Where Can I Get A Web Server

• Window Professional Versions– 2000 and XP professional

• Apache– www.apache.org

• IndigoPerl– www.indigostar.com

• Your Computer Network– Ask Your Computer Support Staff

• SAT May Come Bundled with Web Server– Depends on Early Marketing Findings


Company Overview

• CassBeth established in August 1997• Early Goals Were to Study Internet Technologies

• CassBeth Incorporated in 2000• Large Traffic at E-Commerce Sites

• SAT is a CassBeth Product• Direct Result of Cassbeth web Mining Solutions


Links / References• www.cassbeth.com/sat

• satc.gsfc.nasa.gov/metrics

• www.seecforum.unisa.edu/SEECTools.html#tiger

• www.indigostar.com

• www.apache.org

• www.plainlanguage.gov

• www.faa.gov/language/

• www.pupress.princeton.edu/titles/7929.html

• www.archive.org

• www.incose.org


References

Bob Ferguson, INCOSE Presentation - Automated Natural Language Analysis of Requirements, Carnegie Mellon University, Software Engineering Institute, June 21 2005.

Professor Harry G. Frankfurt, On Bullshit, Princeton University, ISBN: 0-691-12294-6

F. Fabbrini, M. Fusani, S. Gnesi, G. Lami, An Automatic Quality Evaluation for Natural Language Requirements, Istuto di’Elaborazione dell’Informazione del C.N.R. - Pisa Italy

Pradip Kar, Michelle Bailey, Characteristics of Good Requirements, 1996 INCOSE Symposium

William M. Wilson, Linda H. Rosenberg, Lawrence E. Hyatt, Automated Quality Analysis of Language Requirement Specifications, SATC NASA

Joseph E. Kasser, Xuan-Linh Tran, Simon P. Matisons, Prototype Education Tools for Systems and Software (PETS) Engineering, 14th Annual AAEE Conference, September 2003.


Back Up Slides

Specification Analysis Tool(SAT) Overview


Where Do Req’s Come FromClean Slate System

AnalysisPrevious Systems

Similar Systems

Architecture

Algorithms

Automation HMI

Timing & Sizing

TEMPEST

Rel Maint (RMA)

Train Supt (ILS)

Specs = ReqsA-Level

Segment

Subsystem

Hardware

Software

Interface

Mechanical

TestUnit

Integration

Subsystem

System

Performance

Functional

Ops Suitability

Interoperability

Key Site


Where Do Req’s Come FromIncremental System

AnalysisReuse Specs

New Analysis

Specs = ReqsLevels As Needed

TestTests As Needed


A NASA View

• Imperatives are words and phrases that command something must be provided

• Imperatives are in descending order of strength as a forceful statement of a requirement– Shall, Must, Must Not, Is Required To, Are Applicable,

Responsible For, Will, Should

• The NASA requirements documents judged most explicit have majority of imperative counts associated with the upper list items

Automated Req’s Measurement (ARM) tool


A NASA View

• Shall– dictates provision of a functional capability

• Must or must not – establishes performance req’s or constraints

• Is required to– spec statements written in passive voice

• Are applicable – includes by reference, standards or other documentation

as additions to specified req’s



A NASA View

• Responsible for – systems with predefined architectures

– e.g. "The XYS function of the ABC subsystem is responsible for responding to PDQ inputs."

• Will– cite operational or development environment things

provided to capability being specified

– e.g. "The building's electrical system will power the XYZ system”



A NASA View

– In a few instances "shall" and "will" used interchangeably containing both req’s and descriptions of operational environment

– system boundaries not always sharply defined

• Should – not frequently used as imperative

– when used, statement is always very weak

– e.g. "Within reason, data files should have same time span to facilitate ease of use & data comparison”



An FAA View

• "Shall" is one of those officious and obsolete words that has encumbered regulations and other documents for many years.

• The message that "shall" sends to the reader is, "this is boring deadly material.”

• "Shall" is imprecise. It can indicate either an obligation or a prediction.

• Dropping "shall" is a major step in making your regulation more reader-friendly.

Prepared by the Plain English Network (PEN) (www.plainlanguage.gov)

Adapted for FAA, February 2000 (http://www.faa.gov/language/)


An FAA View

• Don't be intimidated by the argument that using "must" will lead to a lawsuit.

• Many agencies already use the word "must" to convey obligations with no adverse legal effects.

• You can avoid "shall" by substituting "must" to indicate an obligation or "will" to indicate that an action will occur in the future. Be careful to consider which meaning you intend to communicate to your readers.

Prepared by the Plain English Network (PEN) (www.plainlanguage.gov)

Adapted for FAA, February 2000 (http://www.faa.gov/language/)


Walt’s View

• Keep it Simple– Remove all special cases that need to be remembered

– No time for fine distinctions of English syntax

• Use “shall” for ALL req’s– Use SRDB attributes like “future”

• Don’t try to spec other systems– Spec your interface to other systems

• Minimize descriptive text to minimize noise– Use analysis and design documents

CassBeth Inc. Specification Analysis Tool (SAT) Overview October 2005.

Documents

services slide

optimized slide

walts view slide

control panel slide

portal demo slide

final spec doc slide

nasa report

cmsei report