1 Human Systems Integration Presentation to Orlando INCOSE 09 June 2005 John Burns & Jerry Gordon Sonalysts, Inc.

1

Human Systems Integration

Presentation to Orlando INCOSE09 June 2005

John Burns & Jerry GordonSonalysts, Inc.

2

Human Systems Integration & Systems Engineering

I. Introduction– Definition & context– Requirements– How did we get here?

II. Systems Engineering & Human Engineering– Mapping systems and human engineering– Bringing them together– What do HSI specialists build?

III. Putting it All Together– What’s going on?– What’s left to solve?– Future needs

3

Part I

Introduction

4

At the End of This Presentation…

You’ll be able to You’ll be able to answer:answer:

• What is HSI?• What is the requirement

for HSI?• How does HSI relate to

the systems engineering organization?

We’ll be able to We’ll be able to discuss:discuss:

• What are the challenges to realizing the goals of HSI?

• Is HSI relevant to systems engineering?

5

For Additional Information…• Handbook of Human Systems Integration, Harold

R. Booher, 2003, ISBN: 0-471-02053-2, 1024 pages

• Human Systems Integration Symposium 2005: – http://www.navalengineers.org/Events/HSIS2005/HSIS05Index.html

• Human Systems Information Analysis Center: – http://iac.dtic.mil/hsiac/index.htm

• SC-21/ONR Science & Technology Manning Affordability Initiative:– http://www.manningaffordability.com/s&tweb/index_main.htm

6

What’s the Problem?

MH-60S

MPA RDT

Console

GCSGround Crew

USV

BAMS

LCS

UUV

7

HSI Defined

• Human Systems Integration is defined as a process that optimizes the human part of the total system equation by integrating human factors engineering, manpower, personnel, training, health, safety, survivability, and habitability considerations into the system acquisition process

8

HSI Domains• MP&T and, to a lesser

extent, HFE are intransitive

• Safety, Health, Survivability, and Habitability are governed by laws and directives

• Each domain has experts, methods, & tools

HSIHSI

HSI is part of Systems Engineering

Tra

inin

g

Sa

fety

He

alth

HF

E

Su

rviv

ab

ility

Ha

bita

bili

ty

Ma

np

ow

er

Pe

rso

nn

el

9

What is the HSI Requirement?• Chairman of the Joint Chiefs of Staff Manual,

Operation Of The Joint Capabilities Integration And Development System (CJCSI 3170.01A)– Change from a requirements-based acquisition

process to a capabilities-based acquisition process.

• Department of Defense (DoD) Instruction 5000.2, 12 May 2003, “Operation of the Defense Acquisition System

The PM shall have a comprehensive plan for HSI in place early in the acquisition process to optimize total system performance, minimize total ownership costs, and ensure that the system is built to accommodate the characteristics of the user population that will operate, maintain, and support the system. HSI planning shall be summarized in the acquisition strategy and address the following:

• Human Factors Engineering• Personnel• Habitability• Manpower• Training• Environment, Safety, and Occupational Health• Survivability

The PM shall have a comprehensive plan for HSI in place early in the acquisition process to optimize total system performance, minimize total ownership costs, and ensure that the system is built to accommodate the characteristics of the user population that will operate, maintain, and support the system. HSI planning shall be summarized in the acquisition strategy and address the following:

• Human Factors Engineering• Personnel• Habitability• Manpower• Training• Environment, Safety, and Occupational Health• Survivability

• MIL-HDBK 29612 Training System Acquisition

10

HSI…Putting the Human Back into the System

HardwareHardware

SoftwareSoftware

OperatorsOperators MaintainersMaintainers

HardwareHardware

SoftwareSoftware

Traditional System ViewTraditional System View More Accurate Representation of SystemMore Accurate Representation of System

11

HSI is Not Just About Tools

12

Error Quantification 1. A Technique for Human Event Analysis2. Absolute Probability Judgment3. Accident Sequence Evaluation Program4. Human Error Assessment and Reduction Technique5. Human Error Reliability Methodology for Event Sequences6. Human Reliability Management System7. Justification of Human Error Data Information8. Method of Paired Comparisons9. Quantitative Risk Assessment System10. Relex Reliability Prediction Software11. Success Likelihood Index Method12. Systems Analysis Programs for Hands-on Integrated Reliability Evaluations13. Technique for Human Error Rate Prediction

Usability 1. Carlow Usability Test Tool for Evaluation and Research2. Cognitive Walkthrough3. Conceptual Walkthrough4. DENIM5. Diagnostic Recorder for Usability Measurement6. Distributed Usability Evaluation Tool7. Heuristic Analysis8. Hiser Element Toolkit9. IBM Ease of Use guidelines10. Questionnaire for User Interface satisfaction11. System Usability Measurement Inventory (SUMI)12. System Usability Scale13. Technique for Human Error Assessment14. Usability Problem Classification (UPC)15. User Talkthrough16. User Test17. WebMetrics 1.018. Website Analysis and Measurement Inventory19. WebTango

Situation Awareness1. Attention-detection2. Cognitive Compatibility SART3. Crew Situation Assessment4. Critical cue detection5. Critical Decision Method6. Performance measures7. Situation Awareness Behaviorally Anchored Rating Scale8. Situation Awareness Global Assessment Technique9. Situation Awareness Rating Technique10. Situation Awareness Verification and Analysis Tool11. Situation Present Assessment Method

Incident Investigation1. Apollo RCA software w/ Reality Charting2. Boeing Safety Management System3. British Airways Human Factors Reporting System4. Cabin Procedural Investigation Tool5. Incident Analysis Tool - Modified6. Maintenance Error Decision Aid7. Paper Root cause analysis8. Procedural Event Analysis Tool9. Ramp Error Decision Aid10. REASON Root Cause Analysis11. Team Root Cause Analysis

Error Representation1. Boeing Fault Tree Analysis software2. Boeing paper-based fault tree method3. Cause Consequence Analysis4. Event Tree5. Fault Tree6. Faultrease7. NASA Fault Tree Handbook with Aerospace Applications8. Relex Fault tree/event tree

Decision Making1. Adaptive User Model2. Analytical Hierarchy Process3. Applied Cognitive Task Analysis4. Course of Action Training Tool5. Knowledge Analysis of Tasks

Error Identification 1. Generic Error Checklist2. Generic Error Modeling System3. Human Hazard and Operability Study4. KSC HF PFMEA5. Murphy Diagrams/Critical Action and Decision Approach6. Process Failure Mode and Effects Analysis7. Process Hazard Analysis Pro8. Relex FMEA

Overexertion1. 3-D Static Strength Prediction Program2. American Conference of Government Industrial Hygienists Lifting Guide3. Auditory Hazard Assessment Algorithm4. National Institute for Occupational Safety and Health Lifting Guide5. Utah Compression Force

Anthropometry1. Anthropometric Data Analysis 2. Crew Chief3. DOD-HDBK-743A, Anthropometry of U.S. Military Personnel4. Generator of Body Data5. Human Engineering Analysis and Requirements Tool6. HumanScale7. Mannequin BE8. Soldiers Day

Comprehensive1. Anthropos ErgoMax2. Jack3. Mannequin Pro4. Panel Layout Automated Interactive Design5. Vision 3000

Repetitive Trauma1. @Work Office2. American Conference of Government Industrial Hygienists Hand Activity Level3. Continuous Safety Sampling Method4. Ergo Intelligence Upper Extremity Assessment5. Ergonomics Audit Program Checklist6. FAA Air Traffic Control Checklist7. Level 1 Ergonomics Methodology Guide for Maintenance/Inspection Work Areas8. Moore-Garg Strain Index9. Occupational Safety and Health Agency computer workstation checklist10. Rapid Entire Body Assessment11. Rapid Upper Limb Assessment12. Washington Safety and Health Agency checklist

Mental Workload 1. Automated Neuropsychophysical Assessment Metrics2. Bedford Workload Scale3. Cognitive Neurometric System4. Complex Cognitive Assessment Battery5. Embedded secondary task method6. External secondary task method7. Frequency Weighted Task Complexity Index8. Instantaneous Self Assessment9. Modified Cooper-Harper Workload Scale10. NASA Bipolar rating scales11. NASA Task Load Index12. Performance and Usability Modeling in ATM13. Primary task method14. Psycho-physical Assessment Test System / Workload Assessment Monitor15. Subjective Workload Assessment Technique

Info Processing 1. APEX2. ATLAS3. GOMS Language Evaluation and Analysis4. iGEN5. Improved Performance Research Integration Tool6. Integrated Performance Modeling Environment7. Man-Machine Integration Design and Analysis System8. Ship System Human Systems Integration for Affordability and Performance Engineering

Task Analysis1. Boeing Task Analysis paper method using Micrografx and Designer2. Critical Decision Method for Task Analysis3. Decision-Action diagram4. Event Sequence Diagram5. Hierarchical Task Analysis6. Linear Task Analysis7. Rasmussen's SRK framework8. Tabular TA9. Timeline TA

Fatigue1. Energy Expenditure Prediction Program2. Fatigue Avoidance Scheduling Tool3. FMR Alertness Monitor4. Rodgers Priority for Change

Perception1. Legibility Modeling Tool2. Locate3. Standard observer for spatio-chromatic detection

Toolkit1. ErgoIntelligence MMH2. ErgoMaster3. ErgoWeb JET Software (Job Evaluator Toolbox)4. VISION 3000 Software

NASA HF Tools

13

NAVSEA 03 Human Performance

Modeling ToolsTool3DSSPPACT-RC3TRACECARTCASHE-PVSENVISION-ERGOErgoIntel MMHErgoMasterFASTiGENIPME

Manning & Affordability Human Engineering Tools

ACTADIVAALPHA/SimASSESSCASACOMBIMANComputerManCREW CHIEFCUTTERDestinationENVISION/ERGOEPICHuman ScaleI-CANiGENIMAGEIMPACT

IMPRINTINDIIPMEI-TASKLMTMacSHAPAMDHMSOASYSORCAPATS/WAMPRICE HLRECAPROMANSAFEWORKSIMWANTramsom JackWINCREW

NAVY HF Tools & Methods

14

HSI: Why Get Involved Early?FY FY

+4(IO

C)

FY+8

FY+1

2(F

OC

)

FY+1

6

FY+2

6

FY+3

4

FY+4

2

FY+5

0(R

etire

men

t)

$M

Construction

Operations

RDT&E

65 %

33 %

Decisions made here...

lock in 80-90%of costs here...

anddetermine mission capability here

Early decisions drive TOCEarly decisions drive TOC

15

HSI: Early & Often

IOC

BA

System Development& Demonstration Production & Deployment

Pre-Systems Acquisition

Systems Acquisition

Operations & Support

C

Sustainment

LRIP Full-Rate Prod & Deployment

Sustainment

Disposal

FOC

The Defense Acquisition Management Framework

Joint Capability Integration and Development System

New System Update Spec ECP, OR

Human Systems Integration

Legacy driversTrade Space

System RequirementsPerformance Prediction

16

Human Engineering Process• ONR Manning and Affordability (1998) gave us the Top

Down Functional Analysis (TDFA) methodology• Process for analyzing and allocating functions at console

level of detail– Mission Analysis– Requirements Analysis– Function Analysis– Functional Allocation– Design – Verification

• A systematic approach to considering the impact of the system on the human

17

Part II

Systems Engineering & Human Engineering

18

Systems Engineering Process View

Requirements Analysis

Function Analysis

Design Synthesis

System Analysis and Control

EIA – 632 Systems Engineering process

19

System Engineer Data View

FFBD

UML

Requirements

20

Human Engineering Process View

21

Mission Performance Requirements

Human Performance Requirements

System Performance Requirements

• Missions• Mission Essential Tasks

• Operational Processes• Equipment• Organizations

• Individual/Team/Collective Tasks

Missions

Functions

Tasks

Mission Performance Requirements

Human Performance Requirements

System Performance Requirements

• Missions• Mission Essential Tasks

• Operational Processes• Equipment• Organizations

• Individual/Team/Collective Tasks

Missions

Functions

Tasks

Human Engineering Data View

22

Mission

Function

Task

Requirements

Function Allocation

Design SynthesisRole - Resource

Resources

System Behavior

System Behavior

Linking the ViewsWhat is being done?

What is doing it?

23

Task Network Modeling•Executable Models•Hierarchical•Represent the Sequencing of Operators Performing Actions•WSM and/or TSCM

24

How HSI and SE pieces Fit Together

Hardware

Software

“System”

Maintainers

Operators

Requirements

Describe

Interface to

Organization Capabilities

Describe

DODAF SV

DODAF OV

25

What is a “Function”?

Hardware

Software

“System”

Something this does?

Maintainers

Operators

Which implies that these must do something?

Or something that this whole organization does?

Hardware

Software

“System”

Maintainers

Operators

26

Function Allocation

Hardware

Software

“System”

Maintainers

Operators

Increasingly ManualIncreasingly Automated

Optimized for Total System Performance

27

The DODAF Recognizes This Relationship

Operational Views (OV) System Views (SV)

Organizational CompositionOrganizational FunctionsOrganizational RulesInformationCapabilities

Machine CompositionMachine FunctionsMachine RulesDataRequirements

28

Humans in the System Boundary

• Notions of Human Capital Objects (HCO) as system components

• System “behavior” (i.e. functional description) is now organizational in nature

• Humans must be interfaced to the machine (Kybernetica) via Human factors, ergonomics and health and habitability

29

HSI and Systems Engineering

• Conventionally, human issues were grouped as “logistics” issues and system was “hardware and software”

• Manning and Affordability (USN) addressed HSI as a “Human Engineering Process” in parallel with the “systems engineering process”

• We think that HSI should be part of Systems Engineering• HSI and SE would then address the fabrication and

support of human components (i.e. Human Capital Objects) in the systems engineering trade space

30

Fabrication of Human Capital Objects

• Production Process – Selection– OJT– Accession training– Professional development

• Quality Assurance – Skills and aptitudes– Qualifications and certifications– 5VM

• Logistics Inventory – Recruiting– Manpower levels

• Interface Specification– Human Factors Engineering– Health and Habitability

• Performance Specification– SEAPRINT data– Training Standards/ILE

• Process Improvement – Science of learning– Human Performance Centers

31

Human Capital Object

• Acquiring human capital objects (HCOs) has the same concerns as building machines: performance expectations, production facilities, fabrication and logistics support costs, supply pipeline, development schedule, & technical risk

32

Systems Engineering for the Total System

Requirements Analysis

Function Analysis

Design Synthesis

System Analysis and Control

What is the desired capability?

What does that require organizationally in the form of the human-system architecture?

How do I staff, support and train that organization?

Maintain HSI presenceCoordinate HSI planHSI concerns addressed in requirementsHSI concerns affect architectural approach

EIA – 632 Systems Engineering process

33

Part III

Putting It All Together

34

Initiatives

• JCIDS

• MANPRINT, SEAPRINT, AIRPRINT,

• US Navy Virtual SYSCOM

• Human Performance Centers

• OPNAV N12 REORG

35

Math “isms” that get in the way of Human Performance Analysis

• Measures and Constructs used by human performance specialists often “collapse” these issues

• Preserved in the best practices of a particular specialty, consistent with a traditional problem space

• Communication is best when the receiver has a model of the transmitter

• We can move data, but we use information

36

Variable Order is Important

• Recursive data relationships• Coupled Solution Spaces (intransitive)

Total Life Cycle Costs

37

Boundaries and Assumptions

New System

Consoles

New SystemUAV

Operators

External Agency

Old View

New View

Virtual System 1

Virtual System 2

Aggregations and Compositions are not strictly hierarchical

38

Level of Detail Confidence and Precision

Organizational - Architectural

Teams – Space Arrangement and C2

Individual - Consoles

Cognitive- Application

Emergent EffectsEmergent Effects

39

Correlation Does not Equal Causality

1-1

1-*

0-1

*-*1-*

1-1

40

Pareto Principle (80/20 rule)

Frequency Of usage of feature

Relative Effect on Total Cost / Performance/cost Impact of

Dropping Feature

41

Averages & Medians versus RangesThreshold

Weighted average

median

42

Time domain versus frequency domain

Cue 1Process 1a Process 2

Process 1b

If Process 2 uses best of raw (cue 1) and processed (1b) data, is process 1b completed timely enough to be used?

From performance range in previous slide, WHEN does the below threshold performance occur?

43

Axiom #1- Boundaries • The use and arrangement of human or machine components will

have an impact on the functional decomposition of the system– Information clustering– Frequency response– Activity type

• Corollary – “requirements” presupposes a notional allocation between human and machine. When SE “allocate” requirements back to humans, they are load shedding based on developmental cost/schedule constraints with no explicit attempt to evaluate impact on performance expectations or total ownership cost (TOC).

• Functional Decomposition is tied to selected analysis boundary• Organizations talk of Capabilities – Constructed systems talk of

requirements

44

Axiom #2- Object Oriented• Analyzing complex systems requires a reductionist

approach. • Object-Oriented Requires defining components• Components must be crafted from decoupled sub-

systems• Componency facilitates successful integration as well as

facilitating reuse necessary for system of systems (SoS)/family of systems (FoS) concept.

• Logical Components must include the appropriate contribution of humans.

45

Axiom #3- Precision

• Design and requirements decisions should only be made at the level of detail for which confidence in the information is high.

• Every level of detail has constraints and variables which have effects on constraints and decisions for both finer and coarser grains

46

Axiom #4- Decoupling• Optimization of Cost, Schedule and

Performance requires:– Logical Components must be analyzed

individually for sensitivity to potential design changes, and the most sensitive components addressed first in the trade space

– Fabrication and Support (including HSI and traditional ILS) issues with each component must be addressed in the proper order to ensure a decoupled trade space

47

Axiom #5- Formalism • In order to succeed and fully support systems

engineering, HSI must adopt the formalism, language and paradigm of systems engineering.

• SE supports acquisition by creating and monitoring artifacts

• ICD, CDD, CPD• Milestone decisions• HSIP, NTSP, MER

• Acquisition strategy drives the nature of the design problem

• In order to be useful, HSI must be informed by, and directly support the artifacts of, system acquisition.

48

The New System Over-View

MH-60S

MPA RDT

Console

GCSGround Crew

USV

BAMS

LCS

UUV

49

Final Thoughts

• What are the challenges to realizing the goals of HSI?

• Is HSI relevant to systems engineering

• Whither HSI? Should HSI be incorporated (back into) integrated logistics support?

50

Questions/Discussion

Thank You!