2/13/2014 1 Engineering & Technology Management Group The Delta Forum 2003 Operating within a Risk Averse Aerospace Environment – Coping with the Unknown.

04/10/23 1Engineering & Technology Management Group

The Delta Forum 2003Operating within a

Risk Averse Aerospace Environment – Coping with the Unknown

Sponsored by the

AIAA Engineering & Technology Management Group

Session Chair: JoAnne Rocker

Eng

inee

ring

Technology

ManagementTracking the Constant of Change

SystemsEngineering

ManagementHistory

Society Legal Aspects

Economics

LogisticsSupply Chain

Risk

TechnicalInformation

MultidisciplineDesign

ProductDevelopment


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

• Introduction – Carolyn Griner, ETM Director• Risk Acceptance and Society at Large

– Tim Howard, Society & Aerospace Technology TC• Balancing Technology Risk and the Return on Investment

– Paul Collopy, Economics TC• Legal Considerations Affecting Risk Acceptance

– Scott Johnson, Legal Aspects TC• Risk Mitigation through Knowledge Management

– Joanne Rocker, Technical Information TC• Risk Mitigation and System Integration

– Peter Rutledge, Systems Engineering TC• Multidisciplinary Design Optimization as a Powerful Risk Mitigation

Tool– Achille Messac, MultiDisciplinary Design TC

• Integrating Risk Management – Richard Raiford, ManagementTC

• Panel Discussion – All Speakers

Overview


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

• Our Group’s TCs examine the issues affecting the aerospace industry in the areas of Engineering and Technology Management

– Our disciplines include:• Economics• History• Legal Aspects• Logistics• Management• Multidisciplinary Design Optimization• Product Development• Risk Management• Societal Aspects• Supply Chain Management• Systems Engineering• Technical Information

Introduction


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

• Our purpose today is to examine a single issue affecting the aerospace industry, from the several perspectives of the various disciplines within the Group

• This year’s theme addresses the change in how the aerospace industry accepts the risks inherent in our technologies, to develop, integrate, and deliver goods and services in a global marketplace

• Our speakers come from across the industry to address topics in– Risk and Society– Intellectual Property– Investment– Knowledge Management– Systems Engineering– Mutlidisciplinary Design– Aerospace Management

• Our panel at the end will draw together the separate perspectives in response to your questions

Introduction


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Risk Acceptance

And

Society at Large

Tim Howard

Society & Aerospace Technology TC

Topic 1


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Overview

• Historical Perspective of Risk

• Society’s Response to Risk

• Society, Risk and Aerospace

• Aerospace, Risk and Society

Against the Gods; The Remarkable Story of Risk

by Peter L. Bernstein

Against the Gods; The Remarkable Story of Risk

by Peter L. Bernstein


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Risk

“The word ‘risk’ derives from the early Italian risicare, which means ‘to dare.’

(Bernstein)

“The word ‘risk’ derives from the early Italian risicare, which means ‘to dare.’

(Bernstein)


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

A Historical Perspective of Risk

• Before 1200s - Gambling, Arabic numbers & Fibonacci (1, 2, 3, 5, 8)

• The Renaissance

– 1500s – Cardano, gambling & probability

– 1600s – Pascal & Fermat – Probability; Graunt & statistical sampling – London population tables

• 1700s

– Daniel Bernoulli: information quality & utility value

– De Moivre – normal distribution & standard deviation (The Bell Curve)

– Insurance & the coffee houses – Lloyd’s

• 1875 – Galton & Regression to the Mean (sweetpeas)

• 1900s –The Rationality Debate

– Uncertainty & failure of invariance – same problem presented differently results in different choices


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Society’s Response to Risk• Until 1960s, prevailing idea is that

individuals are naturally risk averse• 1960s - Loss aversion more

powerful motivation than risk aversion – we hate to lose, and will take the gamble to avoid a sure loss– Aerospace losses often more

catastrophic, and more sensational than other technology or system losses

• Risk tolerance increases in relation to our attitudes about the future

• Issue presentation key to the acceptance or rejection of risk and hence loss

• Need to identify society’s level of acceptable loss

Version A: $30 to start; Choices are coin flip with heads = wins $9, tails = lose $9; no flip = keep the $30Possible outcomes are $21, $30, $3970% choose to flip coinVersion B: $0 to start;Choices are coin flip with heads = wins $39, tails = wins $21; no flip = given $30 Possible outcomes are $21, $30, $3943% choose to flip coin

Version A: $30 to start; Choices are coin flip with heads = wins $9, tails = lose $9; no flip = keep the $30Possible outcomes are $21, $30, $3970% choose to flip coinVersion B: $0 to start;Choices are coin flip with heads = wins $39, tails = wins $21; no flip = given $30 Possible outcomes are $21, $30, $3943% choose to flip coin


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Society’s Response to Risk

• Society needs risk to grow; historical outlets are exploration and commerce– Exploration & development of old frontiers

driven by commercial needs for new markets• The Phoenicians, Columbus, The British

Empire• 10,000 years of maritime technologies enabled

exploration and commerce• <500 years of risk management to improve

probability of success• Insurance, economic forecasting,

investment strategies


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Society, Risk and Aerospace

• The first century of flight made us the new risk outlet for societal growth– Air travel, cargo movement, satellite systems and

information transport fundamental to the success of the new global economy

– We enable exploration and development • Shackleton, Byrd, & the 19th century frontier• Space is the new frontier for human exploration &

commercial development– 100 years of aerospace technologies available to

support the need to grow– New fields use risk management to improve probability

of success (National defense, air traffic control, space launch)

– We develop new technologies and new applications of existing technologies to provide social growth


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Aerospace, Risk and Society

• Our industry was founded by risk takers; now we operate in a risk-averse mode

– Societal growth stunted, or turned inwards; not the natural process

• We need to place our mean to the right of where we are, and start regressing – we need to take more risks

– Rejuvenate our industry

– Rekindle interest in our profession

– Reclaim technology leadership role

society needs us to play

• IT can only give us a virtual

frontier; we have already delivered the real thing

• We need to embrace our role as the new enabler of societal growth

– It’s time for a fresh deck of cards…


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Risk

… if we dare… if we dare


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Topic 2

Risk and the Return on Investment

Paul D. Collopy

Economics Technical Committee


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Two Messages

• Managers: Embrace Risk– Eliminate unnecessary downside risk– Know what you are signing up for– Use strategies like hedging to improve risk+return

• Analysts: Don’t lose sight of the Fundamentals– E[NPV] is the basic metric (return)– Adjust for risk aversion when > 4% Equity

• otherwise never trade E[NPV] for


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Risk Mitigation

• Great idea, if there is no cost

– in product performance

– in development time

• Risk free development Sunset Industry

• Performance > SoA = Technical Risk

Everybody loves but


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

What is Risk?

Risk = Uncertainty

400 500 600

Risk

Combat Range400 500 600

Combat Range

Low Risk Program


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Military Aerospace Development

0 5 10 15 20

Years

Con

cept

Stu

dies

Tec

h D

evel

opm

ent

Dem

/ V

al

E &

MD

Pro

duct

ion

Ent

ry in

to S

ervi

ce

Careful Risk Mitigation ensures the system will achieve obsolescence and military irrelevance upon entry into service


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Venture Star


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Caveats

• Choose your Battles

• Hope for Success / Plan for Failure

• Do not Bet the Company


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Choose your Battles

• Risks Interact:

• ~ 5 Technical Risks can be managed at once• Use Spiral Development for more ambitious

programs

0 1Average Probability of Failure

Dev

elo

pm

ent

Co

st

or

Sch

edu

le

Diseconomy of Scale


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Hope for Success / Plan for Failure

• Quote expected performance

• Plan backup designs

– Low risk

• Prepare to take a performance hit

• There is always next time


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Betting the Company

0%

4%

8%

12%

16%

20%

0% 4% 8% 12% 16% 20%

% of Equity

Va

lue

/ E

qu

ity

Typical Business Value of Money

Neutra

l Attit

ude to Risk

16 u $ . 0 e

$

.

1 0 16

Risks < 4% equity: value = avg. return

For Risk = 20% eq.business is strongly

risk averse


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Using the Value of Money Curve

0%

4%

8%

12%

16%

20%

0% 4% 8% 12% 16% 20%

% of Equity

Va

lue

/ E

qu

ity

p

1 - p- 8%

+ 8%Arbitrary Base Point

Downside

Upside = 3%

Downside=5% Value of deal =

3%p + 5%(1-p)Upside


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Summary

• Risk has upside and downside

• Risk-free designs condemn the industry to mediocrity

• Good risks need to be managed not mitigated


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Legal Considerations

Affecting Risk Acceptance

R. Scott Johnson

Legal Aspects of Aerospace TC

McKee, Voorhees & Sease, LLP

Topic 3


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

What is risk?

• Potential for


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Negligence

• Duty• Breach of Duty• Injury• Causation


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Negligence

• Duty– Generally: Exercise Reasonable Care

• Breach of Duty• Injury• Causation


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Negligence

• Duty– Specifics: Statutes, Rules, and Regulations

• Breach of Duty• Injury• Causation


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Specifics: Statutes, Rules, and Regulations

• Federal Aviation Act of 1958– 49 U.S.C.

• Provides regulatory authority for:– Department of Transportation

• Aviation Economic Regulations or AERs 14 C.F.R. §

200-400– Federal Aviation Administration

• Federal Aviation Regulations or FARs 14 C.F.R. § 1 -

199– National Transportation Safety Board

• 49 C.F.R. § 830, 31, and 45


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Specifics: Statutes, Rules, and Regulations

• State products liability laws– “liability of a manufacturer or seller of a

chattel which is defective and/or unreasonably dangerous and causes” injury.

– William R. Prosser, Handbood of the Law of Torts 641 (4th ed. 1971)

– Three theories of recovery:• Warranty (contract remedy)• Strict liability (tort remedy)• Negligence (tort remedy)


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Negligence

• Duty• Breach of Duty• Injury• Causation


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

The Bad

• Create broad range of compliance duties• Create investigation bodies

– NTSB determines “probable cause” for all aircraft accidents in the U.S.

• Create Enforcement Procedures– Civil and criminal penalties


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

The Bad

• Other Sources of Liability– Contract provisions– Other Federal and State Statutes and

Regulations• Environmental• Intellectual Property• Employment• Corporate

– International Treaties


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

The Good

Statutes can be good?


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

The Good

• General Aviation Revitalization Act of 1994 (42 U.S.C. § 40101) – 18 year statute of repose

• prohibits any legal action for any aircraft, engine or part at issue

– Applies to all general aviation aircraft


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

The Good

• General Aviation Revitalization Act of 1994 (42 U.S.C. § 40101) - Exceptions– Manufacturer “knowingly misrepresented or

concealed or withheld” from the FAA information relating to:

• The type certification• components• airworthiness


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

The Good

• General Aviation Revitalization Act of 1994 (42 U.S.C. § 40101) - Exceptions– claimant was a passenger for purposes of

receiving emergency or medical care


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

The Good

• General Aviation Revitalization Act of 1994 (42 U.S.C. § 40101) - Exceptions– suit is brought under the manufacturer’s

written warranties


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

The Good

• State statute of limitations• International Treaties

– Warsaw Convention • Other liability limiters

– Sales contract provisions • shift liability away• limit liability

– Insurance agreements• cover yourself


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

The Beautiful

• Risk Management Programs– By whom?– Government Initiated

• NTSB or FAA– Insurance Company– In-house


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

The Beautiful

• Risk Management Programs– Evaluate loss potential– Use established risk management

techniques– Put a plan in action


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

The Beautiful

• Risk Management Programs– Evaluate loss potential

• Total potential loss X % Chance loss will occur

• Account for “all” losses (Economic & Social)

– Use established risk management techniques

– Put a plan in action


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

The Beautiful

• Risk Management Programs– Evaluate loss potential– Use established risk management

techniques• Eliminate (Don’t Do It)• Reduce (Rapid Response)• Transfer (Insurance)• Retain (Savings Plan)

– Put a plan in action


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

The Beautiful

• Risk Management Programs– Evaluate loss potential– Use established risk management techniques– Put a plan in action

• Identify the risks– Maintain database of accidents,

lawsuits, liability situations• Create contingencies and risk reducing

methods• Example: British Airways BASIS program


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Thank you!

R. Scott JohnsonMcKee, Voorhees & Sease, PLC801 Grand Avenue, Suite 3200Des Moines, Iowa [email protected]


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Risk Mitigation through Knowledge Management

JoAnne Rocker

NASA Scientific and Technical Information Program

Chair, AIAA Technical Information Committee

Topic 4


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Discussion Topics

Risk mitigation

Knowledge management

• Communities of practice

• Lessons Learned

• Portals

Technology and cultural change

Conclusion

Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Risk Management (RM) is an organized, systematic decision-making process that efficiently identifies, analyzes, plans (for the handling of risks), tracks, controls, communicates, and documents risk

Purpose: increase the likelihood of achieving program/project goals

Risk mitigation depends upon risk analysis

Many useful sources of information should be compiled to provide input for risk analysis, for example:

• Test data

• Expert opinions

• Hazard Analyses, Failure Modes and Effects Analyses

• Lessons learned data and historical information from other programs/projects

• Software verification and validation

• Risk data generated in other steps in the process

Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Risk mitigation is a process involving information analysis from a variety of data sources

Risk Mitigation

Risk Management Procedures and Guidelines, NPG 8000.4


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Knowledge management refers to strategies and structures for maximizing the return on intellectual and information resources

• Tacit form (human education, experience and expertise)

• Explicit forms (documents and data)

Creates new value by improving the efficiency and effectiveness of individual and collaborative efforts to organize and use information resources

Increases innovation and improves decision-making

From: “Defining Knowledge Management”, destinationKM.com,

http://www.destinationkm.com/articles/default.asp?ArticleID=949

Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Knowledge Management


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopmentWhy does the aerospace industry need KM?

• Aerospace is a competitive knowledge-based industry

• Continual learning and re-using information to be competitive, respond to change, stay innovative, and discover new products and processes

• Long-term life cycle for technology development and knowledge can be lost or forgotten if no KM process is in place (KM provides continuity as information is captured, stored, and made available for use)

• Virtual workforce (national and internationally located) need to share resources and information for collaborative work

• Cyclical hiring practices hinders knowledge sharing between experienced employees and new hires

Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Knowledge management strategies and structures

• Communities of practice

• Lessons learned databases

• Portals

Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

From: “Communities for knowledge management,” by Stephen Denning, article found at http://www.stevedenning.com/communities_knowledge_management.html

“Definitions of community of practice vary somewhat, but are usually taken to mean a group of practitioners who share a common interest or passion in an area of competence and are willing to share the experiences of their practice.”

Communities of practice can also be known as:

• Thematic groups (World Bank)

• Learning communities or networks (Hewlett Packard)

• Best practice teams (Chevron)

Communities exist within companies and as well as across company boundaries

Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopmentCommunities of Practice


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopmentCommon traits of communities of practice (CoP)

• Strong sense of identity with the group - members often from the same line of work (salespeople, technicians, engineers, webmasters, etc).

• Practice is not captured in formal procedures – people learn how to do the work and are seen as competent or not based on working with others

• Use the same the same tools, methods, techniques

• Express themselves in a common language

• Cross-disciplinary not attached to organizational chart

• Differ from projects or teams

• Groups are self-selecting

• Emphasis sharing knowledge and experiences

Source: Fred Nickols, “Communities of Practice: Definition, Indicators & Identifying Characteristics,” http://home.att.net/~discon/KM/CoPCharacteristics.htm

Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

“Lessons learned is a narrative account of an actual experience providing an analysis of what happened, what was expected to happen, an understanding of why there were differences, and what was learned.”

Source: Schlumbergerhttp://www.slb.com/oilfield/index.cfm?id=id27859

Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Lessons Learned


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

http://llis.nasa.gov/llis/plls/index.html


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment



Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment



Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment



Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

“A portal is a site featuring a suite of commonly used services, serving as a starting point and frequent gateway to the Web (Web portal) or a niche topic (vertical portal). Web portal services often include a search engine or directory, news, email, stock quotes, maps, forums, chat, shopping, and options for customization.”

From: Marketing Terms.com, http://www.marketingterms.com/dictionary/portal/

Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopmentPortals


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Example: aerospace portal (http://aerade.cranfield.ac.uk/)


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Example: aerospace portal (http://aerade.cranfield.ac.uk/)


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Technology but part of process to implement knowledge management but changing culture is essential to KM success

Source: William Matthews, “Knowledge management's 'worst‘” Federal Computer Week, April 25, 2002, http://www.fcw.com/fcw/articles/2002/0422/web-know-04-25-02.asp

Gartner Inc. conducted a survey on how well organizations utilize knowledge management and found:

• Survey participants scored from high of 78% to low of 2%

• Worst offender was the government with 2%

• Government workers didn’t understand what knowledge management was and how to use it

• Effective knowledge management means information sharing and collaboration and agencies are not used to doing that

French Caldwell, Gartner Research Director, "Knowledge management is a business process that has to be approached with discipline. It is not a technology. You can't buy it in a box."

Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

No KM technology or tool will be successful without a fundamental paradigm shift in corporate culture

• KM initiatives need acceptance and be seen as a way to do things differently

• Means user training, emphasizing the benefits of the new way of working

• KM is not a one-time investment but rather an ongoing process

• Top level management are the champions of KM initiatives

• Success or failure depends on a company's ability and willingness to make real cultural changes

Source: Henry Newberry, “Knowledge Management and Corporate Culture,” Lotus Advisor Magazine, November 2002, p6, http://businessintelligenceadvisor.com/doc/11262

Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Risk Mitigation and knowledge management

• Purpose of managing risk is to increase the likelihood of achieving program/project goals

• Way to mitigate risk is through process of risk analysis which draws upon many different types of data and information sources

• Knowledge management provides the structure for collecting, organizing, and making information available for use (communities of practice, lessons learned, portals)

• Leverages corporate knowledge and creates a collaborative work environment

• Business process - not a technology - that needs top-level support to be effective

Knowledge management should be a business practice in the aerospace industry

• Knowledge-based industry

• Highly specialized technologies and processes making it vital to capture and retain corporate knowledge

• Innovation and knowledge creation are keys to survival in competitive marketplace and knowledge management is about getting the right information to the right people at the right time

Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Conclusion


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Topic 5

Risk Management and Systems Engineering

Pete Rutledge, Ph.DOffice of Safety & Mission Assurance

NASA HeadquartersWashington, DC

Systems Engineering Technical Committee


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

• Both are concerned about mission success.

• Both deal with uncertainty.

• Both must work the entire life cycle.

• Both need the “Big Picture.”

• Both must deal with all system elements.

• Both are concerned about interfaces and dependencies.

• Both are threatened by complexity and tight coupling.

• Both can help help with trade-offs to optimize cost-effectiveness.

• Both benefit from many of the same tools.

• Both can provide valuable support to decision-makers.

Risk Management & Systems Engineering: Many Common Attributes


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Mission Success: What Can Go Wrong?

“What can go wrong?”• Hardware can fail• Software can contain errors• Humans can make mistakes

There is a lot at stake!• Human life• Hardware• Software• Facilities• Scientific knowledge• Time• Money Mars Climate Orbiter:

•Spacecraft•Science•Program delayed•$250 Million

Challenger:•7 lives•Orbiter•Payload•Fleet grounded•$2 Billion+


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Uncertainty: Probabilities & Consequences

• What can go wrong with

this project?

• How likely is it to go wrong?

• What would be the

consequences if something

does go wrong?

• How soon do we need to act?

• What can be done to prevent

things from going wrong,

or at least reduce the

probability or severity of

the consequences?

• What risk mitigation would be

effective?Risk = Probability X Consequences


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

System Life Cycle: From Cradle to Grave

• Risk originates and impacts system in all phases of the system life cycle. – Formulation

• Advanced Studies• Preliminary Analysis• Definition

– Implementation• Design• Development• Operations (including disposal)

• 70-90% of safety-related decisions (good & bad) are made early.

• Early decisions are cheap to make but have big effect on life cycle cost.

• Design process errors are the root cause of many failures.

• Systems Engineering & Risk Management need to begin early for maximum effectiveness.


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

The “Big Picture”: All Elements, Interfaces, & Dependencies

All system elements contribute to risk:

• Hardware

• Software

• Human

Systems Engineering & Risk Management must deal with all:

• Elements

• Interfaces

• Dependencies


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Confounding Combinations & Complications

• The Hardware element– Behavior and failure modes are relatively well understood– Modeling and analysis is routine

• The Software element– Systems are increasingly dependent on large, complex codes– Difficult to exhaustively test large, complex code– Often failures trace back to the errors in the requirements– Many models for predicting reliability—but which one to use?

• The Human element– Human error is said to contribute to 70-90% of accidents– Human performance is difficult to model, analyze, and predict

• Interfaces & Dependencies– Confounding combinations of complex and tightly coupled

elements– Demands a Systems Engineering & Risk Management approach


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Complexity & Tight Coupling• Practically limitless,

unforeseeable combinations of failures.

• System “unravelings” with an intelligence of their own; exposing hidden connections, neutralizing redundancies, bypassing firewalls, and exploiting chance circumstances for which no engineer could reasonably plan.

• Cascading failures can accelerate out of control, confounding human operators and denying them a chance for recovery.

• Systems Engineering & Risk Management become essential.

Mouse Trap—a game by Milton-Bradley

Accidents are inevitable -- “normal.”Charles Perrow, Normal Accidents


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Optimizing Cost-Effectiveness: Risk as a Resource


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Tools of the Trades

• Hazard Analysis (PHA, SHA, SSHA, O&SHA)

• Reliability Analysis (for HW, SW, Human; RBD, prediction)

• Fault Tree, Success Tree, Event Tree Analysis

• Cause & Effect Diagrams (“Fishbone” Diagrams)

• Probabilistic Risk Assessment (PRA)

• Probabilistic Structural Analysis

• Failure Modes & Effects Analysis (FMEA)

• Pareto Analysis

• Monte Carlo simulation

• Cost and schedule analysis

• Data, trend, and statistical analysis

• And more….

Common to both Systems Engineering and Risk Management.


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Systems Engineering & Risk Management for Decision Support

• Given the common attributes just discussed, the Systems Engineering function is ideally suited employ Risk Management methodology to provide valuable support to aerospace engineering decision-makers.


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Dr. Achille MessacMultidisciplinary Design and Optimization LaboratoryMechanical, and Aeronautical Engineering Department

Rensselaer Polytechnic Institute

Multidisciplinary Design Optimization as a Powerful Risk Mitigation Tool


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Presentation Outline

1- Multidisciplinary Design Optimization

What is it?

A bit of history…

2- Aerospace and other MDO Applications

3- Multiobjective reality -- Physical Programming

4- Venturing into the nondeterministic environment of RISK

– The role of MDO

5- Optimization in Conceptual Design

6- Concluding Remark


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopmentFrom Early Timid Optimization Applications

to MDO

From 50’s to 70’s, engineering optimization is

- Primarily Uni-disciplinary Aerospace >> Structure – min. mass

- Computationally Challenged

- Of Timid Problem Scope

- Heavily reliant on the defective MO method -- weighted sum


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopmentFrom Early Timid Optimization Applications

to MDO -- Cont.

In early to mid eighties

- Computers become faster, and start decreasing in cost

- Optimal Control grows in popularity

- Large Space Structures research grows

- Concurrent Engineering becomes a cherished word

- Bi-disciplinary optimization: Control-Structure (Messac, Hale)

- NASA, Air Force, and JPL initiate MDO, CSI programs

- Government financial resources relatively abundant (human, computer – however slow)

- First Pre-MDO conference is held at Langley (Sobieski – 1984)


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

From Early Timid Optimization Applications to MDO -- Cont.

Mid 80’s to Mid 90’s

- Computers are much faster and cheaper

- AIAA MDO TC is born (Transition from Bi- to Multi-disciplinary)

- MDO is still largely Deterministic

- MDO is still implemented in later stages of designNot at Conceptual Design Stage

- MDO is still largely addressing traditional disciplinesAero, Structure, Control – Neglecting Cost, Management, etc.

While…

- MDO’s popularity and use grows many-fold (e.g. GE heating element)

- MDO research enjoys strong international growth

- MA&O Conference grows more and more successful


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

From Early Timid Optimization Applications to MDO -- Cont.

Mid 90’s to Present

-MDO is still implemented in later stages of designNot at Conceptual Design Stage-- New Conceptual Design optimization method emerged

(s-Pareto based Conceptual Design – Mattson-Messac)

-Optimization is still often used using the defective weighted sum paradigm

- Computer cost and speed are dramatically reduced

- MDO grows within government AND industry

-MDO begins to address non-traditional disciplinesCost, Management, Manufacturability, Profit, Safety, Risk, etc.

- MDO becomes more appropriately treated as Multiobjective

- MA&O Conference continues to prosper: 2002 Atlanta; Sept. 2004 Near Saratoga Springs NY


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Multidisciplinary Design Optimization (MDO)

CFD Structures

Performance

Aero Loads

Deformations

Mach Number

Str. WeightAero LoadsTOGW Design space discipline 1

Design space discipline 2Design Variables

Performance

MultidisciplinaryOptimal Design

Discipline 1 Optimum

Feasible Design Space

SuboptimalDesign

Conventional Trades

MDO Search

Discipline 2 Optimum

• Effective Integration of Individual Disciplines/Subsystems to Capture the Interactions• Novel Solution Procedures to Enable Improved System Solutions:

- Account for Interdisciplinary Couplings & Integrated Product Team (IPT) settings.

• MDO = { Design Optimization, Design Exploration, Cross-Attribute Optimization, InterDisciplinary Optimization, System Synthesis }


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

NVH

FrontalCrash

Offset Crash

Roof Crush

Minimize: Vehicle Weight

Subject to: NVH design targetsFrequencyBending DisplacementTorsion DisplacementFrontal crash design targetsDummy HICDummy Chest GProbability of severe injury

Roof crush design targetsMaximum resistant force

50% Frontal Offset crash design targetsIntrusion

Side Impact design targets

Displacements

Viscous Criterion

Safety

Ford-SGI - Vehicle System MDO Project

HPC/MDO for NVH & Safety on the Origin 3800

Side Impact

Everything influenceseverything else!


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Aerodynamics

Propulsion

Controls

Hydraulics

Avionics

ElectricalsystemFuel

system

Landing gear

• Structural optimizationis at MDO roots

AIRFRAMESTRUCTURE

In Vehicle Everything Couples to Structure


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Car Body Optimization for Noise, Vibration, Harshness, and Crash

• Vehicle Roof Crush is a federally mandated requirement to enhance passenger safety during a rollover event.

• NVH must be constrained for passengers comfort

•Modified NVH Model with a Square Ram to Perform Roof Crush as specified by regulations

• Finite Element Model of 390,000+ dof, 6000+ boundary conditions;

• 20 design variables

• Optimization executed using Response Surface approximations to crash analysis and sensitivity-based approximations to NVH

• Implemented on a computer with 256 processors

• Single processor computer would need 257 days to do this optimization• It was condensed to 1 day on the multiprocessor machine.

Courtesy:Ford Motor Co./R-J. Yang


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Air Borne Laser System Design

BMCBMC44I I • Boeing

•8-10 March

Beam Control SystemBeam Control System• Turret Assembly

•Large Optics•Four Axis gimbals•Transfer optics

• Beam Transfer Assembly•Sensor Suite•Active Mirrors•Illuminators•Electronics•Software/Processors

747F Aircraft - 747F Aircraft - • BoeingBoeing • CDR 29 Feb - 3 Mar

Chemical Oxygen IodineChemical Oxygen IodineLaser (COIL) Laser (COIL) • TRW• 21-23 March

System Level DesignSystem Level Design• Boeing• CDR 25-27 April

12


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopmentSupersonic Business Jet Test Case


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

• MDO necessary to protect very small payload margin• Massively computational problem• Candidate MDO method : design data bases for subsystems and disciplines precomputed off-line using multiprocessor computing

Examples of applications that will need all that

Configuration “A” Configuration “B”

Courtesy: NASA LRC/Troutman


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Obtaining an effective objective function is difficult in practice.

Weighted-Sum (WS) method for optimization has its inherent significant drawbacks. It is incapable of capturing a large class of potentially desirable Pareto solutions. It is also the most popular!

Compromise Programming (CP) method can be used to generate a complete set of efficient solutions, but its reliance on meaningless weights is a serious problem. Pareto ...

Etc.

Prevailing Practice


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Obj. Fun. Line

1 1 2 2J w f w f Optimal Solution UsingWeighted-Sum


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

1 1 2 2J w f w f Pareto Solutions Using Weighted-Sum


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

1 1 2 2J w f w f

Obj. Fun. Line

Weighted-Sum in Non-Convex Pareto Frontiers


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Obj. Fun. Line

1 1 2 2n nJ w f w f

Weighted Compromise Programming for Non-Convex Pareto Frontiers


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Quantify Preference for Each Design Metric

Ex: Mass of Beam

Highly Desirable < 250 (kg)

Desirable 250 - 275

Tolerable 275 - 300

Undesirable 300 - 325

Highly Undesirable 325 - 350

Unacceptable > 350

Physical Programming (cont.)


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Expressing Designer Preference During the Optimization Process

Expressing Designer Preference During the Optimization Process


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Behavior of the PP Class Functionfor Change in Preferences

Behavior of the PP Class Functionfor Change in Preferences


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Numerical Example 1 (of 2)Numerical Example 1 (of 2)

Minimize 25

24

23

22

211 xxxxxf

354

3212 )(01.0

323 xx

xxxf

25.02 54321 xxxxxSubject to

05.06.08.024 254321 xxxxx

1025

24

23

22

21 xxxxx

WS CP PPResults

Behaviors of different objective functions for a convex frontier


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Example 2Example 2

1 1min { ( ), ( )}J f f

sin1 f

72 sin1f

2532.15326.0

WS CP PPResults

Behaviors of different objective functions for a concave frontier


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

The Reality of Uncertainty and Risk

1- Non-Deterministic Forum -- SDM

2- Traditional Factor-of-Safety approach is rapidly becoming an anachronistic relic of the past

3- Development of probabilistic design optimization methods

4- New MDO methods emerge to model risk and uncertainty

5- Profit as a design metric gains acceptance

6- Growing recognition that optimization in Conceptual Design is a necessity

7- TC has moves to Engineering & Technology Management Group to explore expanded collaboration


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

• Not Primarily a Technical Problem• Critical Issues are: EDUCATION, MANAGEMENT, AND ORGANIZATION “CULTURE”

A Challenge?

INJECTING MDO INTO INDUSTRIAL PRACTICE:


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopmentConcluding RemarksConcluding Remarks

• The picture is somewhat muddied!Much of good news; yet much to be desired.

• MDO’s power is evident.

•Yet its strongest influence can be felt in this decade if -- its application in a holistic setting comes to pass, and-- key methodological developments take place in collaboration with non-traditional participants

• MDO is a unifying indispensable glue, if we are to effectively design the ultra-safe, profitable, and effective design of the future

• For more information: www.rpi.edu/~messac


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopmentTopic 7

Integrating Risk ManagementThe Integrated Management Framework (IMF)

Systems Engineering Applied to Program Management

Richard Raiford, Chair

AIAA Management Technical Committee


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Integrating Risk ManagementIntegrated Management Framework (IMF)Agenda

• What is IMF?• Where Is IMF Applicable?• What does IMF Look Like When

Implemented?• How Do I Implement IMF on An Existing

Program?


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopmentIntegrated Management Framework (IMF)

IMF Is a

Scalable Program InfrastructureScalable Program Infrastructure

That Uses System Engineering Principles To Implement an Executable Program

-----------------

• Proactive Boundary Spanning• Integrated Processes• Controlled Baselines


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

The System - An Executable Program

Enterprise

Customer’s Program

Operating Environment

Geo-Political

Political

Operational Capability

Assessments

Apply SE Techniques to Everything Relevant to an

Executable Program

Suppliers

Executable ProgramExecutable Program

Operational System(s)

Product Elements

Project Elements


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Some Fundamental Concepts of IMF

• The System is the Executable Program

• Proactive Boundary Spanning (a.k.a. Integration)

– Boundaries Are a Fact of Life (Organization, Product, Program)

– IPTs Don’t Eliminate Boundaries, They Just Re-Arrange Them

• Top-Down: Event-Based Management

– Manage Progress Toward Key Program Events

• Bottom-Up: Requirements-Based Task Accountability

– Know And Track Requirements

• “Pull” to Continually Identify the Risks

– Somewhere, Someone Knows We Are at Risk of Not Meeting A Req’t

• Control Program Baselines

– Everyone On the Same Page, Marching in Unison to the Same Objective


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

IMF – Defined RAARoles of the Program Mgr

**From the AF Critical Process Assessment Tool (CPATS) for PM

Responsibilities of the Program Manager**

(Program Manager)*

Manage the Program

Integrate the Program

Architect the System

(Deputy PM)* (SEIT, PIT, SEMT)*(System

Architect)*

Keep the Program SoldIPTs Consisting of:• Business Mgmt• Planning• Contract Mgmt• Subcontract Mgmt• Materiel• Manufacturing• Engineering• Logistics• Systems Engineering• Security

• Requirements Mgmt• Baseline Mgmt• Performance Mgmt• Risk Mgmt• Analysis and Integration (Boundary

Spanning)• Planning• Integrate Processes and Systems

• Requirements & Functional Analysis and Allocation

• Synthesis• System Analysis

and Control• Verification

The 4 Roles Shown May Be In One Person, or Divided Depending on the Type, Phase,

and Complexity of the Program. In any case, the PM Has Ultimate RAA for All Roles.

4 Roles of the Program Manager

*A Large Program Would Subdivide the Roles

110402


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

CharacteristicsStyle

Hierarchy of Program Styles - From Reactive to Proactive

• Program AND Enterprise Mgmt Frequently Involved in Cost, Schedule, and/or Performance Problems that Threaten Viability of the Program;

• Chaos, Frustration, “Finger Pointing”

Crisis Crisis ManagementManagement1

• Mgmt Has Inconsistent Visibility of Performance; Fluid Rating Criteria• Teams Resist Quantitative Metrics; No Controlled Perf. Baseline

• Frequent Problems Due to Failure to Meet Plan Milestones

IssueManagement2

• Mgmt Has Visibility of Status of Program Performance to Current Plan• Issues Related to Predicted Failure to Meet Plan Milestones

• Surprises as Evolving Environment Overtakes Current PlansPerformance Performance ManagementManagementPerformance Management3

• Ongoing Integrated Assessments of Likelihood of Meeting Objectives in Light of Current Status and Projected Environment

• Plans Updated as Required; Streamlined by “Checkbook”RiskRisk

ManagementManagement4

• Ongoing, Integrated Assessments of Evolving Program & Environment; • Entire Organization Understands Current and Potential Customer Needs;• Integrated Strategies Take Advantage of Opportunities

Opportunity Opportunity ManagementManagement5


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Program Styles Are Based On How We Manage

OpportunityOpportunityMgmtMgmt

RiskRiskMgmtMgmt

PerformancePerformanceMgmtMgmt

IssueIssueMgmtMgmt

Crisis MgmtCrisis Mgmt

Performance

Project

Product

No Process to Update Plans as Program Evolves

Static EV Work Pkgs

No Perf. Baselines

No Visibility Architecture

Schedules not Controlled

Scope Undefined by Contractor

No Risk Mgmt

No Planned Approach for Issue Resolution

No Baseline

Rolling Wave EV Work Pkgs

Subjective, Performance Baselines

Unstructured Reporting

Risk System Available But Improperly Used

Baseline Fluid, Incomplete

Sched Controlled; Incomplete Integration

Scope Defined & Controlled; Team & Customer Misunderstandings

Risk Identification & Mitigation Based on Event Look-Ahead

Issue Mgmt In Place

Design Defined & Controlled

Plans Updated & Published - What-If Exercises & “Chkbk”)

Rolling Wave + Mgmt Reserve “Checkbook”

Objective, Event Based Performance Baselines with Improvement Goals

Uniform, Drill-Down, Event Based Architecture

Integrated, Multi-Level, Risk Mgmt Accounts for Evolving Environment

Integrated “What-If” Analysis and Structured Decision Process

Integrated LCC System Optimization Procedures

Customer KnowsContracted Scope

1 2 3 4 5

Product Baseline Integrated Across Product Life Cycle

Objective, Event > Task Based Performance Baselines

Vertical & Horizontal Schedule Integration

Causal Analysis & Tracked Corrective Action

No SE

Some Subsystem SE

Integrated SE Directed by Qualified System Architect

System SE, But No “Enforcement”

Structured, Integrated System SE with Intermediate Work Products

Scope Controlled; System for ‘Negotiation’ with Customer

Unclear RAA Unilateral Charters Integrated Charters Chartered RAA Integrated Across Schedules

OpportunityOpportunityMgmtMgmt

RiskRiskMgmtMgmt

PerformancePerformanceMgmtMgmt

IssueIssueMgmtMgmt

Crisis MgmtCrisis MgmtOpportunityOpportunity

MgmtMgmtRiskRisk

MgmtMgmtPerformancePerformance

MgmtMgmtIssueIssueMgmtMgmt

Crisis MgmtCrisis Mgmt

Performance

Project

Product

No Process to Update Plans as Program Evolves

Static EV Work Pkgs

No Perf. Baselines

No Visibility Architecture

Schedules not Controlled

Scope Undefined by Contractor

No Risk Mgmt

No Planned Approach for Issue Resolution

No Baseline

Rolling Wave EV Work Pkgs

Subjective, Performance Baselines

Unstructured Reporting

Risk System Available But Improperly Used

Baseline Fluid, Incomplete

Sched Controlled; Incomplete Integration

Scope Defined & Controlled; Team & Customer Misunderstandings

Risk Identification & Mitigation Based on Event Look-Ahead

Issue Mgmt In Place

Design Defined & Controlled

Plans Updated & Published - What-If Exercises & “Chkbk”)

Rolling Wave + Mgmt Reserve “Checkbook”

Objective, Event Based Performance Baselines with Improvement Goals

Uniform, Drill-Down, Event Based Architecture

Integrated, Multi-Level, Risk Mgmt Accounts for Evolving Environment

Integrated “What-If” Analysis and Structured Decision Process

Integrated LCC System Optimization Procedures

Customer KnowsContracted Scope

1 2 3 4 5

Product Baseline Integrated Across Product Life Cycle

Objective, Event > Task Based Performance Baselines

Vertical & Horizontal Schedule Integration

Causal Analysis & Tracked Corrective Action

No SE

Some Subsystem SE

Integrated SE Directed by Qualified System Architect

System SE, But No “Enforcement”

Structured, Integrated System SE with Intermediate Work Products

Scope Controlled; System for ‘Negotiation’ with Customer

Unclear RAA Unilateral Charters Integrated Charters Chartered RAA Integrated Across Schedules


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

IMF – Four Program BaselinesStructure the Program By Structuring the Information

Enterprise Baseline

ProductProductBaselineBaseline

ProjectProjectBaselineBaseline

PerformancePerformanceBaselineBaseline

What the Company Brings to the Effort

(HR, Financial Systems, People, Skills, Processes, Systems, Etc.)

The Framework for the Program (Contract, SOW,

Schedule, Etc.)

The Reason for the Program’s Existence; The Product Baseline Evolves from “Back of the Envelope” to “10,000 Parts flying at

50,000 Feet”

How Performance to Plan Will Be Measured

(Metrics, Thresholds, Formats, Reporting Frequency, Etc.)


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Program Integration with IMF

Defined RAA for Everyone• Program is a System of Systems• Proactive Boundary Spanning

Integrated Plans & Processes

• Req’ts• Baselines (CM)• Performance• Risk• Integration How What

Who

IMFIMF

1

2

3

Managed Information

Enterprise Baseline

Project Baseline

Product Baseline

Performance Baseline


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

- Defined Ownership- No Data Redundancy

IMF - Four Program BaselinesExample Contents

The Baselines . . . Evolve!

Rate of ChangeEnterprise = 0Project = 0/LowProduct = Very High

ProjectProjectBaselineBaseline


EnterpriseEnterpriseBaselineBaseline


Shop & Assy

Planning

ToolDesign

Specs

Requirements

Analytical Models

Bill of Materials

Design

Test Procedures

Tech Orders

Risk db

Shop & Assy

Planning

ToolDesign

Specs

Requirements

Analytical Models

Bill of Materials

Design

Test Procedures

Tech Orders

Risk db

Contract

WBS

Deliverables

Integrated Mgmt

Plan & Schedule

(IMP / IMS)

Program Mgmt

Plan (PMP)

Tailored Processes &

Procedures

Earned Value Mgmt System

SOW

Risk Mgmt

Plan

Stakeholder Mgmt: Communication,

Conflict, Etc., Plans

Materiel, Suppliers, MRP Systems

Human Resource Systems & Databases

Facilities

Policies, Common ProcessesFinancial/

Accounting Systems

Functions & CoEs

Analytical Tools

ITSystems

Data Mgmt Tools & Systems

Labs,M&S


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

IMF – Five Core Processes

• Manage Requirements – Project: Contract, SOW, IMP/IMS, Financials, Suppliers, etc.– Product Life Cycle: (Classical “Systems Engineering”)

• Manage Baselines– Enterprise, Project, Product, and Performance Baselines– Identify, Lock Location, Control Configuration, Audit Compliance

• Manage Performance to Plan– Status (Metrics), Risk, Decision Mgmt, Corrective Action– Drill-Down Integrated with Req’ts, Baseline, and Risk Mgmt Systems

• Manage Risk and Opportunity– Focus on Risk to Meeting Requirements– Multi-Level From Tasks to Major Program Events

• Integrate the Program– Proactive Boundary Spanning Processes, Customer Expectations,

Planning, Communication, Information, Systems and Processes, Functions, Technical, Training, and Closeout


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Integrated Management Framework (IMF)

Where is IMF Applicable?


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

IMF and Program Type / PhaseIMF Is Most Powerful When Product Baseline is Evolving

Concept Development /

Technology Demonstration

System Concept

Selection, Down Select

Operational Needs &

Requirements Analysis

System Development

& Demonstration

Production & Deployment

4 5 632

Technology Upgrade & Insertion

Technology Development& Risk Reduction

Sustainment & Disposal

7

Concept Chart

The ‘Area’ of the Program Type / Phase Implies the Degree to Which IMF Will ‘Help’ the Effort

1


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Questionnaires Guide Tailoring IMF to Program

Type & Phase Complexity Infrastructure Maturity

System

Operational Needs & Req’ts Analysis

Concept Selection, Down Select

Concept Development / Demonstration

System Development & Demonstration

Production & Deployment

Sustainment & Disposal

Team Members Co-Located

# of Skill Sets

Length of Program

Other Bus Areas / Sectors on Team

Suppliers on Team

Done Similar Tasks Before

Cost

Priority

# of People

Project Req’ts

Baseline Mgmt

Performance Mgmt

Integration

• Comprehensive Scope Mgmt System

• WBS, IMP, IMS• Schedules Integrated Vertically &

Horizontally & Config. ControlledProduct Req’ts

• Integrated, Traceable System: Req’ts Analysis, Functional Analysis /Allocation, Trades, Decisions, V&V

• Proactive, Customer Mgmt, Planning, Communication, Information, Systems & Processes

Risk Mgmt • Event Based, Look-Ahead System

• Baselines with Rating Criteria• Structured Decision Making• Corrective Action System

• Program & Project Info Structured, Configuration Controlled, Audited, Available When Needed

Program

• Program Described (Road Map, Funding, SOW, Etc)

• Key Personnel In Place / Trained• Qualified System Architect • Defined Enterprise Baseline

Technology

Development

Insertion

Identification

Worked with Same Team Before

Funding Source Internal or External

Worked with Same Customer Before


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Integrated Management Framework (IMF)

What does IMF Look Like When

Implemented?

Examples of Some Key Elements of a Program Under IMF

Manage Requirements

(What are We Supposed to

Do?)

Manage Baselines (BLs) (Config. Mgmt)

(Write It Down)

Evaluate Performance

(How are We Doing?)

Program Integration (Integrated Program Mgmt Plan)• IPT Handbook • Or Communication Plan• Schedule Tree• PIT Assessment Cycle• Program Control Board• Risk / Opportunity Board• Process / System Board• Weekly Program Reviews

System Integration (System Integration Plan)• Analysis/Model ICPs• SEIT Cycle • Config Control Boards• Eng Review Board• Design Reviews

Element Integration• Drawing Tree• Interface Control Plans • SEIT Meetings

• BL Mgmt Plan/Proc• Des, Build, Supt. Plans• JPCB, PCB• Process Control Board• Four Program Baselines

• Config Mgmt Plan• CCB, SRB, ERB• Interface Control

Working Group• Product Baselines

• Designs, Anal & Test Plans & Reports

• Components / Parts, Subsys, SW

• Interfaces, ICDs

• Risk Mgmt Plan/Proc• Risk / Opp. Mgmt Board• Look-Ahead, 90 day & IMP• Risk/Opportunity DB• Issue/Action DB

• Technical Risk Mitigation Plans

• Risk ID Matrices• Simulations• Waterfall Charts

• Make / Buy Decisions• Part Shortage Mgmt

Process Steps

• Identify• Analyze• Plan• Track• Control

• Identify Customer Req’ts• Develop System Req’ts• Develop Product Req’ts• Analyze & Validate Req’ts• Define (Technical) Solution• Verify and Validate Product

• Plan & Identify BLs• Publish• Manage Change• Track Status• Audit Integrity

Core Process

• Req’ts Mgmt Plan/Proc• Quality & Security Plans• Decision Mgmt Process• Supplier Mgmt Plan/Proc• Program Control Board• Process Quality Audits• Contract > SOW > Sched >

Resources > Budget

• ORD; MND• System Specs (Tree)• Specialty Integration

Plan• Req’ts Anal / Func Anal

Integrated Test Plans• “VCRI”• Slate

• EDRM• Element / Subsystem

Specs• Standard Parts• Process Specs• Material Specs• Component Acceptance

Criteria

• Baseline• Collect Data• Integrate• Evaluate• Report

• Performance Mgmt Plan• CPAR Mgmt Process• Wkly Program Review• EVMS, Sched Tracking,

Issues Traffic

• TPM Plan(s) & Process• Config. Change Traffic• Technical Performance

Measurands

• Shortages, Rejects, MRB• IPD: Sched Commitment

Tracking: Part: Design, Buy, Install, etc.

IMF Implementation Matrix

Plans, Processes, DocumentsMeetings, Boards, (Team Interaction)Inputs, Outputs, Tools, etc.

Key:

Manage Risk / Opportunity

(Will we Do It? & Can We Do it

Better?)

Plans, Documents, Mtgs, Boards, Tools, & Work Products (Examples


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Result of Sample Plan for Implementing IMFSituation at End of Month Three

• IPMP and IPT Handbook Published• Availability of Key Personnel Controlled• Regular Meetings Controlled: Agendas, Attendance, Minutes• Baseline Documents Indexed and Configuration Controlled• IMP In Place • Schedule Tree In Place & Schedules Vertically Integrated

– Horizontal Integration Is In Work• Integrating Charters (RAA) In Place• Req’ts Mgmt System – Front End Implemented

– Contract Parsed, Specs Depend on Maturity of Original System• Event Based Performance Mgmt Implemented (Still Subjective)• Decision Mgmt System – Front End Implemented

– Program Control Board In Place– Corrective Action Board Meeting Monthly

• Risk Mgmt System Initiated (Depends on Maturity of Original System)


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopmentProgram Information Systems - Example

Project Performance

Baselines

Performance Rpts Repository

Performance Mgmt System Decision Mgmt System

Tech. Performance

BaselinesCorrective Action

System

Corrective Action Repository

Performance Mgmt Info System Interface

Req’ts Mgmt Info System Interface

Req’ts Mgmt System (SLATE)Req’ts

Reports Repository

Design Decision Data Repository

(Trades, Mtgs, Decision Memos, Etc.)

Design Decision Index System

Design Support Data Repository (Specs, Analysis, Test Req’ts & Reports)

Risk Mgmt Information System Interface

Risk Tracking System (eRisk)

Risk Mgmt Reports

Repository

Risk Analysis / Decision Data

Repository

Risk Decision Indexing System

Risk Mgmt Reporting

System

Risk Decision System

Risk Database

PMIS Index & Repository

Program Mgmt Info System Interface

Interfaces to Other Systems

Project Mgmt Info System Interface

Project Mgmt Repository / Index

Schedules

EVMS

Budget

Staffing

Capital

Etc. . . .

Portal

Product Data Mgmt System Interface

Design Data

C/PIOSBuy, Fab,

Build Data

Sup-port

Req’ts Mgmt

System

Test Sys

Configuration (& Change) Control System

Links


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Org/TeamLevel

ProgramLevel

TaskLevel

Metric

Task &Resp

person

Timeline | | | | | |

S R # ofEvents

| | | | | |Timeline

Plan

Actual to PlanOR

Resp Org - - - -- - - -

Product / Event | | | | | |

Resp Sub Org

- - - -- - - -

Event / Accom | | | | | |

IMF - Performance Management Model, Procedures, Guides, & Checklists

• Program Management and Control Plan

• Performance Management Plan

• Define Performance and Reporting Req’ts Procedure

• Performance Management Checklist

• Performance Visibility & Database Mgmt System

• Flowdown of Performance Mgmt Req’ts to Suppliers

• Metrics Selection

• Technical Performance Measurement (TPM) Procedure

• Quantitative Mgmt Measurement Work Instruction

• Performance System and Metrics Audits Work Inst

• Performance Analysis


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopmentIntegrated Management Framework (IMF)

How Do I Build an IMF on an Existing

Program?

Create the IPMP, IPT Handbook, and Supporting Docs from Templates That Include Tutorials and Example Formats


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

IMF Tailored & Implemented By Two Documents

• Integrated Program Management Plan (IPMP)• Implements/Tailors IMF for the Program• Objectives, Schedules, Priorities• Mgmt Approaches for Every Area• Policy, Procedures, and Authorities• Replaces Multiple Independent Plans

• IPT Handbook• Foundation for Program Integration• Tells Program Personnel

• How the Program Will Operate• Charters and RAAs• Communication & Issue Resolution• Meetings – Who, What, When, Where


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Use Templates to Generate Integrated Program Mgmt Plan and Supporting Procedures

Business Mgmt

Corporate, Sector Shared Services, and

Business Area / Homeroom Processes, Policies, and Guides

Materiel

HR

EL&T

LOQA

Contract

Integrated Program Mgmt Plan

Government Regulations

TailoredProcedureTemplates

RequirementsRequirements

PlansPlans ProceduresProcedures

1. Introduction, Scope, Purpose

2. Executive Summary

3. Program Overview

4. Organization

5. Administration

6. Management Systems

7. Project Req’ts Mgmt

8. System Life Cycle Mgmt

9. Baseline Management

10. Performance Mgmt

11. Risk / Opportunity Mgmt

12. Program Integration

Integrates (Combines) • Program Management Plan• System Engineering Mgmt Plan• Risk Management Plan• Configuration Management Plan• Performance Management Plan

Interactive Templates

IPT Handbook


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopmentCycle Weekly Program Reviews

Special Topics / Action Items / CalendarCustomer Satisfaction / Top Program Issues and Risks

Strategic Rhythm / Horizontal Integration Strategic Rhythm / Horizontal Integration

Week 1

• Shortages• Staffing• Rework• Audits• Req’ts Volatility

ProcessProcess

Week 2

• Team Reports• Accomplishments• Customer

Satisfaction• TPMs• Changes

ProductProduct

Week 3

• EVMS Data & Indicators

• By Program & by Team

• Performance to IMP/IMS

Cost & ScheduleCost & Schedule

Week 4

• Top Issues & Risk and Associated Closure/ Mitigation Plans

• Corrective Action

Risk / Opp.Risk / Opp.


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Integrated Management Framework (IMF)Summary

• IMF Is a Scalable Program Infrastructure That Uses System Engineering Principles To Implement an Executable Program

• Features – Proactive Boundary Spanning– Four Baseline Categories– Five Integrated Core Processes

• Implemented by Interactive Creation of– Integrated Program Management Plan– IPT Handbook

Integrating Risk Management is Critical for Success


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Panel Discussion

All Speakers

Topic 8


Eng

inee

ring Technology


ManagementHistory



SystemsEngineering

Economics

Risk



ProductDevelopment

Join Us Next Year for

The Delta Forum 2004

Designing and Integrating 21st Century Systems

Next Year

Thank You for joining us!

2/13/2014 1 Engineering & Technology Management Group The Delta Forum 2003 Operating within a Risk Averse Aerospace Environment – Coping with the Unknown.

Documents

areas of engineering

word risk

remarkable story of

aerospace aerospace

risk acceptance scott

etm director risk acceptance

multidisciplinary design

aerospace industry