Enabling Systems Thinking to Accelerate the Development of Senior Systems Engineers INCOSE Presentation February 2007 Heidi L. Davidz.

Enabling Systems Thinking to Enabling Systems Thinking to Accelerate the Development of Accelerate the Development of Senior Systems EngineersSenior Systems Engineers

INCOSE PresentationINCOSE PresentationFebruary 2007February 2007Heidi L. DavidzHeidi L. Davidz

February 15, 2007 © Massachusetts Institute of Technology Slide 2 Heidi Davidz, [email protected]

* Introduction * MethodsResults

ImplicationsConclusion

Acknowledgment of Acknowledgment of Research SupportResearch Support

• Doctoral Committee– Professor Deborah Nightingale (chair)– Professor Tom Allen– Dr. Joel Cutcher-Gershenfeld– Dr. Eric Rebentisch – Dr. Donna Rhodes

• Research Sponsored by the Lean Aerospace Initiative (LAI)– Additional Reader: Professor John Carroll


AgendaAgenda

• IntroductionIntroduction

• Research MethodsResearch Methods

• ResultsResults

• ImplicationsImplications

• ConclusionConclusion



Enabling Systems Thinking to Accelerate the Enabling Systems Thinking to Accelerate the Development of Senior Systems EngineersDevelopment of Senior Systems Engineers

Heidi DavidzHeidi DavidzAdvisor: Professor Deborah NightingaleAdvisor: Professor Deborah Nightingale

Results•Even though systems thinking

definitions diverge, there is consensus on primary mechanisms that enable or obstruct systems thinking development in engineers

•Enabling mechanisms include experiential learning, certain individual characteristics, supportive environment

•Developed a framework and conceptual illustration for systems thinking

Implications• Identified implications for government,

industry, and academia• Highlighted inconsistencies between

policy & effective mechanisms• Need to evolve intervention maturity• Government should set enabling policy• Industry should utilize primary

mechanisms• Academia should continue studying

how systems thinking actually develops

Methods1. Literature Review2. Pilot Interviews3. Field Study with Interviews & Surveys

• 205 Participants, 10 Companies • Expert Panelists, Sr. Systems

Engineers, Sr. Technical Specialists & Jr. Systems Engineers

4. Blue Chip Interviews5. Data Analysis6. Theory Synthesis

Motivation• Increasing interest in systems thinking

• Data needed on systems thinking development

InnatetraitsJob

rotations

Systems work roles

Trainingclasses

What are the mechanisms that develop systems

thinking in engineers?

Universityprograms

Exploratory

Inductive

How do senior systemsengineers develop?


MotivationMotivation

• Increasing complexity of engineering systems and the Increasing complexity of engineering systems and the corresponding need for systems professionalscorresponding need for systems professionals

• Importance of systems engineering, demonstrated in Importance of systems engineering, demonstrated in policy mandatespolicy mandates

• Importance of systems engineering workforce issues, Importance of systems engineering workforce issues, also shown in policy documentsalso shown in policy documents

• DataData needed on systems thinking development in needed on systems thinking development in order to know which methods are most effective in order to know which methods are most effective in developing systems thinking in engineersdeveloping systems thinking in engineers



Need for DATA on Systems Thinking Development


Key ResearchKey ResearchQuestionsQuestions

1.1. What are enablers, barriers, and What are enablers, barriers, and precursors to the development of precursors to the development of systems thinking in engineers?systems thinking in engineers?

2.2. How do senior systems engineers How do senior systems engineers develop?develop?

3.3. What are the mechanisms that develop What are the mechanisms that develop systems thinking in engineers?systems thinking in engineers?




• Broad literature found on “systems thinking” Broad literature found on “systems thinking” – Lack of a central, ongoing discussionLack of a central, ongoing discussion– Systems thinking literature found in disparate fields and Systems thinking literature found in disparate fields and

journals, from systems dynamics to systems engineering journals, from systems dynamics to systems engineering to general philosophyto general philosophy

• Very limited literature on “systems thinking Very limited literature on “systems thinking development” and mechanisms for developmentdevelopment” and mechanisms for development

• Heavy dependence on heuristics of how systems Heavy dependence on heuristics of how systems thinking developsthinking develops

Introduction

* Methods * Results


Literature ReviewLiterature Reviewand Existing Theoryand Existing Theory

Scant Literature on Systems Thinking Development


ResearchResearchMethodsMethodsLiterature review

Inductive

Exploratory

Pilot Interviews (N=12)

Additional Interviews with “Blue

Chip” Proven Experts (N=2)

(c) Expert Panelists (N=37)

• Completed survey and interview

• Identified subjects for 3 follow-on groups

(d) Follow-On Subjects• Completed interview

• Completed survey

(a) Contacted Company

• 10 companies participated

• Primarily U.S. aerospace companies

(b) Point-of-contactIdentified POC to work with others to identify

Expert Panelists

1. Senior Systems Engineers (N=62)

2. Senior Technical Specialists (N=53)

3. Junior Systems Engineers (N=53)

(Total of 205 interviews and 188 surveys)

Field Study

Data Analysis Using QSR N6, SPSS, MS Excel

Theory Synthesis


Company Site System Context

The Aerospace Corporation

Systems Engineering in Chantilly, VA & Los Angeles, CA

FFRDC - Global Positioning System (GPS), Air Force Satellite Communications (AFSATCOM) System, etc.1

BMW Systems Architects at BMW Group in Munich, Germany

Commercial - Manufacturer of premium automobiles and motorcycles2

Boeing Boeing Commercial Airplanes, Engineering Liaison group in Renton and Everett, Washington

Contractor - Commercial jetliner manufacturer3

Booz Allen Hamilton

Systems group, multiple locations, referred by a systems partner at headquarters in McLean, VA

Consultant - Strategic management and technology consulting firm to industry and government4

General Dynamics Sites 1 & 2

SE at General Dynamics Advanced Information Systems in Bloomington, MN and in Pittsfield, MA

Contractor - Provider of transformational mission solutions in command, control, communications, computers, intelligence, surveillance and reconnaissance (i.e. Future Combat Systems)5

MITRE Systems Engineering in Bedford, MA & McLean, VA

FFRDC - Global Information Grid, IRS enterprise modernization program, etc.6

Northrop Grumman

Airborne Ground Surveillance & Battle Management Systems, Integrated Systems, Melbourne, FL, SE

Contractor - E-8C Joint Surveillance Targeting Attack Radar System (Joint STARS), Cyber Warfare Integration Network (CWIN), etc.7

Pratt & Whitney

SE in East Hartford, CT Contractor - Design, manufacture, and support of turbine engines8

Sikorsky SE in Stratford, CT Contractor - Design and build advanced helicopters for commercial, industrial and military use9

Participating CompaniesParticipating Companies


Coding in QSR N6Coding in QSR N6

Q: How to make sense of 205 interviews each with a 4-5 Q: How to make sense of 205 interviews each with a 4-5 page transcript?page transcript?

A: Use “content analysis” to categorize key ideas and A: Use “content analysis” to categorize key ideas and thoughts from the interviewthoughts from the interview– This categorization process is called “coding”This categorization process is called “coding”– The resulting categories are called “nodes”The resulting categories are called “nodes”– The nodes were recorded and organized in a The nodes were recorded and organized in a

qualitative data management tool called QSR N6qualitative data management tool called QSR N61010

– Nodes were organized in hierarchies, with Level 2 Nodes were organized in hierarchies, with Level 2 as a sub-node of Level 1as a sub-node of Level 1

Content Analysis Performed Using QSR N6 Tool

Introduction

* Methods * Results



Introduction

* Methods * Results


Content Analysis:Content Analysis:From Raw Data to NodesFrom Raw Data to Nodes

Question Response

Node Hierarchies Organize Conceptual Patterns

Screen Shot

• Coded As:Coded As:– Level 1 Node - “Experience”Level 1 Node - “Experience”– Level 2 Node - “Job/opportunity to see Level 2 Node - “Job/opportunity to see

systems view”systems view”• Individual lines of ~1000 pages of Individual lines of ~1000 pages of transcripts coded in this waytranscripts coded in this way

• Yield of 908 nodesYield of 908 nodes


Additional Additional Data AnalysisData Analysis

• Interview data exported from QSR N6 to MS Excel Interview data exported from QSR N6 to MS Excel to determine top interview responsesto determine top interview responses

• Interview data exported from QSR N6 to SPSS to Interview data exported from QSR N6 to SPSS to run statistical testsrun statistical tests

• Results reported at both Level 1 and Level 2 of Results reported at both Level 1 and Level 2 of the node hierarchy to address aggregation biasthe node hierarchy to address aggregation bias

• SPSS used to analyze survey dataSPSS used to analyze survey data• Manual content analysis performed on pilot Manual content analysis performed on pilot

interviews and blue chip interviews interviews and blue chip interviews

Utilized Multiple Data Exploration Methods

Introduction

* Methods * Results



Underlying Underlying Research ResultResearch Result

Even though systems thinking definitions

diverge, there is consensus on primary

mechanisms that enable or obstruct systems

thinking development in engineers

IntroductionMethods

* Results * ImplicationsConclusion


Consensus on Primary Consensus on Primary Enabling MechanismsEnabling Mechanisms

• There is consensus on primary There is consensus on primary mechanisms that enable systems mechanisms that enable systems thinking development in engineersthinking development in engineers

1.1. Experiential learningExperiential learning

2.2. Individual characteristicsIndividual characteristics

3.3. Supportive environmentSupportive environment

Data Show Consensus

IntroductionMethods



Q: How can people agree on mechanisms that enable Q: How can people agree on mechanisms that enable systems thinking when their definitions of systems systems thinking when their definitions of systems thinking do not agree?thinking do not agree?

A: Though the articulation of the systems thinking A: Though the articulation of the systems thinking definitions diverge, there are common themes:definitions diverge, there are common themes:

(a) Functions and behaviors at the (a) Functions and behaviors at the contextual edgecontextual edge

(b) (b) InteractionsInteractions of elements and how large scale things relate of elements and how large scale things relate

The primary mechanisms cited enable and encourageThe primary mechanisms cited enable and encourage(a) Translation across contextual edges(a) Translation across contextual edges

(b) Consideration of interactions(b) Consideration of interactions

(c) Higher impact learning (c) Higher impact learning

Solving the PuzzleSolving the Puzzle Introduction

Methods



• Expert Panelists and follow-on subjects Expert Panelists and follow-on subjects were asked:were asked:

– ““How do you define systems thinking?”How do you define systems thinking?”– Considering a given systems thinking definition, what Considering a given systems thinking definition, what

aspects do you agree or disagree with and whyaspects do you agree or disagree with and why

• 205 interviews, 205 unique definitions205 interviews, 205 unique definitions• Data show that when people refer to the Data show that when people refer to the

phrase “systems thinking” they are often phrase “systems thinking” they are often not articulating the same conceptnot articulating the same concept

Divergent Systems Divergent Systems Thinking DefinitionsThinking Definitions

Systems Thinking Definitions Diverge

IntroductionMethods


Interview Question

s


Example Systems Example Systems Thinking DefinitionsThinking Definitions

• “Big picture”• “Interactions”• “Worrying about everything”• “System thinking is the ability to think about a system or system architecture

holistically, considering the design elements, complexities, the “ilities”, the context that product or system will be used in, etc.”

• “You have to think extremely broadly. You can’t focus on a specific aspect. Think from the application of what a product is. Think from what the customer wants explicitly. Be able to think in all the areas that are related to that device. It’s broad and deep thinking. If you can’t do both, then you shouldn’t do systems stuff. You must be organized. Think without boundaries at the start. If you think that your job is the requirements, then you are a clerk, not a systems engineer.”

• “Connecting lots of dissimilar disciplines and weighing trade offs between them…”

IntroductionMethods


More Definitions


Reconciling Systems Reconciling Systems Thinking DefinitionsThinking Definitions

• Synthesis of the definitions from the field study and the definitions Synthesis of the definitions from the field study and the definitions in the literature yielded an original framework of systems thinkingin the literature yielded an original framework of systems thinking

• Five foundational elements:Five foundational elements:1.1. COMPONENTIAL - What types of things are consideredCOMPONENTIAL - What types of things are considered

2.2. RELATIONAL - Interconnections, interactions, and interdependencies RELATIONAL - Interconnections, interactions, and interdependencies both within the system of interest and between the system of interest both within the system of interest and between the system of interest and other systemsand other systems

3.3. CONTEXTUAL – The nested and embedded nature of systemsCONTEXTUAL – The nested and embedded nature of systems

4.4. DYNAMIC – Links system in time to future and past, includes feedback, DYNAMIC – Links system in time to future and past, includes feedback, uncertainty, risk, and the “ilities”uncertainty, risk, and the “ilities”

5.5. MODAL – Aids to understand and comprehend systemMODAL – Aids to understand and comprehend system

• Systems thinking is utilizing modal elements to consider the Systems thinking is utilizing modal elements to consider the componential, relational, contextual, and dynamic elements of the componential, relational, contextual, and dynamic elements of the system of interest. system of interest.

IntroductionMethods



IntroductionMethods


Conceptual IllustrationConceptual Illustrationof Systems Thinkingof Systems Thinking

© 2005 Andreas Davidz, Elizabeth Davidz, Heidi Davidz. All rights reserved. Used with permission.

Tools &Methods

Types ofThinking

Models &Simulations

Processes &Frameworks


Coding ResultsCoding Results

““Systems Thinking Mindset”Systems Thinking Mindset”• This MUST be decomposed, since understandings can be This MUST be decomposed, since understandings can be

contradictorycontradictory• Before designing an intervention, know what you are trying to Before designing an intervention, know what you are trying to

produceproduce

System-of-Systems SE Traits System-of-Systems SE Traits Not detail focusedNot detail focused

Thinks out-of-the-boxThinks out-of-the-boxCreativeCreative

Abstract thinkingAbstract thinking

Process-Centered SE TraitsProcess-Centered SE TraitsDetail orientedDetail oriented

StructuredStructuredMethodicalMethodicalAnalyticalAnalytical

Define the Goal then Design the Intervention

IntroductionMethods



Difficulties with Difficulties with Determining Strength of Determining Strength of Systems ThinkingSystems Thinking

Systems Thinking Definitions Diverge

IntroductionMethods


Divergent Definitions

• Divergent systems thinking definitions are Divergent systems thinking definitions are problematic since strength of systems problematic since strength of systems thinking is determined by observation and thinking is determined by observation and subjective measure subjective measure

• In addition, many of the respondents do not In addition, many of the respondents do not know how strength of systems thinking is know how strength of systems thinking is determined in their organizationdetermined in their organization


IntroductionMethods


How does your company determine if an How does your company determine if an employee displays strong systems thinking?employee displays strong systems thinking?

Difficulty

Observation & Subjective Measure

Level

Determination of Strength Determination of Strength of Systems Thinkingof Systems Thinking


Subjective Determination of Subjective Determination of Strength of Systems ThinkingStrength of Systems Thinking

IntroductionMethods


How does your company determine if an How does your company determine if an employee displays strong systems thinking?employee displays strong systems thinking?

Do not know

Observation& Subjective Measures


Consensus on Consensus on EnablersEnablers

IntroductionMethods


Consensus on primary mechanisms that enable or obstruct systems thinking

development in engineers

1. Experiential learning2. Individual characteristics3. Supportive environment






IntroductionMethods


Experiential Learning Experiential Learning Develops Systems ThinkingDevelops Systems Thinking

Remarkable Consensus for Data Solicitation Format

Q: What were key steps in your life that developed your systems thinking abilities?Q: What were key steps in your life that developed your systems thinking abilities?


Top Ranked Categories Are All Experiential Learning

Experiential Learning – Inside and Outside WorkExperiential Learning – Inside and Outside Work


IntroductionMethods



Q: In your experience, what enablers or barriers have you seen to the Q: In your experience, what enablers or barriers have you seen to the development of systems thinking in engineers?development of systems thinking in engineers?

Top Node Category for “Enablers” is Experiential Learning


IntroductionMethods



3 of 4 Top Node Categories Are Experiential Learning


• Blue chip interviewees also support experiential learningBlue chip interviewees also support experiential learning““When I was involved in the mid-60s, programs went from concept to operation in 3-5 years. When I was involved in the mid-60s, programs went from concept to operation in 3-5 years. In a period of 15 years of experience, an engineer would work on 3-5 programs. They would In a period of 15 years of experience, an engineer would work on 3-5 programs. They would work up progressively to larger and larger responsibilities. work up progressively to larger and larger responsibilities. There was a whittling down There was a whittling down process so that we could pick the systems engineer. There would be 3-5 programs with 4-5 process so that we could pick the systems engineer. There would be 3-5 programs with 4-5 segments each, so we could pick the systems engineers for the new programs from this pool.segments each, so we could pick the systems engineers for the new programs from this pool. We would have 3 to 5 to 8 people to pick from, and we could pick the best. We would have 3 to 5 to 8 people to pick from, and we could pick the best.

We never had a problem with training, since this was provided by on-the-job training and We never had a problem with training, since this was provided by on-the-job training and experience.experience. We never thought about setting up training until the 2001 timeframe when we We never thought about setting up training until the 2001 timeframe when we thought about how to fix the problems in space acquisition…thought about how to fix the problems in space acquisition…

The training was all on-the-job. We would have young guys work on a section of the program, The training was all on-the-job. We would have young guys work on a section of the program, then they would move up to be in charge of a particular element, then they would work there then they would move up to be in charge of a particular element, then they would work there for 4-5 years, then they would move to a subsystem level, then they would move up to be for 4-5 years, then they would move to a subsystem level, then they would move up to be responsible for a segment of the programresponsible for a segment of the program. Each time, we could pick from 5-8 engineers to . Each time, we could pick from 5-8 engineers to move up to the position at that higher level.”move up to the position at that higher level.”

IntroductionMethods





IntroductionMethods










IntroductionMethods


Q: Are there certain individual characteristics or innate traits that seem to Q: Are there certain individual characteristics or innate traits that seem to predict the development of systems thinking? If so, what are they?predict the development of systems thinking? If so, what are they?

Individual Characteristics Individual Characteristics Enable Systems ThinkingEnable Systems Thinking

Personality is Top Node Category for Individual Characteristics


IntroductionMethods


Individual CharacteristicsIndividual CharacteristicsEnable Systems ThinkingEnable Systems Thinking


IntroductionMethods


Note: Junior Systems Note: Junior Systems Engineers add “Anxiety”Engineers add “Anxiety”

High in:High in:• Openness to Openness to

IdeasIdeas• CompetenceCompetence

Low In:Low In:• Self-Self-

ConsciousnesConsciousnesss

• Tender-Tender-MindednessMindedness

Different SampleDifferent Sample

Explanation of “Openness to Ideas”

Background on NEO PI-R™

Results of NEO PI-R™ Results of NEO PI-R™ Personality TestPersonality Test1212 from from One CompanyOne Company


IntroductionMethods


• Results correlate to findings by the Interdisciplinary Studies Project at Project Zero, Harvard Graduate School of Education led by Howard Gardner and Veronica Boix-Mansilla13,14

• “At the individual intellectual level, the paper characterizes exemplary interdisciplinary workers as embodying a disposition toward curiosity, risk-taking, open mindedness and humility.”

• “Curiosity in multiple areas of knowledge was a mobilizing force for the interdisciplinary workers in our study. Curiosity emerged implicitly in their accounts of professional growth as well as explicitly as a driving force of interdisciplinary work.”

• “Open-mindedness is the second trait repeatedly attributed to interdisciplinary workers and collaborators.”

Outside Study Also Emphasizes Curiosity and Open-Mindedness

Link to Interdisciplinary Link to Interdisciplinary Studies ProjectStudies Project



IntroductionMethods










IntroductionMethods


3 of 5 Top Barriers Are Environmental

Q: In your experience, what enablers or barriers have you seen to the Q: In your experience, what enablers or barriers have you seen to the development of systems thinking in engineers?development of systems thinking in engineers?

Environment Affects Environment Affects Systems ThinkingSystems Thinking


IntroductionMethods


Environment Affects Environment Affects

Systems ThinkingSystems Thinking

Organizations Shape These Node Categories


IntroductionMethods


Differences Are Not Significant Most of the Time

Statistical TestsStatistical Tests

• Multiple statistical tests run to compare differences between groups– Comparison of all classifications– Comparison of Senior Systems Engineers to:

The Expert Panelists The control group of Senior Technical Specialists The control group of Junior Systems Engineers

– Comparison of all companies– Comparison of two opposing companies

• Results show that the differences between groups are not significant most of the time

• The Senior Systems Engineers do not differ from the other classifications for the majority of the top-ranked node categories


Need for Systems Need for Systems OpportunitiesOpportunities

IntroductionMethodsResults

*Implications*Conclusion

New United States Military Aircraft Programs by Decade and Career Lengths of a Typical Engineer (From Murman, Walton et al. 2003, citing Hernandez)15

Declining Opportunities for Experiential Learning




Inappropriate Emphasis Inappropriate Emphasis on Trainingon Training1616

Emphasis is on Training Not Experiential Learning




Intervention MaturityIntervention Maturity




Systems Thinking Interventions Should Be Based on Knowledge and Include Feedback Mechanisms

Intervention MaturityIntervention Maturity


ImplicationsImplicationsfor Governmentfor Government



• Applications of Research for GovernmentApplications of Research for Government1.1. INCENTIVESINCENTIVES - Provide incentives to promote strong systems - Provide incentives to promote strong systems

thinkingthinking

2.2. POLICYPOLICY - Adjust policies to emphasize experiential learning for - Adjust policies to emphasize experiential learning for systems thinking developmentsystems thinking development

3.3. ACQUISITION STRATEGYACQUISITION STRATEGY - Change acquisition strategy to - Change acquisition strategy to provide more programs and opportunities for engineers to provide more programs and opportunities for engineers to develop systems thinkingdevelop systems thinking

4.4. RESEARCHRESEARCH - Promote research on the mechanisms for effective - Promote research on the mechanisms for effective systems thinking developmentsystems thinking development

5.5. SYSTEMS PROGRAMSSYSTEMS PROGRAMS - Encourage systems programs that - Encourage systems programs that teach systems skills and systems thinkingteach systems skills and systems thinking

Set Policy Environment to Enable Systems Thinking Development


Implications Implications for Industryfor Industry



• Applications of Research for IndustryApplications of Research for Industry1.1. INTERVENTION STRUCTUREINTERVENTION STRUCTURE - Structure systems thinking - Structure systems thinking

interventions to emphasize experiential learninginterventions to emphasize experiential learning

2.2. FILTER AND FOSTERFILTER AND FOSTER - Filter and foster identified individual - Filter and foster identified individual characteristics in systems organizationscharacteristics in systems organizations

3.3. SUPPORTIVE ENVIRONMENTSUPPORTIVE ENVIRONMENT - Provide an environment - Provide an environment supportive to the development of systems thinkingsupportive to the development of systems thinking

4.4. COMMUNICATE ASSESSMENTCOMMUNICATE ASSESSMENT - Clearly communicate how - Clearly communicate how strength of systems thinking is assessedstrength of systems thinking is assessed

5.5. SYSTEMS PROGRAMSSYSTEMS PROGRAMS - Offer systems programs to teach - Offer systems programs to teach systems skills and systems thinkingsystems skills and systems thinking

Utilize the Primary Mechanisms That Enable Systems Thinking


ImplicationsImplicationsfor Academiafor Academia



• Applications of Research for AcademiaApplications of Research for Academia1.1. SYSTEMS PROGRAMSSYSTEMS PROGRAMS - Offer systems programs to teach - Offer systems programs to teach

systems skills and systems thinkingsystems skills and systems thinking

2.2. FEEDBACKFEEDBACK - Use feedback mechanisms to continually improve - Use feedback mechanisms to continually improve systems programs and systems coursessystems programs and systems courses

3.3. EMPHASIZE EXPERIENCEEMPHASIZE EXPERIENCE - Structure programs and courses to - Structure programs and courses to emphasize experiential learningemphasize experiential learning

4.4. COURSE STRUCTURECOURSE STRUCTURE - Structure courses and programs to - Structure courses and programs to promote systems thinking by emphasizing context and knowledge promote systems thinking by emphasizing context and knowledge integrationintegration

5.5. RESEARCHRESEARCH - Continue research on the mechanisms for effective - Continue research on the mechanisms for effective systems thinking developmentsystems thinking development

Continue Studying How Systems Thinking Actually Develops


SummarySummaryIntroduction

MethodsResults

Implications

* Conclusion *

• Exploratory and inductive study• Field study with auxiliary interviews• Result: Even though systems thinking definitions diverge, there

is consensus on primary mechanisms that enable or obstruct systems thinking development in engineers

• Divergent systems thinking definitions reconciled with a systems thinking framework, illustration and definition

• Highlights importance of experiential learning• Development is enabled by individual characteristics such as openness to

ideas, curiosity, questioning, strong communication and interpersonal skills• A supporting environment also enables development

• Implications for government, industry, and academia given

Rigorous Exploration of an Extensive Data Set to DiscoverEffective Mechanisms to Develop Systems Thinking in Engineers


Enabling Rigor Enabling Rigor in SE Researchin SE Research


Implications

* Conclusion *

• Many engineers are not familiar with Many engineers are not familiar with research methods applicable to studying research methods applicable to studying systems problemssystems problems

• Ideas for enhancing academic rigor in Ideas for enhancing academic rigor in systems engineering researchsystems engineering research

1.1. SE Research Methods Tutorials SE Research Methods Tutorials 2.2. SE Research Methods Task Force SE Research Methods Task Force 3.3. SE Research Crits SE Research Crits

Ideas for Enabling SE Research Rigor


1)1) http://www.aero.orghttp://www.aero.org2)2) http://www.bmw.comhttp://www.bmw.com3)3) http://www.boeing.comhttp://www.boeing.com4)4) http://www.boozallenhamilton.comhttp://www.boozallenhamilton.com5)5) http://www.generaldynamics.comhttp://www.generaldynamics.com6)6) http://www.mitre.org http://www.mitre.org 7)7) http://www.northgrum.comhttp://www.northgrum.com8)8) http://www.pratt-whitney.comhttp://www.pratt-whitney.com9)9) http://www.sikorsky.comhttp://www.sikorsky.com10)10) QSR N6 Student Mini-Manual, copyright by QSR International Pty. Ltd. Melbourne, Australia, March QSR N6 Student Mini-Manual, copyright by QSR International Pty. Ltd. Melbourne, Australia, March

2002.2002.11)11) Maier, M. W. and E. Rechtin, “The Art of Systems Architecting,” CRC Press LLC, 2002.Maier, M. W. and E. Rechtin, “The Art of Systems Architecting,” CRC Press LLC, 2002.12)12) Costa, J., Paul T. and R. R. McCrae, “NEO PI-R Professional Manual, Revised NEO Personality Costa, J., Paul T. and R. R. McCrae, “NEO PI-R Professional Manual, Revised NEO Personality

Inventory (NEO PI-R) and NEO Five-Factor Inventory (NEO-FFI),” Psychological Assessment Inventory (NEO PI-R) and NEO Five-Factor Inventory (NEO-FFI),” Psychological Assessment Resources, Inc., 1992.Resources, Inc., 1992.

13)13) http://www.pz.harvard.edu/interdisciplinary/research.html, 2006.http://www.pz.harvard.edu/interdisciplinary/research.html, 2006.14)14) Mansilla, Veronica Boix, Dan Dillon, and Kaley Middlebrooks, “Building Bridges Across Disciplines: Mansilla, Veronica Boix, Dan Dillon, and Kaley Middlebrooks, “Building Bridges Across Disciplines:

Organizational and Individual Qualities of Exemplary Interdisciplinary Work,” Interdisciplinary Studies Organizational and Individual Qualities of Exemplary Interdisciplinary Work,” Interdisciplinary Studies Project, Project Zero, Harvard Graduate School of Education, 2000. Project, Project Zero, Harvard Graduate School of Education, 2000.

15)15) Murman, E., M. Walton, et al. "Challenges in the Better, Faster, Cheaper Era of Aeronautical Design, Murman, E., M. Walton, et al. "Challenges in the Better, Faster, Cheaper Era of Aeronautical Design, Engineering and Manufacturing." Massachusetts Institute of Technology Engineering Systems Engineering and Manufacturing." Massachusetts Institute of Technology Engineering Systems Division White Paper, paper to appear in The Aeronautical Journal, 2003, citing HernandezDivision White Paper, paper to appear in The Aeronautical Journal, 2003, citing Hernandez ..

16)16) Skalamera, R. J., “Implementing OSD Systems Engineering Policy,” 2004.Skalamera, R. J., “Implementing OSD Systems Engineering Policy,” 2004.

ReferencesReferences

Thank You!Thank You!

Questions or Comments?Questions or Comments?

Enabling Systems Thinking to Accelerate the Development of Senior Systems Engineers INCOSE Presentation February 2007 Heidi L. Davidz.

Documents

development of systems

systems dynamics

developing systems

developsscant literature

engineersenabling mechanisms

senior systemsengineers

additional interviews

interviews surveys