Top Banner
1 Stevens Ins)tute of Technology & Systems Engineering Research Center (SERC) Transforming Systems Engineering through a Holis)c Approach to ModelCentric Engineering Presented to: NDIA 2015 By: Dr. Mark R. Blackburn Dr. Mary Bone Dr. Gary Witus
27

StevensInstuteofTechnology Systems’Engineering’Research ... · StevensInstuteofTechnology & Systems’Engineering’Research’Center’ ......

Aug 25, 2018

Download

Documents

truongbao
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: StevensInstuteofTechnology Systems’Engineering’Research ... · StevensInstuteofTechnology & Systems’Engineering’Research’Center’ ... discipline!engineers!during!detailed!design!process!with!

                                       1

Stevens  Ins)tute  of  Technology    &  

Systems  Engineering  Research  Center  (SERC)  

Transforming  Systems  Engineering  through  a  Holis)c  Approach  to  Model-­‐Centric  Engineering    

Presented  to:  NDIA  2015  By:  

Dr.  Mark  R.  Blackburn  Dr.  Mary  Bone    Dr.  Gary  Witus  

 

Page 2: StevensInstuteofTechnology Systems’Engineering’Research ... · StevensInstuteofTechnology & Systems’Engineering’Research’Center’ ... discipline!engineers!during!detailed!design!process!with!

                       Mark  R.  Blackburn,  Ph.D.                        2

Outline  

• Context,  Problem  and  Objec>ves  

• Four  Tasks  

• Perspec>ves  on  findings  –  extends  informa>on  from  NDIA  2014  

• Conclusions  

• Acknowledgments  

•  Image  credits  

Page 3: StevensInstuteofTechnology Systems’Engineering’Research ... · StevensInstuteofTechnology & Systems’Engineering’Research’Center’ ... discipline!engineers!during!detailed!design!process!with!

                       Mark  R.  Blackburn,  Ph.D.                        3

Problem  Statement  

•  It   takes   too   long   to  bring   large-­‐scale   air   vehicle   systems   from  concept  to  opera>on  

•  NAVAIR   is   par>ally   constrained   by   their   own   monolithic,  serialized,  paper-­‐driven  process  

Page 4: StevensInstuteofTechnology Systems’Engineering’Research ... · StevensInstuteofTechnology & Systems’Engineering’Research’Center’ ... discipline!engineers!during!detailed!design!process!with!

                       Mark  R.  Blackburn,  Ph.D.                        4

Study  Objec)ves  

Primary  ques>on  Is   it   Technically   Feasible   to   have   a   Radical   Transforma)on  through  Model  Based  Systems  Engineering  (MBSE)  and  achieve  a   25   percent   reduc)on   in   the   )me   to   develop   large-­‐scale   air  vehicle  system?    

Corollary  How   do   we   know   that   models/simula>ons   used   to   assess  Performance   have   the   needed   Integrity   to   ensure   predic>ons  are  accurate  (i.e.,  that  we  can  trust  the  models)?  

Page 5: StevensInstuteofTechnology Systems’Engineering’Research ... · StevensInstuteofTechnology & Systems’Engineering’Research’Center’ ... discipline!engineers!during!detailed!design!process!with!

                       Mark  R.  Blackburn,  Ph.D.                        5

Sponsor’s  Vision  at  Kickoff  Mee)ng:  Cross-­‐Domain,  Mul)-­‐Physics,  Models  Integra)on  

Con>nuous  refinement  of  models  through  cross-­‐domain  &  mul>disciplinary  analysis  suppor>ng  virtual  V&V  from  CONOPS  to  manufacturing  

Integrated  Environment  to  Produce  Digital  System  Model:  Single  Source  of  Technical  Truth  

Page 6: StevensInstuteofTechnology Systems’Engineering’Research ... · StevensInstuteofTechnology & Systems’Engineering’Research’Center’ ... discipline!engineers!during!detailed!design!process!with!

                       Mark  R.  Blackburn,  Ph.D.                        6

Four  Tasks  to  Assess  Technical  Feasibility  of    “Doing  Everything  with  Models”  (Everything  Digital)  

2) Develop Common Lexicon for Model Levels, Types, Uses, and Representations

1) Global scan and classification of holistic state-of-the-art MBSE

3) Model the Vision of Everything Done with Models and Relate to “As Is” process

4) Fully integrate model-driven Risk Management and Decision Making

•  Use discussion framework to survey government, industry and academia

•  Quantify, link and trace realized modeling capabilities to Vision (task 3)

Campaign  

Mission  

Engagement  

Engineering  

Model Types

Structure/Interfaces

Behavior (functions)

Concurrency

Resources/Environment

Address two classes of risk: •  Airworthiness and

Safety •  Program Execution

Page 7: StevensInstuteofTechnology Systems’Engineering’Research ... · StevensInstuteofTechnology & Systems’Engineering’Research’Center’ ... discipline!engineers!during!detailed!design!process!with!

                       Mark  R.  Blackburn,  Ph.D.                        7

Task  1:  Industry,  Government  and  Academia  Visits  and  Discussions  

• We  had  open-­‐ended  discussions    Tell  us  about  the  most  advanced  and  holis)c  approach  to  model-­‐centric  engineering  you  use  or  seen  used  

• Did  not  single  out  specific  companies  

• Spectrum  of  informa>on  was  very  broad  

• There  really  is  no  good  way  to  make  a  comparison  

• We  have  a  report  that  summarizes  the  aggregate  of  what  we  heard  

Page 8: StevensInstuteofTechnology Systems’Engineering’Research ... · StevensInstuteofTechnology & Systems’Engineering’Research’Center’ ... discipline!engineers!during!detailed!design!process!with!

                       Mark  R.  Blackburn,  Ph.D.                        8

• Organiza>onal  discussed:  ―   Model-­‐Based  Engineering  (MBE),  Integrated  Model-­‐Centric  Engineering,    Interac>ve  Model-­‐centric  Systems  Engineering  (IMCSE),  Model-­‐Driven  Development,  Model-­‐Driven  Engineering  (MDE),  and  even  Model-­‐Based  Enterprise,  which  brings  in  more  focus  on  manufacturability  

― Digital  Thread  envisions  frameworks  that  merges  physics-­‐based  models  generated  by  (cross)discipline  engineers  during  detailed  design  process  with  MBSE’s  conceptual  and  top-­‐level  architectural  models,  resul>ng  in  a  single  authorita>ve  representa>on  of  the  system  

• MCE  characterizes  the  goal  of  integra>ng  different  model  types  with  simula>ons,  surrogates,  systems  and  components  at  different  levels  of  abstrac>on  and  fidelity  across  discipline  throughout  the  lifecycle  with  manufacturability  constraints  

• We  could  have  used  the  words  Digital  Engineering,  which  we  have  heard  used  too  

 

Model  Based  System  Engineering  (MBSE)  versus  Model-­‐Centric  Engineering  (MCE)  

Page 9: StevensInstuteofTechnology Systems’Engineering’Research ... · StevensInstuteofTechnology & Systems’Engineering’Research’Center’ ... discipline!engineers!during!detailed!design!process!with!

                       Mark  R.  Blackburn,  Ph.D.                        9 **Derived from Ernest S. "Turk" Tavares, Jr. and Larry Smith

Surrogates, traditional materials, hardware, processes

Base airframe with some advanced materials (composites) hardware (SIL assets)

Final Config: advanced materials (composites/exotics) advanced

hardware, final avionics

Phase:   SRR SFR PDR CDR

V&V  Focus:  

Operational level

models

High level performance. (Aero,

some P&FQ)

Macro-level integration, some system functionality,

full P&FQ

Full integration and systems functionality

Design/  Payload  Maturity:  (w/Models)  

High level need: Aircraft Mid level need:

take off, land, fly Lower level need:

Employ legacy weapons Lowest level need: employ advanced

weapons; stealth, etc.

Use  Dynamic  Models  and  Surrogates  to  Support  Con)nuous  “virtual  V&V”  

•  Integra>on  of  computa>onal  capabili>es,  models,  sogware,  hardware,  plahorms,  and  humans-­‐in-­‐the-­‐loop  allows  us  to  assess  the  system  design  in  the  face  of  changing  mission  needs  

Page 10: StevensInstuteofTechnology Systems’Engineering’Research ... · StevensInstuteofTechnology & Systems’Engineering’Research’Center’ ... discipline!engineers!during!detailed!design!process!with!

                       Mark  R.  Blackburn,  Ph.D.                        10

DARPA  META  Concept  

• More  con>nuous  and  itera>ve  using  successive  refinement  of  tradespace  alterna>ves,  with  considera>ons  for  manufacturability  leading  to  “executable  requirements”  with  con>nuous  test  at  increasing  levels  of  fidelity  

Page 11: StevensInstuteofTechnology Systems’Engineering’Research ... · StevensInstuteofTechnology & Systems’Engineering’Research’Center’ ... discipline!engineers!during!detailed!design!process!with!

                       Mark  R.  Blackburn,  Ph.D.                        11

Are  we  nearing  a  )pping  point  driven  by  the  Industrial  Internet?  

• Mission-­‐level  simula>ons  are  being  integrated  with  system  simula>on,  digital  assets  &  products  providing  a  new  world  of  services  

Page 12: StevensInstuteofTechnology Systems’Engineering’Research ... · StevensInstuteofTechnology & Systems’Engineering’Research’Center’ ... discipline!engineers!during!detailed!design!process!with!

                       Mark  R.  Blackburn,  Ph.D.                        12

Leaders  are  Embracing  Change  and  Adap)ng  To  Use  Digital  Strategies  Faster  Than  Others  

• Enabling  digital  technologies  are  changing  how  companies  are  doing  business  using  models-­‐centric  engineering  

• They  use  model-­‐centric  environments  for  customer  engagements,  but  also  for  design  engineering  analysis  and  review  sessions  

Page 13: StevensInstuteofTechnology Systems’Engineering’Research ... · StevensInstuteofTechnology & Systems’Engineering’Research’Center’ ... discipline!engineers!during!detailed!design!process!with!

                       Mark  R.  Blackburn,  Ph.D.                        13

There  are  modeling  environments  to  Create  Dynamic  Opera)onal  Views  (OV1)  

•  Increasing  need  for  integra>on  to  beker  understand  and  characterize  Mission  Context  for  the  needed  System  Capabili>es  

Page 14: StevensInstuteofTechnology Systems’Engineering’Research ... · StevensInstuteofTechnology & Systems’Engineering’Research’Center’ ... discipline!engineers!during!detailed!design!process!with!

                       Mark  R.  Blackburn,  Ph.D.                        14

1D,  2D  &  3D  Models  have    Simula)on  and  Analysis  Capabili)es  

• Focused  primarily  on  physics-­‐based  design  with  increasing  support  for  cross-­‐domain  analysis  

Page 15: StevensInstuteofTechnology Systems’Engineering’Research ... · StevensInstuteofTechnology & Systems’Engineering’Research’Center’ ... discipline!engineers!during!detailed!design!process!with!

                       Mark  R.  Blackburn,  Ph.D.                        15

Plaaorm-­‐based  Approaches  with  Virtual  Integra)on  Help  Automakers  Deliver  Vehicle  Faster  

• Refresh  and  upgrades  on  periodic  schedules  are  business  cri>cal  

Page 16: StevensInstuteofTechnology Systems’Engineering’Research ... · StevensInstuteofTechnology & Systems’Engineering’Research’Center’ ... discipline!engineers!during!detailed!design!process!with!

                       Mark  R.  Blackburn,  Ph.D.                        16

Modeling  and  Simula)on  in  the  Automo)ve  Domain  is  Reducing  the  Physical  Crash  Tes)ng  

• NAVAIR  wants  to  know  if  it  is  feasible  to  assess  designs  earlier  and  more  con>nuously  by  flying  virtually  

Page 17: StevensInstuteofTechnology Systems’Engineering’Research ... · StevensInstuteofTechnology & Systems’Engineering’Research’Center’ ... discipline!engineers!during!detailed!design!process!with!

                       Mark  R.  Blackburn,  Ph.D.                        17

Organiza)ons  are  Modeling  and    Simula)ng  Manufacturing  Before  Tooling  

• Set-­‐based  delays  design  selec>on  and  increasingly  factors  in  manufacturability  

Page 18: StevensInstuteofTechnology Systems’Engineering’Research ... · StevensInstuteofTechnology & Systems’Engineering’Research’Center’ ... discipline!engineers!during!detailed!design!process!with!

                       Mark  R.  Blackburn,  Ph.D.                        18

Other  Enablers  for  the  End  State  

• A  tool  agnos>c  approach  to  share  seman>cally  rich  data  across  domains/disciplines…  ― Standard-­‐based  ontologies  provide  a  way  to  represent  knowledge  

• Computer  augmenta>on  ― Digital  assistance  will  understand  what  we  are  trying  to  model  through  advances  in  machine  learning  and  integrated  visualiza>on    

― Operate  as  knowledge  librarian  helping  us  to  model  some  aspects  of  the  problem  or  solu>on  at  an  accelera>ng  pace  

• Explosion  of  interac>ve  visualiza>ons  to  understand  data  and  informa>on  derived  from  a  “sea”  of  models  with  HPC  compu>ng  capabili>es  ― Key  relevance  related  to  a  “claimed  radical  transforma>on”  of  companies  that  changed  approach  to  decision  making  through  data  analy>cs  resul>ng  in  decisions  in  hours  vs.  weeks  

Page 19: StevensInstuteofTechnology Systems’Engineering’Research ... · StevensInstuteofTechnology & Systems’Engineering’Research’Center’ ... discipline!engineers!during!detailed!design!process!with!

                       Mark  R.  Blackburn,  Ph.D.                        19

Sociotechnical  Compu)ng  May  Help  Enable  Some  Aspects  of  a  Radical  Transforma)on  

Key  Contribu)on  

Asynchronous  collabora>on,  dynamic  workflow  management  

Observa)ons  

•  Emerging  impacts  of  the  Industrial  Internet  and  Social  Compu)ng  provides  mass  communica>on  of  all  forms,  enabling  a  new  type  of  dynamic  and  con)nuous  orchestra)on  of  work  and  informa>on  for  real-­‐)me  decision-­‐making  

•  Confluence  of  digital  technologies  evolving  at  an  accelera>ng  pace  through  massively  parallel  HPC  and  integra>ons  exemplified  by  the  Internet  of  Things  (IoT)  that  we  have  seen  in  discussions  is  manifes>ng  in  instances  realizing  the  Single  Source  of  Technical  Truth  

•  Emphasis  on  informa>on  that  needs  to  be  produced  and  less  about  process  –  model-­‐centricity  subsumes  the  process  –  we  have  evidence  

Page 20: StevensInstuteofTechnology Systems’Engineering’Research ... · StevensInstuteofTechnology & Systems’Engineering’Research’Center’ ... discipline!engineers!during!detailed!design!process!with!

                       Mark  R.  Blackburn,  Ph.D.                        20

Scope  of  Data  Collec)on  for  Task  1  (not  exhaus)ve)  

Discussion(Topics(not(exhaustive) N

ASA

/JPL

A B C Alta

ir

GE

Sand

ia

DARP

A5M

ETA5(V

B)

DARP

A5M

ETA5(B

AE)

Mod

el5Cen

ter

Autom

otive

CREA

TE

Performance

Integrity

Affordability

Risk

Metho

dology

Single5Sou

rce5of5Tech5Truth

Prioritization5&

Tradeo

ff5Analysis

Concep

t5Engineering

Architecture5&

Design5Analysis

Design5&5Test

Reuse5&5Synthesis

Active5System

Characterizatio

n

Hum

anMSystem

Integration

Modeling5CONOPS x x x x x x xModeling5Patterns x x x x x x x x xMultiMPhysics5Modeling5and5Simulation x x x x x x x x x x x x x x xMultiMDiscpline/Domain5Analysis5and5Optimization x x x x x x x x x x x x x x x x x x xMissionMtoMSystemMlevel5Simulation5Integration x x x x x x x x x x xAffordability5Analysis x x x x x x x x x xQuantification5of5Margins x x x x x x x x x x xRequirement5Generation5(from5Models) x x x x x x x xTool5agnostic5digital5representation x x x x x x x x x x xModel5measures5(thru5formal5checks) x x x x x x x x x xModeling5and5Sim5for5Manufacturability x x x x x x x x x x x x x xProcess5Automation5(workflows) x x x x x x xIterative/Agile5use5of5MCE x x x x x x xHigh5Performance5Computing x x x x x x x x x x x x x x xPlatformMbased5and5Surrogates x x x x x x x x3D5Environments5and5Visualization x x x x x x x x x x x x x x x xImmersive5Environments x x x x x xDomainMspecific5modeling5languages5 x x x x x x x x x x x x x x x xSetMbased5design5 x x x x x x x x xModel5validation/qualification/trust x x x x x x x xModeling5Environment5and5Infrastructure x x x x x x x x x x x x x x x x x x x x x x x x

Instances5where5discussed5(not5exhaustive) From5Kickoff5BriefingCharacteristics

Page 21: StevensInstuteofTechnology Systems’Engineering’Research ... · StevensInstuteofTechnology & Systems’Engineering’Research’Center’ ... discipline!engineers!during!detailed!design!process!with!

                       Mark  R.  Blackburn,  Ph.D.                        21

Integrated  Environment  for  Itera)ve  Tradespace  Analysis  of  Problem  and  Design  Space  

Mul>discipline  Design,    Analysis  and  Op>miza>on  (MDAO)  

Single  Source  of  Technical  Truth:  Tool  Agnos>c,  Seman>cally  Precise  Cross  Domain    Integra>on  &  Interoperability  enabled  by  HPC  

Performance   Integrity  

Secure  Plugin  

Cost  &  Schedule  

Systems,  Surrogates    &  Plahorms  

DocGen  

Appropriate Views for Stakeholders

Knowledge   …

Con>nuous  Workflow  

Orchestra>on  

Computer  Augmenta>on  

&  Training  

PLM  

Rich Modeling Interfaces “Web” Interface integrated

with Rich Visualizations

“Illi>es”  

Page 22: StevensInstuteofTechnology Systems’Engineering’Research ... · StevensInstuteofTechnology & Systems’Engineering’Research’Center’ ... discipline!engineers!during!detailed!design!process!with!

                       Mark  R.  Blackburn,  Ph.D.                        22

Holis)c  Model-­‐centric  Engineering  can  Enable,  But  will  Require  New  Types  of  Coordina)on  

•  In  a  “Digital  Engineering”  environment,  government  and  industry  need  to  work  in  a  different  way  

Page 23: StevensInstuteofTechnology Systems’Engineering’Research ... · StevensInstuteofTechnology & Systems’Engineering’Research’Center’ ... discipline!engineers!during!detailed!design!process!with!

                       Mark  R.  Blackburn,  Ph.D.                        23

Conclusions  

• Over  30  discussions  and  21  onsite  with  Industry,  Government  and  Academia,  with  follow-­‐ups  –  our  summary  is  not  exhaus>ve  

• Developed  common  lexicon  of  over  700  terms  for  model  levels,  types,  uses,  and  representa>ons,  with  many  contributors  

• Models  are  becoming  more  dynamic  and  integrated  across  domains,  as  opposed  to  sta>c  and  isolated,  enabled  by  HPC,  seman)c  precision,  and  visual  analy)cs  

• Several  strategies  have  been  developed  and  applied  for  quan)fica)on  of  model  confidence,  enabled  by  HPC  

• Answer  to  Sponsor:  It  is  technically  feasible  to  radically  transform  systems  engineering  at  NAVAIR  through  MCSE;  however,  the  evidence  does  not  show  conclusively  that  it  will  produce  a  25%  reduc>on  in  acquisi>on  cycle  >me.    

Page 24: StevensInstuteofTechnology Systems’Engineering’Research ... · StevensInstuteofTechnology & Systems’Engineering’Research’Center’ ... discipline!engineers!during!detailed!design!process!with!

                       Mark  R.  Blackburn,  Ph.D.                        24

Acknowledgment  

• We  wish  to  acknowledge  the  great  support  of  the  NAVAIR  sponsors  and  stakeholders,  including  stakeholders  from  other  industry  partners  that  have  been  very  helpful  and  open  about  the  challenges  and  opportuni>es  of  this  promising  approach  to  transform  systems  engineering.  

• We  want  to  specifically  thank  Dave  Cohen  who  established  the  vision  for  this  project,  and  our  NAVAIR  team,  Jaime  Guerrero,  Gary  Strauss,  Brandi  Gertsner,  and  Ron  Carlson,  who  has  worked  closely  on  a  weekly  basis  in  helping  to  collabora>vely  research  this  effort.  We  thank  Howard  Owens  and  Dennis  Reed  who  have  joined  us  in  some  of  the  organiza>onal  visits.  We  also  thank  Larry  Smith,  Ernest  (Turk)  Tavares,  Eric  (Tre´)  Johnsen,  who  worked  Phase  I  &  II  with  us,  but  have  leg  the  project.  

• We  have  had  over  30  discussions  with  organiza>ons  from  Industry,  Government,  and  Academia,  and  we  want  to  thank  all  of  those  stakeholders  (over  180  people),  including  some  from  industry  that  will  remain  anonymous  in  recogni>on  of  our  need  to  comply  with  proprietary  and  confiden>ality  agreements  associated  with  Task  1.    

Page 25: StevensInstuteofTechnology Systems’Engineering’Research ... · StevensInstuteofTechnology & Systems’Engineering’Research’Center’ ... discipline!engineers!during!detailed!design!process!with!

                       Mark  R.  Blackburn,  Ph.D.                        25

Thank  You  

• For  more  informa>on  contact:  ― Mark  R.  Blackburn,  Ph.D.  ― [email protected]  ― Stevens  Ins>tute  of  Technology  ― 703.431.4463  

• Bio:  Dr.  Mark  R.  Blackburn  is  an  Associate  Professor  with  Stevens  Ins>tute  of  Technology.  He  is  the  Principal  Inves>gator  (PI)  on  a  Systems  Engineering  Research  Center  (SERC)  research  task,  co-­‐PI  on  a  related  task  for  Quan>ta>ve  Technical  Risk,  and  has  been  the  PI  on  research  tasks  for  SERC,  Na>onal  Science  Founda>on,  Federal  Avia>on  Administra>on,  and  Na>onal  Ins>tute  of  Standards  and  Technology.  He  develops  and  teaches  a  new  course  on  Systems  Engineering  of  Cyber  Physical  Systems.  He  spent  11  years  building  flight-­‐cri>cal  avionics  sogware  and  applying  model-­‐based  sogware  tools,  which  overlaps  with  25+  years  in  building  modeling  and  analysis  tools,  and  doing  applied  research.  

Page 26: StevensInstuteofTechnology Systems’Engineering’Research ... · StevensInstuteofTechnology & Systems’Engineering’Research’Center’ ... discipline!engineers!during!detailed!design!process!with!

                       Mark  R.  Blackburn,  Ph.D.                        26

CONOPS  Concept  of  Opera>ons  

CDR  Cri>cal  Design  Review  

DARPA  Defense  Advanced  Research  Project    Agency  

DoD  Department  of  Defense  

HPC  High  Performance  Compu>ng  

IMCE  Integrated  Model-­‐Centric  Engineering  

IMCSE  Interac>ve  Model-­‐centric  Systems    Engineering  

IoT  Internet  of  Things  

MBSE  Model-­‐based  System  Engineering  

MBE  Model-­‐Based  Engineering  

MCE  Model-­‐Centric  Engineering  

MCSE  Model-­‐Centric  System  Engineering  

MDE  Model-­‐Driven  Engineering  

NAVAIR  Naval  Air  Systems  Command  

 

OV  Opera>onal  View  

P&FQ  Performance  and  Flight  Quality  

PDR  Preliminary  Design  Review  

PLM  Product  Lifecycle  Management  

SLOC  Sogware  Lines  Of  Code  

SE    Systems  Engineering  

SERC  System  Engineering  Research  Center  

SETR  Systems  Engineering  Technical  Review  

SFR  System  Func>onal  Review  

SRR  System  Requirements  Review  

SoS  System  of  Systems  

SV  System  View  

V&V  Verifica>on  and  Valida>on  

Acronyms  

Page 27: StevensInstuteofTechnology Systems’Engineering’Research ... · StevensInstuteofTechnology & Systems’Engineering’Research’Center’ ... discipline!engineers!during!detailed!design!process!with!

                       Mark  R.  Blackburn,  Ph.D.                        27

Image  Credits  •  Certain  commercial  products,  equipment,  instruments,  or  other  content  iden>fied  in  this  document  does  not  

imply  recommenda>on  or  endorsement  by  the  authors,  SERC,  or  NAVAIR,  nor  does  it  imply  that  the  products  iden>fied  are  necessarily  the  best  available  for  the  purpose.    

•  Image  credits  /  sources  Slide  #3:  Joe  Willeke,  Approved  for  public  release;  distribu>on  is  unlimited.  SPR  Number:  2014-­‐459.  

Slide  #5:  m.plm.automa>on.siemens.com,  mosimtec.com,  www.defenseindustrydaily.com,  www.darkgovernment.com  Slide  #8:  Henson  Graves  

Slide  #9:  www.fightercontrol.co.uk,  en.wikipedia.org,  en.wikipedia.org  

Slide  #10:  Bapty,  T.,  S.  Neema,  J.  Scok,  Overview  of  the  META  Toolchain  in  the  Adap>ve  Vehicle  Make  Program,  Vanderbilt,  ISIS-­‐15-­‐103,  2015.  Slide  #11:  blog.boq.com.au  

Slide  #12:  media.gm.com,  Modeling  and  Simula>on  Applied  in  the  F-­‐35  Program,  Barry  Evans  Lockheed  Mar>n  Aeronau>cs,  2011.  Slide  #13:  Image  credit:  AGI  

Slide  #14:  m.plm.automa>on.siemens.com  

Slide  #15:  itea3.org  Slide  #16:  y.tamu.edu  

Slide  #17:  mosimtec.com  Slide  #21:  www.defenseindustrydaily.com,  www.darkgovernment.com,  NAVAIR  

Slide  #22:  hkp://www.eonreality.com/hardware/