Systems Engineering and Project Management: Similarities and Differences for Cost Estimation Dr. Ricardo Valerdi – Massachusetts Institute of Technology.

Systems Engineering and Project Management: Similarities and

Differences for Cost Estimation

Dr. Ricardo Valerdi – Massachusetts Institute of Technology

Leone Young – Stevens Institute of Technology

COSYSMO Workshop

The University of Southern California (USC)November 4th, 2010

The 25th Int’l Forum on COCOMO & Systems/Software Cost Modeling 1

Workshop Agenda

• Introductions & Objectives • Research Background • Project Management Cost

Models and Feedback• Survey • Inputs & Discussion

2

1:30 – 1:35 pm

1:35 – 1:50 pm

1:50 – 2:05 pm

2:05 – 2:35 pm

2:35 – 2:50 pm

Objectives

• Discuss the similarities and differences between systems engineering and project management

• Obtain feedback on a proposed project management cost estimating model

• Conduct survey to determine the expected level of project management efficiency multipliers

• Obtain inputs on the appropriateness and adequacy of the proposed project management cost model, its cost factors and drivers

• Provide an opportunity for participants to contribute their expertise and perspective regarding project management services and influence the direction of future research

3

4

Research Background: Systems Engineering (SE) & Project

Management (PM)

•Cost Management – Project Air Force by Rand –Development Management = Systems Engineering (SE) & Project

Management (PM)–Defense Industry – US Air Force Programs (Stem et al., 2006)

» Development Management (100/%) = SE (50%) + PM (50%)»SE/PM costs doubled since 1960s

SE/PM as a function of Integrated Logistics Support (ILS) for a typical Air Force program (Stem, et al., 2006)

Aircraft SE/PM Costs as a Percentage of Total Development Cost for All Development Programs, 1960s–1990s (Stem, et al., 2006)

5


Management (PM)

•SE Costs – significant amount of research has been conducted

–The International Council on Systems Engineering (INCOSE) surveyed (Honour, 2004):

•52% of systems projects spent 5% or less of total systems development cost on SE tasks

–The Constructive Systems Engineering Cost Model (COSYSMO)

•As a SE cost estimating tool used by systems engineering, systems cost estimators, etc (Valerdi, 2006)

•PM Cost Estimating?

–Methodology?

–Tools?

6

• Literature – limited information on PM related expenditures or costs

• Organizations often do not identify or measure PM costs, and a survey led by UC Berkeley (Ibbs and Kwak, 2000a, 2000b) shows:

– 80% of companies surveyed spend less than 10% of total project cost (TPC) for PM services

– Average = 6% of TPC, Range = 0.3% ~ 15% of TPC

– Another survey indicated the average = 10% of TPC (Ibbs and Reginato, 2002)

• Evidently, PM costs varies among organizations

– Influential PM Cost Factors: project type, size, # of projects, PM maturity level (Archibald, 2003)


Management (PM)

7

• SE is “a methodical, disciplined approach for the design, realization, technical management, operations, and retirement of a system. SE is the art and science of developing an operable system capable of meeting requirements within often opposed constraints. Systems engineering is a holistic, integrative discipline, wherein the contributions of structural engineers, electrical engineers, mechanism designers, power engineers, human factors engineers, and many more disciplines are evaluated and balanced, one against another, to produce a coherent whole that is not dominated by the perspective of a single discipline.” (NASA, 2007)


Management (PM)

8

• Project Management Institute (PMI) - Project Management Body of Knowledge (PMBOK) guidebook defines PM as “the application of knowledge, skills, tools and techniques to project activities in order to meet or exceed stakeholder needs and expectations from a project” (PMI, 2004)

• NASA defines PM as “the function of planning, overseeing, and directing the numerous activities required to achieve the requirements, goals, and objectives of the customer and other stakeholders within specified cost, quality, and schedule constraints” (NASA, 2007, 2010)


Management (PM)

9

The Overlapping Areas of SE & PM in a Project (Kossiakoff and Sweet, 2003)


Management (PM)

Technical Skills Managerial Skills

10

The Roles of Program/Project Manager and Systems Engineer in the Defense Systems Project Life Cycle Processes

(DOD, 2010)


Management (PM)

11

The Responsibility of Program/Project Manager and Systems Engineer in the Defense Systems Project Life Cycle Processes

(DOD, 2010)


Management (PM)

Research Model: PM Cost Estimating Model

November 3rd, 2010 The 25th Int’l Forum on COCOMO & Systems/Software Cost Modeling 12

SE PM

Case 1

SEScenario 1:

PM effort = f(SE effort)

Scenario 2: PM effort ≠ f(SE effort)

What is the dependency between SE & PM?

• how much SE effort do we need to know before we can determine PM effort independently?

• If possible, how early in LC can we assess PM effort? Or it must be done Interdependently?

Many suggested both SE/PM are related, but their costs and efforts were rarely mentioned…



SE

Case 2

SE

PM

Scenario 1:PM effort = f(SE effort)


• Is SE part of PM?• Eisner’s view (2008) on

SE/PM• Is it common in industries?



PM

Case 3

SE

SE

Scenario 1:PM effort = f(SE effort)


If PM is part of SE…• how much SE effort do

we need to know before we can determine PM effort?

This view is usually shared in SE oriented communities & gov’t entities…

15


Model #1 ~ Synthesized via COSYSMO (Valerdi, 2005)

Where,PMNS = effort in Person Months (Nominal Schedule)A = calibration constant derived from historical project datak = {REQ, IF, ALG, SCN}wk = weight for “easy”, “nominal”, or “difficult” size driverΦk = quantity of “k” size driverE = represents diseconomies of scaleEM = effort multiplier for the jth cost driver. The geometric product results in an overall

effort adjustment factor to the nominal effort.

14

1,,,,,, )(

jj

E

kkdkdknknkekeNS EMwwwAPM

SE PM

Case A: PM = f(SE)

16


Potential model parameters were predetermined through various knowledge sources (e.g. books, scholar publications, research whitepapers, dissertations, professional and government publications, etc.)

•Aerospace Engineering•Civil Engineering•Computer Science•Construction Engineering and Management•Defense/Military•Engineering Management•Government•Information Technology•Management Information Systems•Professional Societies•Project Management•Risk Management•Software Engineering•Systems Engineering

17


The initial 18 PM effort multipliers are listed as following:

These initial PM cost indicators were determined to be possibly correlated to factors that have effects on SE/PM cost adjustment factors (Akintoye, 2000; Anderson and Brown, 2004; Crawford et al., 2005; de Wit, 1988; Hamaker and Componation, 2005; Hartman and Ashrafi, 2002; Honour, 2010; NASA, 2010; Valerdi, 2005)

1. Scope Understanding2. Scope Volatility3. Scope Growth4. Requirements Volatility5. Requirements Growth6. Budget Constraints7. Schedule Span8. Project Complexities9. Systems Complexities10. Documentation Level11. Level of Service Requirements12. Stakeholder Cohesion13. Project Management Maturity14. Project Management Experience/Continuity15. Process Capability16. Technology Maturity and Risk17. Tool Support18. Multisite Coordination



PM SE SystemComplexity

ProjectComplexity

Is PM effort proportional to SE effort?

What if PM ≠ f(SE) ?

19


Model #2

Where,PMNS = effort in Person Months (Nominal Schedule)A = calibration constant derived from historical project datak = {REQ, PCR, CST, SCM, DCL}wk = weight for “easy”, “nominal”, “difficult”, or “low”, “medium”, “high” size driverΦk = quantity of “k” size driverE = represents diseconomies of scaleEM = project management efficiency multiplier for the jth cost driver. The geometric

product results in an overall effort adjustment factor to the nominal effort.

5

1,,,,,, )(

jj

E

kkdkdknknkekeNS EMwwwAPM

20


Requirements & Scope (REQ)

Project Complexity & Risk (PCR)

Constraints (CST)

Stakeholder Cohesion &

Multisite Coordination

(SCM)

Document & Comm Level

(DCL)

REQ Efficiency: PPT

PCR Efficiency: PPT

CST Efficiency: PPT

SCM Efficiency: PPT

DCL Efficiency: PPT

E= PMNS

X

X

X

X

XPPT = Project Management Capability and Maturity on People, Process & Tools

Consolidated 5 Cost Factor Categories

21


Cost Factor #1Requirements & Scope (REQ)

– How well understood is the project?•Scope of requirements•Number of requirements•How well they are defined

•Statement of Work (SOW), Work Breakdown Structure (WBS), etc

•Volatility/Rate at which they are changing•Expect any new/additional requirements ~ Requirement Creep?

Related & Depends on SE

22


Cost Factor #2Project Complexity & Risk (PCR)

– How much risk is there?•What is the level of risk for the project?•What is stakeholders’ risk attitude – risk adverse?•How difficult is it to assess the risk?•How complex is the project?

•Project Complexities•organizational, technological/product

•e.g. PM related integration, coordination, etc•Number of known project complexity and risks

Related & Depends on SE

23


Cost Factor #3Constraints (CST)

– How tight are the constraints?•Schedule Span

•Time constraints•Deliverable date•Amount of slack time allowed

•Budget Constraints •Money/Cost constraints

•Resource Constraints•Human Resources

•Function/Feature •Minimum acceptable features

•Quality •Minimum acceptance by customers

Partially related to SE

24


Cost Factor #4Stakeholder Cohesion & Multisite Coordination (SCM)

– Amount of external PM work to be done•Number of stakeholders•Diversity of stakeholders

•e.g., have opposing goals/objectives, have different world views

•Communication challenges •external clients, internal clients, contractors, languages, time zone difference, etc

25


Cost Factor #5Documentation & Communication Level (DCL)

– Amount of internal PM work to be done•Amount and complexity of required documentation

•e.g., project plan, resource management plan, status reports, etc

•Amount and complexity of required communications•Cubical/office noise

•Solving project issues •e. g., number, length and frequency of meetings, etc

26


Requirements & Scope (REQ)

Project Complexity & Risk (PCR)

Constraints (CST)

Stakeholder Cohesion &

Multisite Coordination

(SCM)

Document & Comm Level

(DCL)

REQ Efficiency: PPT

PCR Efficiency: PPT

CST Efficiency: PPT

SCM Efficiency: PPT

DCL Efficiency: PPT

E= PMNS

X

X

X

X

XPPT = Project Management Capability and Maturity on People, Process & Tools

5 PM Efficiency Multipliers (PPT)

27


Project Management Efficiency Multipliers

Capability & Maturity of People, Process & Tools (PPT)

People Capability – PM Attributes*•Communication skills•PM experience•Information sharing willingness•Delegates appropriately•Well-organized•Supports and motivates project team•Open-minded and flexible•Provide constructive criticism•Positive attitude•Technical competency•Team builder & player•Ability to evaluate and select project resources•Goal oriented•Courage and conflict solving skills•Problem solver•Take initiative•Creativity•Integrator (team, PM activities, etc)•Decision making skills

28


Project Management Efficiency Multipliers (cont’d)

Capability & Maturity of People, Process & Tools (PPT)

Process Maturity**•PM process maturity (CMMI, The Berkeley PM Process Maturity Model)•Organization PM maturity (PMI-OPM3)

•Initial•Repeatable•Defined•Managed•Optimized

Tool Support**•Level of tool automation

•Very few primitive tools•Basic/Micro tools•Extensive/Few Integrative tools•Moderately integrated environment•Fully integrated environment

*Adapted from Software Development Cost Estimating Guidebook (USAF Air Logistics Center, July, 2009) & Essentials of Project and systems Engineering Management (Eisner, 2008)**Adapted from Software Development Cost Estimating Guidebook (USAF Air Logistics Center, July, 2009)

Survey

• Please take some time to answer each question

• Feel free to ask questions

29

Questions for Discussion

• In industry, if the same person who performs both SE/PM functions, how do you estimate such effort?

• Which case and effort function scenario do you think is more realistic? Why?

• How do industry corporations and government estimate PM cost & effort?• What type of tool & method do they use?

• How can the propose model become more practical and applicable for industry use?• What is missing in this research?

30

Additional Questions?

Suggestions?

Comments?

Thank You Very Much for Your Time!!!

31

Systems Engineering and Project Management: Similarities and Differences for Cost Estimation Dr. Ricardo Valerdi – Massachusetts Institute of Technology.

Documents

pm cost estimating

project management services

total project cost tpc

total systems development

technical management

cost estimation

measure pm costs

project type