Martt.anne

Establishing an Enterprise Earned Value Management Capability

Anne V. MarttUnited Space Alliance, LLC.

Page 1

Establishing Enterprise EVM PM Challenge February 2009

Establishing an Enterprise EVM Capability

• What is EVM• Who is USA• Background

– Past Experience

• Implementation Approach – Processes– People and Infrastructure– Tools

• Other ‘Words of Wisdom’• Going Forward

Page 2


Earned Value Management (EVM) - What It Is

• EVM is a management system that – integrates the committed cost and schedule for a program’s work

content into a tightly controlled baseline; – and quantitatively measures progress towards those commitments

• EVM is a technique for program/project managers to– objectively measure contract performance,– have early identification of variances from the baseline, – mitigate risks associated with cost and schedule overruns,– and methodically forecast final cost and schedule outcomes

• EVM is the best known, proven way to integrate: – Content / Schedules / Resources / Risk Management

Successful EVM is critically dependent on a sound planSuccessful EVM is critically dependent on a sound plan

Page 3


EVM – How it Works

• EVM quantitatively measures work accomplished as the basis for variance analysis

– Variance is determined by actual work performed vs. actual cost incurred; not planned vs. actual costs for a given elapsed period

• EVM regularly probes key measurement questions– How much work should have been completed?– How much work was completed?– How much did the completed & in-process work cost?– What is the project risk/opportunity from late or early completion?– What is the project risk/opportunity from higher or lower cost for

completed work?– How much is the total project supposed to cost?– What do we estimate the project actually will cost?

Page 4


United Space AllianceThe Space Operations Company

4

Premier provider of space operations servicesNASA’s Prime Contractor for end-to-end Space Shuttle Operations24/7 Operations Support to International Space StationBringing space operations skills to the Constellation Program

Page 5


EVM at USA – Past Experience

• Pockets of “mini-EVM” throughout the company– Adherence to basic concept; but performed manually on small

scale projects

• An EVM System was implemented in 2001-2002 for use on major Space Shuttle Upgrades development projects

– Selected program product tool• Did so very rapidly; and had to create substantial custom code

– Baselined ‘System Description Document’, established internal processes, and trained personnel

• In parallel with initial EVM execution and Integrated Baseline Review

– Plus had major subcontractors with EV reporting requirements• Established roll-up interface while learning EVM ourselves

Page 6


EVMS at USA – Compelling Need

• With the 2004 announcement of the ‘Vision for Space Exploration”, NASA entered a major acquisition phase for the first time in decades

– Drawing from DoD best practices, EVM is now imposed on virtually all NASA major acquisitions

• USA began an aggressive internal change analysis designed to ensure the long-term viability of the company

– A new environment with new customer relationships, new contractual arrangements, and new requirements from both

– Among the highest priority of many identified major enterprise wide projects was the need for an Enterprise EVM System

Page 7


Implementation Approach – 3 Key Components

• Each component requires unique attention for success

• Process and People take the longest to solidify and cultivate

• Understanding and documenting the process helps ensure appropriate Tool selection

• Building all three at the same time should be avoided– Can be draining and counterproductive for the team

People + Process + Tool = System

Page 8


Facing the Biggest Barrier to Success

• Internal politics and “culture” are by far the biggest barriers to adopting change, including new systems

– There is a strong resistance to change in most organizations– Resistance, many times, stems from the fear of the unknown– It is human nature: people tend to blame tools and processes

when it is really their own lack of knowledge and understanding

• What worked best in addressing this:– Senior Management commitment – visible and strong– Communications – lots of it– Training – early and often– Core Team with “Help Desk” mentality

Page 9


EVMS Implementation – A Full Blown Project

• Project authorized in Feb 2006, to complete in Feb 2007– Project completed (final vendor software installed) in June 2007– Project came in 9% under cost

• Applied sound integrated project management – System requirements definition first– High fidelity estimate of cost and schedule– Integrated baseline for implementation– Assignment of appropriately skilled resources– Clear requirements and selection criteria for tool solution– Allowance for design cycles– EVM – Risk Management

Page 10


Enablers to Implementation Success

• Top management was committed to implementation of an Enterprise EVMS

– Commitment was regularly communicated

• Adequate funding was provided

• Skilled leadership and expertise resources were assigned

• Imminent contractual requirements provided a strong “forcing function”

• Development of a structured training program was given high priority– Train early and often!

Page 11


Process Definition – Requirements First• Develop the Process prior to tool selection

– Provides context to the tool evaluation and selection – Trying to implement tool while defining the process is not cost

effective and will delay implementation

• Define Process based on ANSI-748 guidelines• Create Compliance document to map processes to each of

the ANSI-748 Guidelines– Assures the system will be compliant for DCMA validation – Map to the ANSI-748 very early in process development

• Reduces the risk of significant rework later onIPACS PROCESS

ORGANIZATION

AUTHORIZATION

SCHEDULE BASELIN

E

BUDGET BASELIN

E

SCHEDULE STATU

S

DATA ACCUMULA

TION A

ND

REPORTING

ESTIMATE

AT C

OMPLETE

VARIANCE A

NALYSIS

INDIR

ECT / SERVIC

E CENTE

R

MATERIAL A

ND

SUBCONTRACTS

BASELINE M

AINTENANCE -

INTERNAL

BASELINE M

AINTENANCE -

EXTERNAL

SURVEILLANCE

Applicable ProductsANSI/EIA-748 GuidelinesORGANIZATION

1 2-1a Define authorized work X CWBS, CWBS Dictionary

2 2-1b Identify Program Organization Structure X OBS, RAM

3 2-1cCompany integration of EVMS subsystems with WBS and OBS X X X X

PAD, RAM, WAD, PBL, IMS, Performance Reports by CWBS

4 2-1d Identify organization/function for overhead X FPRA, RMI Organizational Chart

5 2-1e Integrate WBS & OBS, create control accounts X X X X RAM, CAP

PLANNING, SCHEDULING & BUDGETING

6 2-2a Sequential scheduling of work X IMS, CAP, SOW, IMP, WAD

7 2 2bIdentify interim measures of progress, i.e. milestones products etc X IMP IMS CAP

Page 12


Up Front Definition of the Process

• Create a System Description Document which defines the process in detail

• Use flows to define the process based on roles and responsibilities (Pgm Mgr, IPT Lead, Cntl Acct Mgr, etc)

• Evaluate processes regularly and modify based on lessons learned along the way

Review forConsistency,Accuracy and

Logic

EV EngineDevelop Time

PhasedBudgetsfor Work

Packages PlanningPackages &

ProjectManager

IPT Lead/Functional

Manager

ControlAccountManager

208

See ConnectorConAuthorization

209

Update PBL207

EstablishMR

206

ConnectorFfromScheduleBaseline

Set BaselineUpdate PBL

210

No Yes

PP&C

Page 13


Establish the Infrastructure

• Executive Management ownership – EVMS co-owned by the Program/Project Management Process

Owner and the Chief Financial Officer– Joint approval authority on the System Description Document

• Establish highly skilled Core Team– Small number of people, that are experts in EVM and company

rates/accounting structures– To provide leadership, guidance, assistance – and lots of it!– Need the right personality/temperament

• Create a common Project Planning and Control (PP&C) organization with skilled schedulers and cost analysts

– The real “nuts and bolts” people – key to helping the technical folks understand the process and the data

– Responsible for the Process and System Description Document – Provide guidance and knowledge to Project Start-ups

Page 14


Tool Selection – Do Your Homework

• Take the time to develop requirements from the top down and engage stakeholders along the way

– Allows the implementation team to focus on implementation without having to revisit the requirements along the way

• Include all the requirements for a validated EVM system– System Requirements, System Description, Processes & Tools

• Use a requirements hierarchy– High Level Objectives

• First level of communication to stakeholders for buy-in– System Level Requirements– Detailed Requirements

• Include justification and rationale in document• Directly drives evaluation criteria for tool selection

Page 15


Tool Evaluation – Be As Sure As Possible

• Utilize prototypes/demonstrations in tool selection, particularly in high risk areas

– Don’t just believe Suppliers when they say they can do something….make them show you

– Ensure the tool can interface with and accommodate company capabilities and requirements

• Example: demonstration of tool compatibility with internal rate/accounting structure

• Define crisp, specific success criteria for evaluation of tool solution

Page 16


Implementation – Plan Carefully

• Develop a Project Implementation Plan– Resource requirements for all participating organizations– Integrated Master Schedule– EV performance types and metrics

• Plan for and generate all Supporting Documentation –complete before “go live” date

– Operating and desk procedures– Design documents (System Configuration, Interface Control

Documents, etc)– Training Materials

Page 17


Implementation – Manage Carefully

• Perform EVM on the Enterprise EVM Implementation– Start with the previous system (even if manual)– Move to new system at earliest opportunity – excellent component

of the systems’ verification

$-

03/03

/2006

03/17

/2006

03/31

/2006

04/14

/2006

04/28

/2006

05/12

/2006

05/26

/2006

06/09

/2006

06/23

/2006

07/07

/2006

07/21

/2006

08/04

/2006

08/18

/2006

09/01

/2006

09/15

/2006

09/29

/2006

10/13

/2006

10/27

/2006

11/10

/2006

11/24

/2006

12/08

/2006

12/22

/2006

01/05

/2007

01/19

/2007

02/02

/2007

02/16

/2007

03/02

/2007

03/16

/2007

03/30

/2007

Budget Earned Value Actual Cost ETC

WBS: 1 - Enterprise EVMS Project

Summary Level: 1

ME

Dec

Fore

cast

Com

plet

e 3

/16

PMB = $###

MR = $##Target Cost = $###

Current Indices:SPI 0.78CPI 1.35TCPI 0.78% Complete 73.5%

Current:BAC ###EAC ###VAC ##Current:

Budget ###EV ###Actuals ##SV (##)CV ###

Page 18


Implementation – Manage Carefully

• Perform Continuous Risk Management– Embed Risk Management from the start of the project– Create EVMS implementation-unique risk scorecard – Hold regular project risk reviews

5

02

4321

1

01 03, 062

05, 073

4

5

Like

lihoo

d

Consequence

System Certifiability02

Labor Resource Availability01

Training Programs & Curriculum07

PeopleSoft Upgrade Schedule05

Risk TitleRisk ID

Page 19


Implementation – Anticipate and Mitigate

• Even with all this preparation, expect “bumps in the road”– Don’t expect a “one pass” development effort– As hard as you try to cover everything, a project in the real world

will encounter something (likely many somethings) that the preparations and procedures didn’t address

-- Production Server EnvironmentProduction Server Environment--Validated Rates & FactorsValidated Rates & Factors-- Accounting InterfaceAccounting Interface-- Schedule Standards & EVMSchedule Standards & EVM-- Work Around for Budget ExportWork Around for Budget Export-- Work Around for NonWork Around for Non--ProductiveProductive-- Minimal Forms & ReportsMinimal Forms & Reports-- Top Level Procedures onlyTop Level Procedures only-- Direct Support from EVMS TeamDirect Support from EVMS Team

CPASCPAS

CLV S1CLV S1

CEVCEV

IBR CPR IBR

CPR

CPRIBR

Deploy CLV S1ComplianceReview

OperationalOperational

Initial IntegrationInitial Integration

Final IntegrationFinal Integration

Limited DeploymentLimited Deployment

Non-ProductiveLoader & BudgetExport I/F Delivery

OperationalOperationalDeployIntegration &

Test Complete

Original PlanOriginal Plan

Page 20


Preparing the People – Skill Set

Training, training, and more training• It is difficult to fully understand and grasp EV concepts unless actively

using on a project– Some of the training is “Use or Lose” and may require multiple training

sessions over time

• Develop a structured, hierarchical training program– Create different levels of materials for progressive building of knowledge

• EV 101 (general overview)• EV 102 (continues from EV 101 with focus specific to CAM duties

such as trend analysis and performance report generation)– Train based on specific roles and responsibilities (PMs, CAMS, IPT Leads)

• Tailor materials to company specific processes and products– Off-the-shelf courses do not map to internal terminology and procedures

• Also include training on Project Management principles and techniques (EV, scheduling, planning, metrics, risks, etc)

Page 21


Preparing to Prepare the People - Iteration

• Be prepared to iterate on the training materials, levels, and tailoring

– Conduct initial classes; plan for potentially significant modifications to materials

– Conduct classes again; anticipate potential for additional iterations

• Be careful not to underestimate the cost of training program development

– The training is critical to successful implementation on projects

• Be prepared to iterate on the timing of training delivery – Timing vs practical application is difficult – “Re-training” and “just in time” training is necessary

Page 22


Preparing the People - MindsetCommunicate, communicate, communicate• The culture change takes a long time!!!

– Even with loads of training and attention, people gravitate back to “the way we’ve always done it”

• Constant and consistent supportive communications from the top down is required

• Ensure that the Manager of each Project has “the EVM mantra”• Provide an “In-House Consulting” service – for the long haul

– Experienced personnel to help project teams start up projects (planning, scheduling, tool usage, etc.)

– The need for in-house expertise and “hands-on” mentoring of project teams cannot be over emphasized

– It simply is not possible to bring a project team totally up to speed on all aspects of an EVMS through training alone

– The need for this in-house expertise will continue longer than you think (we still don’t know how long)

Page 23


‘Words of Wisdom’ – Other Hard Knocks

Some of our specific most difficult challenges have been• Playing “catch up” after contract award

– Lots of work and the project’s EV plan structure may have permanent major flaws

– Really need to have much of this in place during proposal phase

• Misunderstanding of Management Reserve (MR)– What it is for; how you get it; how you use it– Particular threat in contracts with change thresholds– Additional oversight of MR is highly advised

• Difficulty in grasping forward estimate concepts– Estimate At Complete (EAC), best case, worst case, most likely

Page 24


Going Forward - EVMS Validation

• Validation of the company’s Enterprise EVMS– Driven by contractual requirements– DCMA-led with NASA and prime contractor involvement

• DCMA “24 Step” process for EVMS Validation – Lengthy (up to 18 months) but designed to maximize probability of

success– Process consists of:

• Progress Assistance Visit (PAV)–“Pre-review”–Corrective action period

• Validation Review (VR)• Joint Surveillance

Page 25


EVMS Validation

ID Name

1 Mock Review Outbrief2 Orion IBR Complete3 Initial Visit/Kickoff4 PAV Complete5 VR Complete6 DCMA VR Letter Received7 Progress Assistance Visit8 PAV Review Data Submit18 DCMA/NASA PAV Prep C25 PAV Review Completed29 PAV Corrective Actions C34 Validation Review35 DCMA/NASA VR Prep Co41 VR Review Data Submitte46 VR Review Completed50 VR Corrective Actions Co55 VR Process Completed

08/3009/27

10/2508/05

10/2402/20

Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May2008

Appling the DCMA Validation Review Template………Example

Allows for up to 10 Months Corrective Action Closure

System Description Document

Modifications

Reduced Corrective Action Time Can Shorten this Template

Page 26


EVMS Validation – Current Status

• DCMA Initial Visit – October 2007• DCMA Progress Assistance Visit – March 2008

– Evaluation of USA Enterprise and Orion Contract– USA characterized as “way ahead of the game”– Very low number of discrepancies received

• DCMA Validation Review – October 2008– Streamlined approach enabled by prior successful reviews– Excellent results

• Only 2 non-compliances • USA characterized as “at the top” for this stage

• Targeting final Validation review for April 2009

Page 27


Summary – Best of ‘What Went Right’

• Process in place first; clearly mapped to ANSI-748 • Solid requirements and evaluation approach for tool

selection• Structured EV training tailored to company process,

structure, culture, terminology• Strong management commitment and communication• Core Team for “In-House Consulting”• Central PP&C group for detailed assistance • Anticipation and planning to address issues• Continual evaluation and mitigation• Patience and sense of humor

Page 28


Thanks for your attention

Questions or comments?

Martt.anne

Technology

enterprise evm capabilitywhat

value management evm

process work cost

space exploration

major enterprise wide

final cost

initial evm execution

new requirements