Gonzalez.matthew

Pg 1

Planning and Conducting an Integrated Baseline Review (IBR)

Matthew T. Gonzales, Project Business ManagerJet Propulsion Laboratory

818-354-3629

Used with Permission

Pg 2

Communication

– Early– Frequent– Inclusive

Pg 3

‘If you can't describe what you are doing as a process, you don't know what you're doing.’

W. Edward Deming

Organization

Pg 4

IBR as a Process

Pre-Phase A Phase A Phase B Phase C Phase D

Formulation Implementation

Project Emphasis:

- System Definition- Identify potential contractors and teammates- Identify contractors above $20M

EVM Emphasis:

- EVM implementation team defined- Contractors develop EVM plan- Progress Assessment Visit (PAV); Contractor presents EVM plan- EVM team evaluates - If Validation Review is required, EVM teams works with local DCMA- Conduct follow-up PAV- Support KDP A

Pre-Phase A emphasizes the contractors and projects readiness

for EVM

Phase A emphasizes the EV Process and Initial Baseline

Plan

Phase B emphasizes the validity of baseline to complete

work

Phase C emphasizes use of EVM and the validity of EAC to

complete work

Phase D emphasizes use of EVM and the validity of EAC to

complete work

- Complete System Definition- Establish Requirements- Select Contractors- Award Contracts with EVM DRDs

- Contractor teams develop initial project cost and schedule baseline- Contractors submit EV reports per DRDs- Project Office conducts IBR with emphasis on Process

- Compliance to EV Process- Data Quality- CAM Knowledge- Change Control- Validity of Baseline relative to Scope of Work

- Support KDP B

Project Emphasis:

EVM Emphasis:

Project Emphasis:

EVM Emphasis:

- Project Requirements Baselined- Preliminary Design Established- LL Procurements Let

- EV System is mature- Reports Submitted- Logs Established- Change Control Exercised

- Project Office conducts IBR with emphasis on Risk to Reserves

- Project Plan content- Completeness of Planning- Change control process- Basis of Estimate for Cost and Schedule- Can design be built with planned resources- Overall robustness of EVM system- Reconciliation to Phase A Baseline

- Support KDP C

Project Emphasis:

EVM Emphasis:

- Integration of Flight HW and SW- Integrate Payload and Launch Vehicle

- Implement EV- Project Office conducts IBR with emphasis on Risk to Reserves

- Does Project Plan support I&T Program- Is EV data reliable as a predictor- Quality of VARs- Is the project using EVM data to manage- Test EAC process- Quality BOEs- Can I&T be completed for estimated cost- Is there sufficient schedule- Are Risks known and mitigated- Reliable reporting

- Support KDP E

Project Emphasis:

EVM Emphasis:

- Complete Flight and Ground Designs- Conduct component and SS CDRs- Build BB and EMs- Initiate Acceptance Test Program for Flight HW

- Implement EV- Project Office conducts IBR with emphasis on Risk to Reserves

- Does Project Plan support detail design- Is EV data reliable as a predictor- Quality of VARs- Is the project using EVM data to manage- Test EAC process- Quality BOEs- Can HW be built for estimated cost- Is there sufficient schedule- Are Risks known and mitigated- Reliable reporting

- Support KDP D

Pg 5

Key to Success

– Communication– Organization

Pg 6

Impetus

IBR Steps

Planning

Conducting

Final Report

Caucus and review

Out-brief

Pg 7

Impetus

Impetus for an Integrated Baseline Review Comply with customer requirements (7120.5d) Significant change to system design New Launch Date ‘Grass roots’ EAC Developed Over Target Baseline incorporated

Pg 8

Planning

In preparing for battle I have always found that plans are useless, but planning is indispensable.Dwight D. Eisenhower

Pg 9

ProjectRe-plan

EV DatabaseComplete

1st

MonthEV update

cycle

Review&

Analysis

Turbo estimates imported into MPM. PMB established.

MS Project milestones imported in MPM.

Weight applied. EV Plan Set

Project integrated baseline complete.

Set ‘Over Target Baseline’

Schedule updated on weekly cadence. Weekly metrics implemented.

Update to MPM database.

ACWP and BCWP for July month end

Generate and submit CPR Format 1 in

support of customer MMR

EV metrics analysis reveals

planning anomalies.

Corrective actions assigned/reviewed.

Implementation approved by PO and implemented

Changes to S/C and Mtl plans and EV

methods completeprior to Aug close.

Timeline

Jul ‘06

Aug ‘06

Planning Process Flow

IMS restructured for new launch date and

design change

Project IMS resources estimated and loaded into Turbo for pricing

Project Plan approved by SMD PMC.

Authorized to Proceed.

Events Process

Pg 10

Day 4 – Data Review/Informal Out-brief

Review Survey Results3-4 slide contractor feedback

ProjectCDR

ProjectIBR

Project office generates and briefs

IBR information package.

Corrective actions closed. EV and VARs

reviewed at project MMR

Contractor, HQ/DPO and project team

briefed on IBR plan

Update to MPM database.

ACWP and BCWP for July month end

Submit CPR Format 1. Planning changes incorporated. EV metrics improve.

EV metrics analysis reveals

planning anomalies.

Timeline

Sep ‘06

Nov ‘06

Planning Process Flow

IBR date negotiated with all affected

parties. Set for Nov.

Events Process

2ndMonth

EV update cycle

Oct ‘06

IBR scope, schedule and team

membership finalized.

IBR Coordination meeting with Review Team week before

review

Day 1 -IBR In-BriefJPL Project Office

BATC EV SurveillanceBATC Project Office

Day 2 – Data ReviewReview team audits

CAM notebooks.Team caucus.

Day 3 – CAM InterviewsProject Technical Lead

conducts interview. All team members complete survey

Formal Out-brief

Pg 11

Planning

Plan for your review Anticipated KDP Project Phase Working with contractors

Define IBR as early as possible Scope, Goals, Purpose Team membership with roles and responsibilities Required documents for review Work with contractor on logistics

Communicate 3 briefings in preparation for review Need to train interviewers as well as interviewees Provide contractor with questions ahead of time

Pg 12

Preamble

The In-Brief

Pg 13

John Wooden: A Coaching Legend

UCLA's basketball program has the international reputation of being No. 1. There is a major reason for that his name is John Robert Wooden, who announced his retirement after the 1974-75 season (his 27th campaign) as the Bruins' head coach with the winningest record in all of the sport's history. Wooden celebrated his 92nd birthday on Oct. 14, 2002.

Wooden concluded his 40 years as a head coach that season and his 885-203 overall career win-loss record (a percentage of .813) is unequaled. A large part of that success was at UCLA. In 27 years as Bruin coach, his teams registered 620 wins, and only 147 losses while earning far more national honors than any other university.

Under Wooden, UCLA won an unprecedented 10 NCAA championships, including seven consecutive (1966-73). Included in the string is one of the most amazing win streaks in all of sports, 38 straight NCAA tournament victories.

In addition, there is the all-time NCAA consecutive winning-streak record of 88 games over four seasons, which included consecutive 30-0 seasons in 1971-72 and 1972-73. UCLA also won 149 of 151 games in Pauley Pavilion during his Bruin tenure.

John Wooden is the only coach to compile four undefeated seasons of 30-0 and his Bruin teams captured 19 conference championships (the record of which Wooden is most proud).

Coach Wooden is the first person to be inducted into the National Basketball Hall of Fame as both a player and coach

Pg 14

John Wooden Quotes“It’s the little details that are vital. Little things make big things happen”

“If you don’t have time to do it right, when will you have the time to do it over”

“Winning takes talent, to repeat it takes character”

“It is what we learn after we know it all that really counts”

“Be prepared and be honest”

“Things turn out best for the people who make the best of the way things turn out”

Pg 15

Use your review In-brief as a time to

Inspire

Pg 16

The In-Brief

Project OverviewGoalsSuccess CriteriaProcess OverviewDocumentsLogistics/ScheduleGeneral Comments

Pg 17

• Control Account Managers Notebooks

– Turbo reports

– Schedules , by WBS element and by IPT

– Control Account Plans to Work Package level of detail and Earned Value Milestone Report

– Current WBS and WBS Dictionary

– Estimating methodology and documentation to enable assessment of the basis of the estimates

– Work Authorization Documents for authorized control accounts.

– Performance Measurement Reports (CPR Format 1)

– Budget Change Requests (if applicable)

– Approved liens and KVRs and recovery plans

• Other Ball documentation required for the IBR:

– Current Contract SOW

– BATC System Description, Kepler EVM Plan Supplement

• Reviewing for completeness and consistency in the plans.• Is the work fully accounted for and plan appropriately.• Are all resources planned and available when required.• Is the time phasing of cost and schedule consistent.• Is EV planned in a way to yield accurate results.• Are analysis tools available and utilized.• Are changes tracked and managed• Are they adhering to the BATC way of planning and managing projects.

Data Review - Wed, Nov. 15th 8am – 5pm

Pg 18

Conducting Interviews

Pg 19

Conducting InterviewsSelect WBS elements for review

Risk driven (technical and/or programmatic) Phase specific

Select appropriate questions Phase specific Basis of Estimate Discern risk Communicate early

Develop interview teams Led by technical representative Small teams Provide training on questions

Discussions/Interviews Conversational ~ 2 hours Document results

Pg 20

CAM Interviews – Nov. 16th 8am – 5pm

DRAFTWBS WBS Description Day Interview Time6.1.1 FS Management Thurs 08:00 – 10:005.2.6 Attitude Control Thurs 08:00 – 10:00 5.2.8 Communications Thurs 08:00 – 10:00 6.1.3 FS Mission Assurance Thurs 10:00 – 12:00 6.3.2 FPAA Thurs 10:00 – 12:006.3.3 Electrical Thurs 10:00 – 12:006.3.4 Telescope Thurs 1:00 – 3:00 5.2.3 Mechanical Subsystem Thurs 1:00 – 3:00 6.3.5 Photo Integration Test Thurs 3:00 – 5:005.2.11 Spacecraft I&T Thurs 3:00 – 5:005.3 System ATLO

IBR CAM Interview Schedule

WBS WBS DescriptionPeg Frerking

Leslie Livesay

Jim Fanson

Sarah Gavit

Michael Moore

Renee Leck

Pam Marcum

Jack Graham

Steve McClard

Nate Porter

John Keidel

Robert Bitten

Dan Graham

Maura Fujieh

Bill Epping

Tim Clark

Cal Cambers

Joe Fischetti

6.1.1 FS Management 1 1 1 15.2.6 Attitude Control 1 1 1 15.2.8 Communications 1 1 1 16.1.3 FS Mission Assurance 1 1 1 16.3.2 FPAA 1 1 1 16.3.3 Electrical 1 1 1 16.3.4 Telescope 1 1 1 15.2.3 Mechanical Subsystem 1 1 1 16.3.5 Photo Integration Test 1 1 1 15.2.11 Spacecraft I&T 1 1 1 15.3 System ATLO

SupportIBR CAM Interview Schedule NASA Reps Discovery Program OfficeTechnical Leads

Pg 21

CAM QuestionnaireQuestions Rating Low Med High

1.What is the Manager’s scope of work? Does the Manager know what scope of work is his/her responsibility? Cost 1 2 3 4 5

–The Manager should be able to show the team a Statement of Work (SOW) paragraph, a WBS, PWBS, or CWBS narrative, and a Work Authorization Document that correlates to the task. Schedule 1 2 3 4 5

Comments:Technical 1 2 3 4 5

2.How did the Manager plan the scope of work into control accounts? Cost 1 2 3 4 5

–The SOW defines the effort. The WBS, PWBS, or CWBS provides specifics, such as work definition. The work authorization and change documentation should show information such as the dollars/hours, period of performance, and description of the scope of work and any changes.

Schedule 1 2 3 4 5

Comments:Technical 1 2 3 4 5

3.How did the Manager ensure that all elements of the scope of work were planned? Cost 1 2 3 4 5

–The Manager should be able to show the team the work packages breaking down the scope of work, and the budget associated with each work package. The sum of the work packages should be the control account budget. Each control account should contain a specific scope of work.

Schedule 1 2 3 4 5

Comments:Technical 1 2 3 4 5

Pg 22

Caucus and Review

Pg 23

Caucus and Review

Review survey responses By interview, by question Look for outliers and discuss. Opportunity to share thoughts and observations Normalize metrics

Do metrics support subjective risk observations

Pg 24

Caucus and ReviewLow Med High

12 36 60WBS CAM Intvr No Questions Cost Sched Tech Total5.2.6 CS SG 1 1.What is the Manager’s scope of work? Does the Manager

know what scope of work is his/her responsibility? 1 2 3 6

5.2.6 CS DG 1 1.What is the Manager’s scope of work? Does the Manager know what scope of work is his/her responsibility? 1 1 2 4

5.2.6 CS RB 1 1.What is the Manager’s scope of work? Does the Manager know what scope of work is his/her responsibility? 1 1 1 3

5.2.6 CS BE 1 1.What is the Manager’s scope of work? Does the Manager know what scope of work is his/her responsibility? 1 1 1 3

4 5 7 165.2.6 CS DG 2 2.How did the Manager plan the scope of work into control

accounts? 1 1 1 3

5.2.6 CS SG 2 2.How did the Manager plan the scope of work into control accounts? 1 1 1 3

5.2.6 CS RB 2 2.How did the Manager plan the scope of work into control accounts? 1 1 1 3

5.2.6 CS BE 2 2.How did the Manager plan the scope of work into control accounts? 1 1 1 3

4 4 4 125.2.6 CS DG 3 3.How did the Manager ensure that all elements of the scope of

work were planned? 2 2 2 6

5.2.6 CS SG 3 3.How did the Manager ensure that all elements of the scope of work were planned? 1 1 1 3

5.2.6 CS RB 3 3.How did the Manager ensure that all elements of the scope of work were planned? 1 1 1 3

5.2.6 CS BE 3 3.How did the Manager ensure that all elements of the scope of work were planned? 1 1 1 3

5 5 5 155.2.6 CS BE 4 4.How did the Manager obtain the resources for assigned work? 3 3 3 9

5.2.6 CS RB 4 4.How did the Manager obtain the resources for assigned work? 2 2 1 5

5.2.6 CS DG 4 4.How did the Manager obtain the resources for assigned work? 1 1 1 3

5.2.6 CS SG 4 4.How did the Manager obtain the resources for assigned work? 1 1 1 3

Risk Rating

Totals

Totals

Totals

Pg 25

Survey Results

…and the survey says…

Pg 26

Cost Risk Rating

Cost RiskWBS No Questions Cost6.1.3 FS MISSION ASSURANCE 2.776.3.2 FPAA 2.386.3.4 TELESCOPE 2.295.2.6 ATTITUDE CONTROL 1.846.3.5 PHOTOMETER I&T 1.756.3.3 ELECTRICAL 1.565.2.3 MECHANICAL 1.526.1.1 FLIGHT SEGMENT MANAGEMENT 1.445.2/5.3 SPACECRAFT/ATLO 1.395.2.8 COMMUNICATIONS 1.25

0.001.002.003.004.005.00

FS M

ISSI

ON

ASS

UR

AN

CE

FPA

A

TELE

SCO

PE

ATT

ITU

DE

CO

NTR

OL

PHO

TOM

ETER

I&T

ELEC

TRIC

AL

MEC

HA

NIC

AL

FLIG

HT

SEG

MEN

TM

AN

AG

EMEN

T

SPA

CEC

RA

FT/A

TLO

CO

MM

UN

ICA

TIO

NS

Ris

k R

atin

g

Pg 27

Schedule Risk Rating

Schedule RisksWBS No Questions Sched6.3.2 FPAA 2.396.3.4 TELESCOPE 2.286.1.3 FS MISSION ASSURANCE 2.065.2.6 ATTITUDE CONTROL 1.836.3.5 PHOTOMETER I&T 1.766.3.3 ELECTRICAL 1.655.2/5.3 SPACECRAFT/ATLO 1.525.2.3 MECHANICAL 1.525.2.8 COMMUNICATIONS 1.376.1.1 FLIGHT SEGMENT MANAGEMENT 1.36

0.001.002.003.004.005.00

FPA

A

TELE

SCO

PE

FS M

ISSI

ON

ASS

UR

AN

CE

ATT

ITU

DE

CO

NTR

OL

PHO

TOM

ETER

I&T

ELEC

TRIC

AL

SPA

CEC

RA

FT/A

TLO

MEC

HA

NIC

AL

CO

MM

UN

ICA

TIO

NS

FLIG

HT

SEG

MEN

TM

AN

AG

EMEN

T

Ris

k R

atin

g

Pg 28

Technical Risk Rating

Technical RisksWBS No Questions Tech6.3.2 FPAA 2.196.3.4 TELESCOPE 2.165.2.6 ATTITUDE CONTROL 1.646.3.5 PHOTOMETER I&T 1.596.3.3 ELECTRICAL 1.385.2/5.3 SPACECRAFT/ATLO 1.356.1.3 FS MISSION ASSURANCE 1.295.2.8 COMMUNICATIONS 1.245.2.3 MECHANICAL 1.246.1.1 FLIGHT SEGMENT MANAGEMENT 1.20

0.001.002.003.004.005.00

FPA

A

TELE

SCO

PE

ATT

ITU

DE

CO

NTR

OL

PHO

TOM

ETER

I&T

ELEC

TRIC

AL

SPA

CEC

RA

FT/A

TLO

FS M

ISSI

ON

ASS

UR

AN

CE

CO

MM

UN

ICA

TIO

NS

MEC

HA

NIC

AL

FLIG

HT

SEG

MEN

TM

AN

AG

EMEN

T

Ris

k R

atin

g

Pg 29

Risk Rating by WBS

WBS No Questions Cost Sched Tech Total6.1.1 1 1.What is the Manager’s scope of work? Does the Manager know what

scope of work is his/her responsibility? 1.33 1.33 1.40 3.83

6.1.1 2 2.How did the Manager plan the scope of work into control accounts? 1.17 1.33 1.20 3.50

6.1.1 3 3.How did the Manager ensure that all elements of the scope of work were planned? 1.20 1.40 1.20 3.17

6.1.1 4 4.How did the Manager obtain the resources for assigned work? 1.33 1.33 1.00 3.336.1.1 5 5.Ask the Manager to describe the resource development process. 1.40 1.17 1.00 3.006.1.1 6 6.How did the Manager come to an agreement of the scope, hours,

dollars and schedule? 1.50 1.33 1.00 3.50

6.1.1 7 7.Ask the Manager to show the team the schedule milestones used in the planning of the control accounts. The Manager should discuss: 1.00 1.00 1.00 2.83

6.1.1 8 8.Ask the Manager how the work was time phased to achieve the schedule. 1.40 1.33 1.00 3.17

6.1.1 9 9.Ask the Manager to identify any known risks that may affect the plan. 1.67 1.17 1.20 3.83

6.1.1 10 10.Are the Manager's resources adequately planned and correctly time phased to meet the plan? Progress of each plan should relate to earned value

1.33 1.17 1.00 3.33

6.1.1 11 11.Ask the Manager why he/she chose a particular type of earned value method? 1.33 1.33 1.40 3.83

6.1.1 12 12.Does the earned value method used make sense for the type of work being performed? Can you measure performance? Is the Manager using milestones to measure accomplishments? Does the Manager use the LOE method; and is the LOE in support of measurable

1.20 1.00 1.00 2.50

6.1.1 13 13.How does the Manager manage his/her plan? 1.50 1.50 1.20 4.006.1.1 14 14.How is the baseline maintained? 2.00 1.80 1.80 4.676.1.1 15 15.What is the project’s risk management process? How is risk

incorporated into the EAC? 1.50 1.50 1.50 3.00

6.1.1 16 16.What is the process for reviewing and updating EACs? 2.00 1.75 1.50 3.506.1.1 17 17.How does the CAM update the schedule? How often are schedules

updated? 1.67 1.67 1.00 4.33

FLIGHT SEGMENT MANAGEMENT Total Scores by WBS 1.44 1.36 1.20 3.49

Risk Rating

Pg 30

Risk Rating by Question

0.00

1.00

2.00

3.00

4.00

5.00

6.00

7.00

8.00

9.A

sk th

e M

anag

erto

iden

tify

any

know

n ris

ks th

atm

ay a

ffect

the

plan

.15

.Wha

t is

the

proj

ect’s

risk

man

agem

ent

proc

ess?

How

is16

.Wha

t is

the

proc

ess

for

revi

ewin

g an

dup

datin

g E

AC

s?7.

Ask

the

Man

ager

to s

how

the

team

the

sche

dule

mile

ston

es u

sed

in4.

How

did

the

Man

ager

obt

ain

the

reso

urce

s fo

ras

sign

ed w

ork?

10

.Are

the

Man

ager

'sre

sour

ces

adeq

uate

ly6.

How

did

the

Man

ager

com

e to

an a

gree

men

t of

the

scop

e, h

ours

,13

.How

doe

s th

eM

anag

er m

anag

ehi

s/he

r pla

n?

5.A

sk th

e M

anag

erto

des

crib

e th

ere

sour

cede

velo

pmen

t12

.Doe

s th

eea

rned

val

uem

etho

d us

ed m

ake

sens

e fo

r the

type

14.H

ow is

the

base

line

mai

ntai

ned?

8.

Ask

the

Man

ager

how

the

wor

k w

astim

e ph

ased

toac

hiev

e th

e2.

How

did

the

Man

ager

pla

n th

esc

ope

of w

ork

into

cont

rol a

ccou

nts?

11

.Ask

the

Man

ager

why

he/s

he c

hose

apa

rticu

lar t

ype

of3.

How

did

the

Man

ager

ens

ure

that

all

elem

ents

of

the

scop

e of

wor

k1.

Wha

t is

the

Man

ager

’s s

cope

of w

ork?

Doe

s th

eM

anag

er k

now

17.H

ow d

oes

the

CA

M u

pdat

e th

esc

hedu

le?

How

ofte

n ar

e

9 15 16 7 4 10 6 13 5 12 14 8 2 11 3 1 17

Ris

k R

atin

g

Pg 31

Risk Rating by Question

No Questions Cost Sched Tech Total9 9.Ask the Manager to identify any known risks that may affect the plan. 2.44 2.40 2.24 6.8615 15.What is the project’s risk management process? How is risk incorporated into the EAC? 2.43 2.45 2.24 6.6916 16.What is the process for reviewing and updating EACs? 2.20 2.07 1.75 5.787 7.Ask the Manager to show the team the schedule milestones used in the planning of the control

accounts. The Manager should discuss: 1.82 1.99 1.43 5.15

4 4.How did the Manager obtain the resources for assigned work? 1.83 1.90 1.43 5.1010 10.Are the Manager's resources adequately planned and correctly time phased to meet the plan?

Progress of each plan should relate to earned value 1.87 1.72 1.49 4.96

6 6.How did the Manager come to an agreement of the scope, hours, dollars and schedule? 1.85 1.72 1.55 4.8813 13.How does the Manager manage his/her plan? 1.68 1.61 1.50 4.785 5.Ask the Manager to describe the resource development process. 1.72 1.66 1.42 4.5912 12.Does the earned value method used make sense for the type of work being performed? Can you

measure performance? Is the Manager using milestones to measure accomplishments? Does the Manager use the LOE method; and is the LOE in support of measurable

1.76 1.61 1.30 4.58

14 14.How is the baseline maintained? 1.84 1.70 1.50 4.588 8.Ask the Manager how the work was time phased to achieve the schedule. 1.64 1.70 1.44 4.522 2.How did the Manager plan the scope of work into control accounts? 1.60 1.52 1.37 4.4811 11.Ask the Manager why he/she chose a particular type of earned value method? 1.64 1.54 1.30 4.443 3.How did the Manager ensure that all elements of the scope of work were planned? 1.57 1.59 1.38 4.421 1.What is the Manager’s scope of work? Does the Manager know what scope of work is his/her

responsibility? 1.45 1.38 1.30 4.08

17 17.How does the CAM update the schedule? How often are schedules updated? 1.58 1.58 1.31 4.06

Risk Rating

Pg 32

The Out-Brief

GoalsSuccess CriteriaObservationsSurvey ResultsConcernsRequest for Actions (RFAs)ImprovementsGeneral Comments

Pg 33

The Final Report

Author and submit report to Program Office, Project Office and appropriate line management

Pg 34

Final Report

Pg 35

Final Report

Pg 36

Walk - off