Pg 1
Planning and Conducting an Integrated Baseline Review (IBR)
Matthew T. Gonzales, Project Business ManagerJet Propulsion Laboratory
818-354-3629
Used with Permission
Pg 2
Communication
– Early– Frequent– Inclusive
Pg 3
‘If you can't describe what you are doing as a process, you don't know what you're doing.’
W. Edward Deming
Organization
Pg 4
IBR as a Process
Pre-Phase A Phase A Phase B Phase C Phase D
Formulation Implementation
Project Emphasis:
- System Definition- Identify potential contractors and teammates- Identify contractors above $20M
EVM Emphasis:
- EVM implementation team defined- Contractors develop EVM plan- Progress Assessment Visit (PAV); Contractor presents EVM plan- EVM team evaluates - If Validation Review is required, EVM teams works with local DCMA- Conduct follow-up PAV- Support KDP A
Pre-Phase A emphasizes the contractors and projects readiness
for EVM
Phase A emphasizes the EV Process and Initial Baseline
Plan
Phase B emphasizes the validity of baseline to complete
work
Phase C emphasizes use of EVM and the validity of EAC to
complete work
Phase D emphasizes use of EVM and the validity of EAC to
complete work
- Complete System Definition- Establish Requirements- Select Contractors- Award Contracts with EVM DRDs
- Contractor teams develop initial project cost and schedule baseline- Contractors submit EV reports per DRDs- Project Office conducts IBR with emphasis on Process
- Compliance to EV Process- Data Quality- CAM Knowledge- Change Control- Validity of Baseline relative to Scope of Work
- Support KDP B
Project Emphasis:
EVM Emphasis:
Project Emphasis:
EVM Emphasis:
- Project Requirements Baselined- Preliminary Design Established- LL Procurements Let
- EV System is mature- Reports Submitted- Logs Established- Change Control Exercised
- Project Office conducts IBR with emphasis on Risk to Reserves
- Project Plan content- Completeness of Planning- Change control process- Basis of Estimate for Cost and Schedule- Can design be built with planned resources- Overall robustness of EVM system- Reconciliation to Phase A Baseline
- Support KDP C
Project Emphasis:
EVM Emphasis:
- Integration of Flight HW and SW- Integrate Payload and Launch Vehicle
- Implement EV- Project Office conducts IBR with emphasis on Risk to Reserves
- Does Project Plan support I&T Program- Is EV data reliable as a predictor- Quality of VARs- Is the project using EVM data to manage- Test EAC process- Quality BOEs- Can I&T be completed for estimated cost- Is there sufficient schedule- Are Risks known and mitigated- Reliable reporting
- Support KDP E
Project Emphasis:
EVM Emphasis:
- Complete Flight and Ground Designs- Conduct component and SS CDRs- Build BB and EMs- Initiate Acceptance Test Program for Flight HW
- Implement EV- Project Office conducts IBR with emphasis on Risk to Reserves
- Does Project Plan support detail design- Is EV data reliable as a predictor- Quality of VARs- Is the project using EVM data to manage- Test EAC process- Quality BOEs- Can HW be built for estimated cost- Is there sufficient schedule- Are Risks known and mitigated- Reliable reporting
- Support KDP D
Pg 5
Key to Success
– Communication– Organization
Pg 6
Impetus
IBR Steps
Planning
Conducting
Final Report
Caucus and review
Out-brief
Pg 7
Impetus
Impetus for an Integrated Baseline Review Comply with customer requirements (7120.5d) Significant change to system design New Launch Date ‘Grass roots’ EAC Developed Over Target Baseline incorporated
Pg 8
Planning
In preparing for battle I have always found that plans are useless, but planning is indispensable.Dwight D. Eisenhower
Pg 9
ProjectRe-plan
EV DatabaseComplete
1st
MonthEV update
cycle
Review&
Analysis
Turbo estimates imported into MPM. PMB established.
MS Project milestones imported in MPM.
Weight applied. EV Plan Set
Project integrated baseline complete.
Set ‘Over Target Baseline’
Schedule updated on weekly cadence. Weekly metrics implemented.
Update to MPM database.
ACWP and BCWP for July month end
Generate and submit CPR Format 1 in
support of customer MMR
EV metrics analysis reveals
planning anomalies.
Corrective actions assigned/reviewed.
Implementation approved by PO and implemented
Changes to S/C and Mtl plans and EV
methods completeprior to Aug close.
Timeline
Jul ‘06
Aug ‘06
Planning Process Flow
IMS restructured for new launch date and
design change
Project IMS resources estimated and loaded into Turbo for pricing
Project Plan approved by SMD PMC.
Authorized to Proceed.
Events Process
Pg 10
Day 4 – Data Review/Informal Out-brief
Review Survey Results3-4 slide contractor feedback
ProjectCDR
ProjectIBR
Project office generates and briefs
IBR information package.
Corrective actions closed. EV and VARs
reviewed at project MMR
Contractor, HQ/DPO and project team
briefed on IBR plan
Update to MPM database.
ACWP and BCWP for July month end
Submit CPR Format 1. Planning changes incorporated. EV metrics improve.
EV metrics analysis reveals
planning anomalies.
Timeline
Sep ‘06
Nov ‘06
Planning Process Flow
IBR date negotiated with all affected
parties. Set for Nov.
Events Process
2ndMonth
EV update cycle
Oct ‘06
IBR scope, schedule and team
membership finalized.
IBR Coordination meeting with Review Team week before
review
Day 1 -IBR In-BriefJPL Project Office
BATC EV SurveillanceBATC Project Office
Day 2 – Data ReviewReview team audits
CAM notebooks.Team caucus.
Day 3 – CAM InterviewsProject Technical Lead
conducts interview. All team members complete survey
Formal Out-brief
Pg 11
Planning
Plan for your review Anticipated KDP Project Phase Working with contractors
Define IBR as early as possible Scope, Goals, Purpose Team membership with roles and responsibilities Required documents for review Work with contractor on logistics
Communicate 3 briefings in preparation for review Need to train interviewers as well as interviewees Provide contractor with questions ahead of time
Pg 12
Preamble
The In-Brief
Pg 13
John Wooden: A Coaching Legend
UCLA's basketball program has the international reputation of being No. 1. There is a major reason for that his name is John Robert Wooden, who announced his retirement after the 1974-75 season (his 27th campaign) as the Bruins' head coach with the winningest record in all of the sport's history. Wooden celebrated his 92nd birthday on Oct. 14, 2002.
Wooden concluded his 40 years as a head coach that season and his 885-203 overall career win-loss record (a percentage of .813) is unequaled. A large part of that success was at UCLA. In 27 years as Bruin coach, his teams registered 620 wins, and only 147 losses while earning far more national honors than any other university.
Under Wooden, UCLA won an unprecedented 10 NCAA championships, including seven consecutive (1966-73). Included in the string is one of the most amazing win streaks in all of sports, 38 straight NCAA tournament victories.
In addition, there is the all-time NCAA consecutive winning-streak record of 88 games over four seasons, which included consecutive 30-0 seasons in 1971-72 and 1972-73. UCLA also won 149 of 151 games in Pauley Pavilion during his Bruin tenure.
John Wooden is the only coach to compile four undefeated seasons of 30-0 and his Bruin teams captured 19 conference championships (the record of which Wooden is most proud).
Coach Wooden is the first person to be inducted into the National Basketball Hall of Fame as both a player and coach
Pg 14
John Wooden Quotes“It’s the little details that are vital. Little things make big things happen”
“If you don’t have time to do it right, when will you have the time to do it over”
“Winning takes talent, to repeat it takes character”
“It is what we learn after we know it all that really counts”
“Be prepared and be honest”
“Things turn out best for the people who make the best of the way things turn out”
Pg 15
Use your review In-brief as a time to
Inspire
Pg 16
The In-Brief
Project OverviewGoalsSuccess CriteriaProcess OverviewDocumentsLogistics/ScheduleGeneral Comments
Pg 17
• Control Account Managers Notebooks
– Turbo reports
– Schedules , by WBS element and by IPT
– Control Account Plans to Work Package level of detail and Earned Value Milestone Report
– Current WBS and WBS Dictionary
– Estimating methodology and documentation to enable assessment of the basis of the estimates
– Work Authorization Documents for authorized control accounts.
– Performance Measurement Reports (CPR Format 1)
– Budget Change Requests (if applicable)
– Approved liens and KVRs and recovery plans
• Other Ball documentation required for the IBR:
– Current Contract SOW
– BATC System Description, Kepler EVM Plan Supplement
• Reviewing for completeness and consistency in the plans.• Is the work fully accounted for and plan appropriately.• Are all resources planned and available when required.• Is the time phasing of cost and schedule consistent.• Is EV planned in a way to yield accurate results.• Are analysis tools available and utilized.• Are changes tracked and managed• Are they adhering to the BATC way of planning and managing projects.
Data Review - Wed, Nov. 15th 8am – 5pm
Pg 18
Conducting Interviews
Pg 19
Conducting InterviewsSelect WBS elements for review
Risk driven (technical and/or programmatic) Phase specific
Select appropriate questions Phase specific Basis of Estimate Discern risk Communicate early
Develop interview teams Led by technical representative Small teams Provide training on questions
Discussions/Interviews Conversational ~ 2 hours Document results
Pg 20
CAM Interviews – Nov. 16th 8am – 5pm
DRAFTWBS WBS Description Day Interview Time6.1.1 FS Management Thurs 08:00 – 10:005.2.6 Attitude Control Thurs 08:00 – 10:00 5.2.8 Communications Thurs 08:00 – 10:00 6.1.3 FS Mission Assurance Thurs 10:00 – 12:00 6.3.2 FPAA Thurs 10:00 – 12:006.3.3 Electrical Thurs 10:00 – 12:006.3.4 Telescope Thurs 1:00 – 3:00 5.2.3 Mechanical Subsystem Thurs 1:00 – 3:00 6.3.5 Photo Integration Test Thurs 3:00 – 5:005.2.11 Spacecraft I&T Thurs 3:00 – 5:005.3 System ATLO
IBR CAM Interview Schedule
WBS WBS DescriptionPeg Frerking
Leslie Livesay
Jim Fanson
Sarah Gavit
Michael Moore
Renee Leck
Pam Marcum
Jack Graham
Steve McClard
Nate Porter
John Keidel
Robert Bitten
Dan Graham
Maura Fujieh
Bill Epping
Tim Clark
Cal Cambers
Joe Fischetti
6.1.1 FS Management 1 1 1 15.2.6 Attitude Control 1 1 1 15.2.8 Communications 1 1 1 16.1.3 FS Mission Assurance 1 1 1 16.3.2 FPAA 1 1 1 16.3.3 Electrical 1 1 1 16.3.4 Telescope 1 1 1 15.2.3 Mechanical Subsystem 1 1 1 16.3.5 Photo Integration Test 1 1 1 15.2.11 Spacecraft I&T 1 1 1 15.3 System ATLO
SupportIBR CAM Interview Schedule NASA Reps Discovery Program OfficeTechnical Leads
Pg 21
CAM QuestionnaireQuestions Rating Low Med High
1.What is the Manager’s scope of work? Does the Manager know what scope of work is his/her responsibility? Cost 1 2 3 4 5
–The Manager should be able to show the team a Statement of Work (SOW) paragraph, a WBS, PWBS, or CWBS narrative, and a Work Authorization Document that correlates to the task. Schedule 1 2 3 4 5
Comments:Technical 1 2 3 4 5
2.How did the Manager plan the scope of work into control accounts? Cost 1 2 3 4 5
–The SOW defines the effort. The WBS, PWBS, or CWBS provides specifics, such as work definition. The work authorization and change documentation should show information such as the dollars/hours, period of performance, and description of the scope of work and any changes.
Schedule 1 2 3 4 5
Comments:Technical 1 2 3 4 5
3.How did the Manager ensure that all elements of the scope of work were planned? Cost 1 2 3 4 5
–The Manager should be able to show the team the work packages breaking down the scope of work, and the budget associated with each work package. The sum of the work packages should be the control account budget. Each control account should contain a specific scope of work.
Schedule 1 2 3 4 5
Comments:Technical 1 2 3 4 5
Pg 22
Caucus and Review
Pg 23
Caucus and Review
Review survey responses By interview, by question Look for outliers and discuss. Opportunity to share thoughts and observations Normalize metrics
Do metrics support subjective risk observations
Pg 24
Caucus and ReviewLow Med High
12 36 60WBS CAM Intvr No Questions Cost Sched Tech Total5.2.6 CS SG 1 1.What is the Manager’s scope of work? Does the Manager
know what scope of work is his/her responsibility? 1 2 3 6
5.2.6 CS DG 1 1.What is the Manager’s scope of work? Does the Manager know what scope of work is his/her responsibility? 1 1 2 4
5.2.6 CS RB 1 1.What is the Manager’s scope of work? Does the Manager know what scope of work is his/her responsibility? 1 1 1 3
5.2.6 CS BE 1 1.What is the Manager’s scope of work? Does the Manager know what scope of work is his/her responsibility? 1 1 1 3
4 5 7 165.2.6 CS DG 2 2.How did the Manager plan the scope of work into control
accounts? 1 1 1 3
5.2.6 CS SG 2 2.How did the Manager plan the scope of work into control accounts? 1 1 1 3
5.2.6 CS RB 2 2.How did the Manager plan the scope of work into control accounts? 1 1 1 3
5.2.6 CS BE 2 2.How did the Manager plan the scope of work into control accounts? 1 1 1 3
4 4 4 125.2.6 CS DG 3 3.How did the Manager ensure that all elements of the scope of
work were planned? 2 2 2 6
5.2.6 CS SG 3 3.How did the Manager ensure that all elements of the scope of work were planned? 1 1 1 3
5.2.6 CS RB 3 3.How did the Manager ensure that all elements of the scope of work were planned? 1 1 1 3
5.2.6 CS BE 3 3.How did the Manager ensure that all elements of the scope of work were planned? 1 1 1 3
5 5 5 155.2.6 CS BE 4 4.How did the Manager obtain the resources for assigned work? 3 3 3 9
5.2.6 CS RB 4 4.How did the Manager obtain the resources for assigned work? 2 2 1 5
5.2.6 CS DG 4 4.How did the Manager obtain the resources for assigned work? 1 1 1 3
5.2.6 CS SG 4 4.How did the Manager obtain the resources for assigned work? 1 1 1 3
Risk Rating
Totals
Totals
Totals
Pg 25
Survey Results
…and the survey says…
Pg 26
Cost Risk Rating
Cost RiskWBS No Questions Cost6.1.3 FS MISSION ASSURANCE 2.776.3.2 FPAA 2.386.3.4 TELESCOPE 2.295.2.6 ATTITUDE CONTROL 1.846.3.5 PHOTOMETER I&T 1.756.3.3 ELECTRICAL 1.565.2.3 MECHANICAL 1.526.1.1 FLIGHT SEGMENT MANAGEMENT 1.445.2/5.3 SPACECRAFT/ATLO 1.395.2.8 COMMUNICATIONS 1.25
0.001.002.003.004.005.00
FS M
ISSI
ON
ASS
UR
AN
CE
FPA
A
TELE
SCO
PE
ATT
ITU
DE
CO
NTR
OL
PHO
TOM
ETER
I&T
ELEC
TRIC
AL
MEC
HA
NIC
AL
FLIG
HT
SEG
MEN
TM
AN
AG
EMEN
T
SPA
CEC
RA
FT/A
TLO
CO
MM
UN
ICA
TIO
NS
Ris
k R
atin
g
Pg 27
Schedule Risk Rating
Schedule RisksWBS No Questions Sched6.3.2 FPAA 2.396.3.4 TELESCOPE 2.286.1.3 FS MISSION ASSURANCE 2.065.2.6 ATTITUDE CONTROL 1.836.3.5 PHOTOMETER I&T 1.766.3.3 ELECTRICAL 1.655.2/5.3 SPACECRAFT/ATLO 1.525.2.3 MECHANICAL 1.525.2.8 COMMUNICATIONS 1.376.1.1 FLIGHT SEGMENT MANAGEMENT 1.36
0.001.002.003.004.005.00
FPA
A
TELE
SCO
PE
FS M
ISSI
ON
ASS
UR
AN
CE
ATT
ITU
DE
CO
NTR
OL
PHO
TOM
ETER
I&T
ELEC
TRIC
AL
SPA
CEC
RA
FT/A
TLO
MEC
HA
NIC
AL
CO
MM
UN
ICA
TIO
NS
FLIG
HT
SEG
MEN
TM
AN
AG
EMEN
T
Ris
k R
atin
g
Pg 28
Technical Risk Rating
Technical RisksWBS No Questions Tech6.3.2 FPAA 2.196.3.4 TELESCOPE 2.165.2.6 ATTITUDE CONTROL 1.646.3.5 PHOTOMETER I&T 1.596.3.3 ELECTRICAL 1.385.2/5.3 SPACECRAFT/ATLO 1.356.1.3 FS MISSION ASSURANCE 1.295.2.8 COMMUNICATIONS 1.245.2.3 MECHANICAL 1.246.1.1 FLIGHT SEGMENT MANAGEMENT 1.20
0.001.002.003.004.005.00
FPA
A
TELE
SCO
PE
ATT
ITU
DE
CO
NTR
OL
PHO
TOM
ETER
I&T
ELEC
TRIC
AL
SPA
CEC
RA
FT/A
TLO
FS M
ISSI
ON
ASS
UR
AN
CE
CO
MM
UN
ICA
TIO
NS
MEC
HA
NIC
AL
FLIG
HT
SEG
MEN
TM
AN
AG
EMEN
T
Ris
k R
atin
g
Pg 29
Risk Rating by WBS
WBS No Questions Cost Sched Tech Total6.1.1 1 1.What is the Manager’s scope of work? Does the Manager know what
scope of work is his/her responsibility? 1.33 1.33 1.40 3.83
6.1.1 2 2.How did the Manager plan the scope of work into control accounts? 1.17 1.33 1.20 3.50
6.1.1 3 3.How did the Manager ensure that all elements of the scope of work were planned? 1.20 1.40 1.20 3.17
6.1.1 4 4.How did the Manager obtain the resources for assigned work? 1.33 1.33 1.00 3.336.1.1 5 5.Ask the Manager to describe the resource development process. 1.40 1.17 1.00 3.006.1.1 6 6.How did the Manager come to an agreement of the scope, hours,
dollars and schedule? 1.50 1.33 1.00 3.50
6.1.1 7 7.Ask the Manager to show the team the schedule milestones used in the planning of the control accounts. The Manager should discuss: 1.00 1.00 1.00 2.83
6.1.1 8 8.Ask the Manager how the work was time phased to achieve the schedule. 1.40 1.33 1.00 3.17
6.1.1 9 9.Ask the Manager to identify any known risks that may affect the plan. 1.67 1.17 1.20 3.83
6.1.1 10 10.Are the Manager's resources adequately planned and correctly time phased to meet the plan? Progress of each plan should relate to earned value
1.33 1.17 1.00 3.33
6.1.1 11 11.Ask the Manager why he/she chose a particular type of earned value method? 1.33 1.33 1.40 3.83
6.1.1 12 12.Does the earned value method used make sense for the type of work being performed? Can you measure performance? Is the Manager using milestones to measure accomplishments? Does the Manager use the LOE method; and is the LOE in support of measurable
1.20 1.00 1.00 2.50
6.1.1 13 13.How does the Manager manage his/her plan? 1.50 1.50 1.20 4.006.1.1 14 14.How is the baseline maintained? 2.00 1.80 1.80 4.676.1.1 15 15.What is the project’s risk management process? How is risk
incorporated into the EAC? 1.50 1.50 1.50 3.00
6.1.1 16 16.What is the process for reviewing and updating EACs? 2.00 1.75 1.50 3.506.1.1 17 17.How does the CAM update the schedule? How often are schedules
updated? 1.67 1.67 1.00 4.33
FLIGHT SEGMENT MANAGEMENT Total Scores by WBS 1.44 1.36 1.20 3.49
Risk Rating
Pg 30
Risk Rating by Question
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
9.A
sk th
e M
anag
erto
iden
tify
any
know
n ris
ks th
atm
ay a
ffect
the
plan
.15
.Wha
t is
the
proj
ect’s
risk
man
agem
ent
proc
ess?
How
is16
.Wha
t is
the
proc
ess
for
revi
ewin
g an
dup
datin
g E
AC
s?7.
Ask
the
Man
ager
to s
how
the
team
the
sche
dule
mile
ston
es u
sed
in4.
How
did
the
Man
ager
obt
ain
the
reso
urce
s fo
ras
sign
ed w
ork?
10
.Are
the
Man
ager
'sre
sour
ces
adeq
uate
ly6.
How
did
the
Man
ager
com
e to
an a
gree
men
t of
the
scop
e, h
ours
,13
.How
doe
s th
eM
anag
er m
anag
ehi
s/he
r pla
n?
5.A
sk th
e M
anag
erto
des
crib
e th
ere
sour
cede
velo
pmen
t12
.Doe
s th
eea
rned
val
uem
etho
d us
ed m
ake
sens
e fo
r the
type
14.H
ow is
the
base
line
mai
ntai
ned?
8.
Ask
the
Man
ager
how
the
wor
k w
astim
e ph
ased
toac
hiev
e th
e2.
How
did
the
Man
ager
pla
n th
esc
ope
of w
ork
into
cont
rol a
ccou
nts?
11
.Ask
the
Man
ager
why
he/s
he c
hose
apa
rticu
lar t
ype
of3.
How
did
the
Man
ager
ens
ure
that
all
elem
ents
of
the
scop
e of
wor
k1.
Wha
t is
the
Man
ager
’s s
cope
of w
ork?
Doe
s th
eM
anag
er k
now
17.H
ow d
oes
the
CA
M u
pdat
e th
esc
hedu
le?
How
ofte
n ar
e
9 15 16 7 4 10 6 13 5 12 14 8 2 11 3 1 17
Ris
k R
atin
g
Pg 31
Risk Rating by Question
No Questions Cost Sched Tech Total9 9.Ask the Manager to identify any known risks that may affect the plan. 2.44 2.40 2.24 6.8615 15.What is the project’s risk management process? How is risk incorporated into the EAC? 2.43 2.45 2.24 6.6916 16.What is the process for reviewing and updating EACs? 2.20 2.07 1.75 5.787 7.Ask the Manager to show the team the schedule milestones used in the planning of the control
accounts. The Manager should discuss: 1.82 1.99 1.43 5.15
4 4.How did the Manager obtain the resources for assigned work? 1.83 1.90 1.43 5.1010 10.Are the Manager's resources adequately planned and correctly time phased to meet the plan?
Progress of each plan should relate to earned value 1.87 1.72 1.49 4.96
6 6.How did the Manager come to an agreement of the scope, hours, dollars and schedule? 1.85 1.72 1.55 4.8813 13.How does the Manager manage his/her plan? 1.68 1.61 1.50 4.785 5.Ask the Manager to describe the resource development process. 1.72 1.66 1.42 4.5912 12.Does the earned value method used make sense for the type of work being performed? Can you
measure performance? Is the Manager using milestones to measure accomplishments? Does the Manager use the LOE method; and is the LOE in support of measurable
1.76 1.61 1.30 4.58
14 14.How is the baseline maintained? 1.84 1.70 1.50 4.588 8.Ask the Manager how the work was time phased to achieve the schedule. 1.64 1.70 1.44 4.522 2.How did the Manager plan the scope of work into control accounts? 1.60 1.52 1.37 4.4811 11.Ask the Manager why he/she chose a particular type of earned value method? 1.64 1.54 1.30 4.443 3.How did the Manager ensure that all elements of the scope of work were planned? 1.57 1.59 1.38 4.421 1.What is the Manager’s scope of work? Does the Manager know what scope of work is his/her
responsibility? 1.45 1.38 1.30 4.08
17 17.How does the CAM update the schedule? How often are schedules updated? 1.58 1.58 1.31 4.06
Risk Rating
Pg 32
The Out-Brief
GoalsSuccess CriteriaObservationsSurvey ResultsConcernsRequest for Actions (RFAs)ImprovementsGeneral Comments
Pg 33
The Final Report
Author and submit report to Program Office, Project Office and appropriate line management
Pg 34
Final Report
Pg 35
Final Report
Pg 36
Walk - off