Infrastructure Lifetimes & Workforce Analytics · 2018-03-20 · Workforce Analytics DATAWorks 2018 Erika Taketa & Bill Romine ... —LLNL DM backlog at $528M in 2017 ... Predictive

LLNL-PRES-XXXXXX

This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under contract DE-AC52-07NA27344. Lawrence Livermore National Security, LLC

Infrastructure Lifetimes & Workforce AnalyticsDATAWorks 2018

Erika Taketa & Bill Romine

March 21, 2018

LLNL-PRES-xxxxxx

2

Outline

▪ Infrastructure Lifetimes

— Standard commercial practice

— LLNL’s “scientific” approach

— Quantitative refinement

▪ 15 minute break

▪ Workforce Analytics

— Workforce modeling

— Persistence of training effect

— Utilization for Life Extension

LLNL-PRES-xxxxxx

3

Infrastructure Lifetimes

LLNL-PRES-xxxxxx

4

▪ LLNL facilities and infrastructure portfolio: $6.9B

▪ Over 700 types of assets (ACs, pipes, roofs)

▪ Deferred maintenance (DM)— DOE definition: maintenance and repairs that were not performed when

they should have been or were scheduled to be and which are put off or delayed for a future period

— “should have been” includes assets that exceed estimated service life— LLNL DM backlog at $528M in 2017

▪ Are assets exceeding their estimated service lives truly approaching imminent failure?

Infrastructure value at Lawrence Livermore

LLNL-PRES-xxxxxx

5

▪ ESL is determined largely by unscientific survey: “At what age did the asset fail?”

▪ ESL is the mean of the ages at “failure”

▪ Why does this not estimate asset type survival correctly?

▪ Do any assets fail before achieving their ESL?

▪ Does the presence of survivors the same age as failures count?

Accepted practice is to replace assets at their ESLEstimated service life (or ESL) varies by asset type

Accepted practice contains fundamental flawsPublishing values that do not depend on individual experience is universal practice

LLNL-PRES-xxxxxx

6

How we do itAvailable data bounded by observation window

LLNL-PRES-xxxxxx

7

How we do it20 years of data collection

▪ Collected data from 1996-2015

▪ 78,526 records, 14,803 failures

▪ Model each asset type separately

▪ Unknown age of failure beyond 2015: right censoring

▪ Unknown assets that failed before 1996: left truncation

Left truncation is omitted even from “expert” commercial software

LLNL-PRES-xxxxxx

8

▪ Survival model incorporating right censoring and left truncation

𝐿 ∝ෑ𝑖∈𝐷

𝑓(𝑥𝑖)ෑ

𝑖∈𝑅

𝑆(𝐶𝑖)ෑ

𝑖∈𝐿

ൗ1 𝑆(𝑇𝑖)

▪ Generalized gamma distribution: GenGamma(k, β, θ)

▪ R packages— survival— flexsurv— eha

How we do itModel building and implementation

D: Complete failure data subset xi: Age of equipment failure

R: Right-censored data subset Ci: Age of equipment at right censor time

L: Left-truncated data subset Ti: Age of equipment at left truncation time

LLNL-PRES-xxxxxx

9

Service lives depend on methodologyAsset specific ESL underestimates true experience

LLNL-PRES-xxxxxx

10

Survivors augment known failure information LLNL service lives consistently older in 57 asset classes

LLNL-PRES-xxxxxx

11

▪ Commercial property managers rely on fixed schedule replacement despite the decades old shift to reliability-centered maintenance and statistical analysis in the aircraft industry

▪ LLNL’s policy of repeated 3 year inspections based on survival curve based probabilities of failure for the most frequently replaced assets is a total life-cycle cost minimization strategy

▪ Cost savings never come for free

▪ Management strategies like having replacement assets available ahead of predicted failures help mitigate break in service risk

SummaryRealities

Fixed schedule maintenance is embraced to minimize observed breaks in service Survival curve-based prediction combined with inspection minimizes cost

LLNL-PRES-xxxxxx

12

▪ RCM Guide, NASA, 2008 (references in Appendix C)

▪ Hiller, C.C., Determining Equipment Service Life, ASHRAE Journal, vol. 42, no. 8, August 2000

▪ Jordan, C.W., Life Contingencies, The Society of Actuaries, 1991

▪ Klein, J.P. and Moeschberger, M.L., Survival Analysis: Techniques for Censored and Truncated Data, Springer, 2005

▪ Meeker, W.Q. and Escobar, L.A., Statistical Methods for Reliability Data, Wiley, 1998

▪ Nowlan, F.S. and Heap, H.F., Reliability-Centered Maintenance, Dolby Access Press, San Francisco, CA 1978.

▪ Romine, B.,Mattimore, B., and Taketa, E., LLNL Infrastructure Component Replacement Prediction, LLNL-BOOK-733884, June 2017

▪ Sondalini, M., A Common Misunderstanding about Reliability Centered Maintenance, www.lifetime-reliability.com

▪ Venables, W.N. and Smith, D.M., An Introduction to R, Network Theory, 2004

ReferencesInfrastructure Lifetimes

http://www.lifetime-reliability.com/

LLNL-PRES-xxxxxx

13

15 minute breakPlease return to hear the conclusion of this talk

LLNL-PRES-xxxxxx

14

Workforce Analytics

LLNL-PRES-xxxxxx

15

▪ Several statistical techniques have been applied to successfully predict attrition[1],[2].

▪ Competence (or career progression) has proven more challenging[3] to model.

▪ Competence in some areas depends on the challenges of meeting customer requirements.

▪ Life Extension Programs (LEPs):— Draw on and build unique competencies— Labor estimates translate into program budgets[4]-[6]

— Prioritization involves both design and production scheduling as-well-as customer requirements

▪ Even abstracted LEP data is hard to acquire.

Workforce ModelingBrief overview

Attrition modeling is a tractable problem. Competency modeling is less well developed.

LLNL-PRES-xxxxxx

16

▪ Once upon a time corporate employers offered defined benefit pension plans to encourage employee loyalty.

— Defined benefit pension plans use a benefit formula weighting both salary and service to compute the present value of a monthly life annuity at normal retirement age• Not everyone stays to vest in the benefit• Each invested dollar today should be worth more in the future

— The corporation needs to make annual payments to pre-fund the liability for each pension plan participant• Actuarial models use numeric tables derived from statistical models of general experience to

calculate both the present value of the monthly life annuity at retirement but also discounts based on pre-vesting voluntary terminations and for anticipated investment return

▪ Expectations have changed and defined benefit pension plans are rare.

▪ Workforce models still provide useful information for planning and authorizing hiring. Both LLNL & LANL use statistical models for this purpose.

Some uses for Workforce Modeling (1) Anticipating hiring in the absence of exogenous events

Defined benefit pension plans provide a starting point for attrition modeling. LLNL, LANL and SNL (NNSA’s Design Agencies) all use statistical attrition modeling.

LLNL-PRES-xxxxxx

17

▪ Once upon a time the credibility of the National Security Enterprise (NSE) was not in doubt.— With notable negotiated gaps this credibility was demonstrated through continual testing

▪ Expectations have changed. In the aftermath of the cessation of underground testing it became common practice to survey the Design Agency staff according to competence categories of skills deemed essential to Stockpile Stewardship.

▪ Given that the specialized skills used in Stockpile Stewardship reside within a classified context, can the process by which knowledge is preserved by described in terms of career progression from new hire to autonomous “master”?

Some uses for Workforce Modeling (2) Convincing others workforce competency is being preserved

Categorization according to competency leads naturally to career progression. The distinction between what we can imagine and what we can prove is important.

LLNL-PRES-xxxxxx

18

Simulating trained population attritionAssuming retirement at empirically determined mean age

LLNL-PRES-xxxxxx

19

Trained population age distributionEmpirical densities capture discipline variation

LLNL-PRES-xxxxxx

20

▪ Once upon a time the credibility of the Nuclear Security Enterprise was not in doubt.— With notable negotiated gaps this credibility was demonstrated through continual testing

▪ Behavioral norms have changed. But management of research and development costs continue to be based on the “Norden-Rayleigh” model developed in the 1960s to model the cost growth of software projects.— P. V. Norden. “Useful Tools for Project Management.” In B. V. Dean (ed.), Operations Research in

Research and Development. New York: John Wiley and Sons, 1963— D. A. Lee, M. R. Hogue, and D. C. Hoffman. “Time histories of expenditure for defense acquisition

programs in the development phase—Norden-Rayleigh and other models.” Presented at the Annual Meeting of the International Society of Parametric Analysis, 1993

▪ For the first time since the end of the “design and test” era LLNL is executing a Life Extension Program. How well does the labor allocation match the model?

Some uses for Workforce Modeling (3) Convincing others specific project costs are realistic

A favorite tool of budget estimators is the Rayleigh distribution. Actual experience suggests a more complex model fits experience.

LLNL-PRES-xxxxxx

21

Utilization of disciplinesDifferent behaviors increase labor cost model complexity

LLNL-PRES-xxxxxx

22

1. Pantano, J., Modeling of ASC Workforce at LANL, NNSA Critical Skills Workshop, May 2011

2. Romine, B. and Pantano, J., Modeling Attrition, Predictive Analytics World, Chicago, 2013

3. Bartholomew, D.J., Statistical Techniques for Manpower Planning, Wiley, 1991

4. Jones, A.R., Project Team Sizing and Cost Forecasting using Norden-Rayleigh Curves, BAE Systems, 2011

5. Brown, T.W., Forecasting Research & Development Program Budgets Using the Weibull Model, Thesis, Air Force Institute of Technology, 2002

6. Davis, D., Christle, G., Abba, W., Using the Rayleigh Model to Assess Future Acquisition Contract Performance and Overall Contract Risk, Center for Naval Analysis, 2009

ReferencesWorkforce Analytics