2015
Analytic Sciences Inc.
PERFORMANCE RECOMMENDATIONS
DARLENE FERRI-KURJACK
PURDUE UNIVERSITY
EDCI 52800
Analytic Sciences Intervention Recommendations
1
Analytic Sciences Inc. Context and Performance Issues
Analytic Sciences, Inc. is a private defense contractor based in Washington, D.C. The firm
describes itself as “a leading provider of advanced systems engineering, integration and decision-support
services to Federal Intelligence and Defense communities”. Analytic Sciences, Inc. prides itself on
customer satisfaction through direct contractor support.
The project work conducted by Analytic Sciences is issued and funded through individual task
orders (TOs). Task orders vary in description and are goaled to support the overall strategic plan of the
government-funded entity. The Customer, deemed the government-funded entity, declares the project
needs relevant to the departmental goals and the contractor assigned support staff provides solutions to
project goals and completes the work required to deliver a funded project. Once a project is approved, it is
scheduled as a deliverable, of which only the customer can change direction, extend, cancel a project, or
accept it as completed.
The customer has final oversight over the acceptance and direction of a project through the plan
and define phase. In result, the work environment consists of regularly scheduled working sessions and
meetings with the customer, as well as delivery checkpoints. In addition, before a product is completed, a
final review is conducted by the customer to provide input to the contractor for changes required for final
approval. Once final changes resulting from customer inputs are implemented the customer reviews the
product and makes requests for additional changes or grants final approval.
There is an abundance of interconnectivity among support teams relative to the integration of
projects for the federally contracted entity and staff members may be supporting several customers. In
result, a typical workday can require the attendance of one to three meetings. Contractor support teams
have become accustomed to working beyond a 40-hour workweek to meet customer demands of
milestones and deliverables in consideration of a fast-paced compressed schedule. The direct contractor
support staff works on salary, however any hours over the contracted 40 hours are billed additionally to
the customer. Team bonuses are aligned to customer satisfaction surveys.
Analytic Sciences Intervention Recommendations
2
Performance Issues
The project manager has been alerted to certain performance issues that pose conflict with the
contracted task order and require investigation and remedial tactics. The issues of concern that are
reflective of a current state of performance deficiency include:
Not Meeting Deliverable Dates for Projects: The desired state of performance is to meet initial
deliverable dates for projects. Not meeting or extending deliverable dates can pose negatively
relative to bi-annual customer satisfaction surveys, and is subject to explanation to the funding
and contracting offices as it effects budgetary and contractual obligations.
Overtime is exceeding 40 hours consistently, rather than by exception: The desired state of
performance is for contractor support staff to maintain a 40-hour workweek on a consistent basis,
with overtime as an exception. Overtime expenses must be justified to contracting officials that
represent both customer and contractor budgeting interests. There is currently no record of
explanation or approval of overtime in place.
An analysis was conducted with focus on project deliverable and overtime issues. Each performance
gap is outlined in Table 1, Appendix A, including symptoms, the desired and actual states, probable root
causes, intervention summaries, and classification according to Gilbert’s Behavioral Engineering Model
(Stolovitch & Keeps, 2004). The performance deficiencies analyzed exposed several contributory
catalysts that exist in the environment, training needs and emotional barriers.
Intervention Development
While the contractor support teams are highly motivated to provide above average support to their
assigned customers, there are several barriers in their work environment that impede their ability to
streamline workload and meet deliverables scheduled or impromptu without exceeding a 40 hour work
week. The Team Lead also faces challenges in the form of resources and training to assist in overseeing
the staff relative to project planning and equitable distribution of work. In order to demonstrate the types
of interventions that could improve performance, several interventions such as Training, Performance
Aids, and Environmental and Emotional classifications are further developed.
Analytic Sciences Intervention Recommendations
3
Recommended Interventions
Based on the analyses, a number of interventions are recommended to assist Analytic Sciences with
the performance issues that require high-priority remediation at this time, being adherence to deliverable
schedules and overtime management. The categories of intervention types are based on those proposed by
Stolovitch and Keeps, 2004. A timeline that represents the time required to implement interventions is
indicated in Appendix C.
Learning Interventions
Lack of skills relative to managing constraints, such as project scope, time, budget and resources
were consistently realized not only among the Team Leads but among the customer end as well.
Observations of team leads often deterred complete control to the customer, instead of one of
informational support. Initial interviews, observations and surveys noted in Appendix A reflect this same
dynamic within the customer/contractor exchange. Given that, the typical role of Team Leads is to guide
the customer it becomes clear that skill training relative to Project Management is needed to support the
customers at the high quality level of customer support that Analytic Sciences reflects in their culture.
Team leads were met with individually by Project Managers to confirm their need and interest in Project
Management training and most all expressed interest in this training to assist them in their tasking. The
training is optional. This lack of skill has posed a contributory factor to managing project scope and
guidance, and contributed to increases to overtime hours due to lack of assignment of work equitably
among support staff. The benefit of this instruction will assist to remedy the performance deficits just
listed as well as improve communication to the customer that accurately conveys the budgetary and
potential constraints to resources that certain changes and impromptu tasking may convey, thereby
leading to a more informed decision making process. Overall, project management training will provide
the necessary tactics to leverage performance among support staff and with the customer as noted in
Table 1.
Table 1: Learning Interventions Overtime/
Deliverables
Intervention Type: Project Management Training-Classroom Training (Live or Virtual)
Team Leads (Participation in company paid Project Management Certification Courses through
Analytic Sciences Intervention Recommendations
4
a certified vendor). The requirements for this training are noted in Appendix B: B1.
Non-learning Interventions
Performance Aids
Through the observation of customers and support staff at status and project planning meetings, it
was noted that there was consistency in difficulties for team leads to track customer changes to direction
outside of meeting notes, where the customer when challenged with the misinterpretation of a change
would simply state “ the team did not capture direction correctly”. Interviews with Team Leads confirmed
communication to be a consistent issue with customers, where they can only assume there was a lack of
mutual communication or the customer simply did not fully remember the details of the new project scope
that had been conveyed a week or two prior. Interview with customers yielded the ownership of creative
freedom for changes to deliverables as noted through the analysis of responses to survey questions in
Appendix A. One intervention that can lend assistance to the issue is a formal basis for tracking changes
that is acheived through the development of an Electronic Tracking Support Tool such as a the electronic
cataloguing of Change Request Forms as described in Table 2 and Appendix B. Such a tool leaves less to
interpretation between the contractor support staff and the customer, thereby improving communication
and the prompting of key considerations reflective of the requested change that may also effect
departments and projects both internal and external to the team. Most importantly in-house staffing can
support the development of this tool as a resource financed through the budgeting office.
Table 2: Performance Aids Overtime N/A
Deliverables Intervention Type: Job Ai- Electronic Project Change Tracking Support Tool
This tracking system is necessary for project management reports and conveying printed paper
resources for tracking product changes with customer approval/signature and acknowledgement of
how change of scope will effect delivery date. Information is required in formal representation by
the contractor funding and contracting offices and serves to assist the project manager with prompt
project status awareness. The project tracking system should provide information linked to a project
identifier (ID). The document is completed and stored electronically in a database for quick access.
The database is updated per the process request of a change and is managed by the team leads and
approved by both project manager and customer. A printed copy is maintained with signatures by
both the Team Lead, Customer and Project Manager. The contracting office receives a copy with the
original signature of the printed form which details information fields is specified in the prototype of
this specific job aid which is found in Appendix B: B2 (Figures 1-3, 5).
Analytic Sciences Intervention Recommendations
5
Environmental
The environment poses several challenges that contribute to the identified deficiencies. The
analysis revealed that overtime is open to the need of the staff to complete work. Implementing an
overtime process that requires approval from the team lead serves the purpose of awareness and provides
the team lead the opportunity to provide management support to the staff in order manage, assign and
prioritize tasking equitably among the team. The goal of this process is to diminish needed overtime by
placing emphasis on teamwork within and across teams, specifically since a team member may be
supporting several customers at once. Table 3 presents a detailed proposal of overtime approval process.
The analysis revealed that the customer had little knowledge relative to overtime costs being charge
relative to project changes and extended deliverables. The second process to improve overtime awareness
involves the daily record keeping of overtime per project identifier. The data can be printed from a simple
database and provided monthly to the customer for review and informational awareness. Table 3 provides
an overview and a prototype of the printed record and can be viewed in Appendix B2: Figure 4.
Analysis of the environment relative to the dynamics between contractor support staff and the
customers revealed communication issues for changes in project scope and extended deliverables as a
result. In order to remediate this, a process intervention is linked to job aid in the form of an Electronics
tracking tool that is completed by the team lead (see Appendix B: B2 Figure 5). The form conveys
detailed information relative to the project chance rationale and impacts of the change. The customer will
review the printed document and approve or make edits. A signature of interested parties is required to
ensure communication of direction is mutually understood and approved by the authority of the customer.
The printed documents with signature will be conveyed to all stakeholders of interest, being the customer,
project manager, and contractor-funding department and contracting office for review, update of status
and record keeping.
Analytic Sciences Intervention Recommendations
6
Table 3: Environmental Interventions Overtime Intervention Type: Process Redesign Overtime
Develop a process for overtime approval, where anticipated overtime is requested daily by
contractor support staff and approved by the allocated team lead.
o Support from team members should be allocated for those workers coping with high
workload.
o Emphasis on teamwork within and across organizational units is conveyed to staff
Justification: Management intervention, supports an awareness overtime, which is responded
through the balance of, managed, assigned and prioritized tasking. the equitable distribution of
workload among and .;5
Process Overview: Overtime is requested in person or via email to team lead with rationale. Team
Lead sends approval/denial response via email for recording purposes. Team Lead may require
project-planning communications of staff to re-allocate or pair staff to complete tasking.
Intervention Type: Increased Management Support (Team Lead)
Managers provide clear direction regarding organizational priorities to assist employees to
focus on the highest-value tasks to meet deadlines.
Intervention Type: Provision of Information
Customers supported are provided a monthly project report of overtime hours and budget status
for additional staffing hours because of added tasking to impromptu deadlines. The goal is to
provide cost awareness and balance tasking. See Appendix B: B2, Figure 4.
Deliverables Intervention Type: Process Change-Deliverables
Process changes relative to the creation of internal guidelines and processes for customer and
contractor relative to project changes, where the projects are tracked and changes to scope and
deliverables are approved through signed customer consent. The justification for this process
extends to the support the process provides to the concurrent implementation of the project
tracking system. See supporting report entry system in Appendix B: B2, Figure 5.
Emotional
The company promotes emphasis on customer satisfaction. This cultural ideal has fed into the
emotional and behavioral state of the contractor support staff to overextend themselves to meet customer
needs at the expense of uncompensated work hours and increased workload. In consequence, overtime on
a consistent basis has sparked budgetary concerns in the contractor funding office. The emphasis on
customer satisfaction is realized and proven effective through an existing tangible incentive in place that
ties bonuses to bi-annual customer satisfaction surveys. The environmental interventions detailed in these
recommendations work to promote overtime by exception, team emphasis and maintaining deliverables.
In order to promote reinforcement of the desired behavior incentives have been shown to prove assistive
(Kirkpatrick & Kirkpatrick, 2006, p. 182) There are intangible incentives that are realized by complying
Analytic Sciences Intervention Recommendations
7
to interventions, such as the realization of less instance of extended work hours and diminished workload
through manager support and the assignment of equitable workload. The incentives that are in place are
goaled to balance customer satisfaction with additional emphasis on working smarter as a team. The
environmental interventions proposed place emphasis on teamwork. Given this perspective, incentives
will be rewarded based on team effort, which aligns to the team rating that assesses the customer
satisfaction bonus. Affirmed by this case description, these professionals require no incentive to work
hard and provide quality work, but instead require incentive to maintain a 40-hour workweek by
communicating workload and working smarter as a team. A nonmonetary reward incentive that is team
based poses an option, such as free quarterly lunch for the whole team. In addition, those teams who
demonstrate the required metrics set by the contracting office for deliverables and overtime will be
allotted a discretionary holiday on a bi-annual basis. The requirements for incentives are noted in Table 4.
Table 4: Emotional Interventions Overtime: Intervention Type: Provision of Incentives
In addition to existing Bonus incentives, contract support teams are assessed for overtime monthly.
Those teams who consistently meet the 40-hour week requirement or keep approved overtime for
the team under 5% and meet customer deadlines as a team are recognized for “smart team
solutions” and smart task management and are awarded with one free paid extended team lunch
from a selected choice of nearby restaurants on a quarterly basis. Time to participate is counted as
“on the company clock”
Deliverables The goals will be successfully met if customer satisfaction is consistent with bonus requirements
that must yield satisfaction at 90% or above. The contractor office has set a standard that 85% of
deliverables should be on time and unchanged from the initial planned date to comply with current
business standards.
Evaluations
These performance interventions will be evaluated based on Kirkpatrick’s four levels of evaluation
(Kirkpatrick & Kirkpatrick, 2006). Each Level will pose evaluation to the following classifications of
interventions as deemed applicable by. Training, Performance Aids, Emotional Aids and Environmental
Aids The roles and stakeholders are identified for all interventions. The specific contents and details for
Levels 1-4 discussed below are found in Appendix D, Table 7.
Analytic Sciences Intervention Recommendations
8
Level 1- Reaction
Contract support teams will be sent a quarterly survey to monitor the effectiveness of interventions
over the course of a year. This survey will be posed to ask a series of questions to gather individual
opinion on the equitable distribution of tasks, satisfaction of management support (Team Lead),
estimations of their average number of work hours per week, estimation of customer satisfaction relative
to project deliverables, and subjective assessment of teamwork quality, satisfaction with incentives and
workload.
Customers will be sent a similar quarterly survey that evaluates their level of satisfaction with
project deliverables, contractor team support, and satisfaction with charged man-hours and quality of
work. The subjective results will be compared for consistency in successful implementation of
interventions.
The contractor funding and budgeting offices will be provided a survey to rate their satisfaction
with the performance aid form and gain any relative comments to the forms benefit of use.
Level 2- Learning
Relative to the recommendations set in this document; learning only applies to team leads and is
optional. Several methods of evaluation serve to assist credibility to training. Upon completion of the
Project Management Training sessions, Team Leads will be assessed via eligible certification through the
passing of certification testing at the expense of the company. An independent certification validates
learning took place relative to basic project management knowledge. Upon the receipt of certification,
basic project management skills will be added to the criteria for annual performance review on an
individual basis. In reflecting upon Kirkpatrick & Kirkpatricks (2004) model, training interventions for
Levels 1, 3 will be evaluated via surveys. Questions are noted in Appendix D, Table 7, and will be
provided to participants to rate their overall benefits of the instruction in managing daily team activities.
While the above evaluations may render credibility to the courses, it is limited to measuring
whether the training influenced the efficiency of daily activities performed by the Team Leads,
knowledge base to perform tasking and manage support staff to desired goals. In order to determine the
Analytic Sciences Intervention Recommendations
9
benefit of cost and time to training we can compare evaluations to those teams leads who participated in
the training to those that opted out, with those that opted out serving as a control group. Utilizing a
control group will assist to determine if any resultant change in behavior is due to the training
intervention or other factors (Kirkpatrick & Kirkpatrick, 2006, p. 43).
Level 3- Behavioral
To evaluate that the behavior that caused the many changes to project scope has diminished; team
leads will demonstrate the use of the Project Change Tracking job aid via a printed a monthly project
tracking report that is shared with the Project Manager during monthly project status meetings. Status will
consist of data resulting from the newly employed project tracking system, which includes project
identifiers, rationales and impacts of changes, as well as revised deliverable status. The first month post
implementation results will be compared to pre-implementation records of changes gained from emails,
meeting notes, and tallied for comparison to assess behavioral change. To evaluate if the behavior that
influenced consistent overtime has changed, the initial monthly status meeting will also include a status of
approved overtime hours associated to project identifiers and inputted via the new project tracking
system. Overtime realized from the initial post implementation results will be compared to pre-
implementation records of overtime. These initial results will capture measured behavioral changes but
other processes, and incentives will be in place to reinforce ongoing support to desired behavior.
Level 4- Result
The two main performance goals consist of adherence to a 40-hour workweek and justification of
extended deliverable dates and the optimal goal of sticking to initial deliverable dates. The interventions
of process and the created job aid associated to these goals will play into the background of realizing the
goals of established metrics relative to customer satisfaction, overtime and deliverable goals. Analysis
will be conducted on an individual team basis in conjunction with existing bi-annual customer satisfaction
surveys promoted by the contracting office. The measures serve as both a pro-active and reactive means
to assure customer and contractual requirements are being satisfied or if further remedial action is
required. The interventions implemented will prove successful if established goals are successfully met.
Analytic Sciences Intervention Recommendations
10
Customer satisfaction goals are successfully met if customer satisfaction is consistent with bonus
requirements that must yield satisfaction at 90% or above. The contractor office has set a standard that
85% of deliverables should be on time and unchanged from the initial planned date in order to comply
with normal operating standards relative to government-funded projects. In addition, the biannual analysis
of overtime will reflect if intervention measures such as the project-tracking tool and in house processes
are successful. The goal is to maintain approved OT at the 5% range for each team in the overall
calculated six-month period.
The success of the incentives in place will be measured statistically within and between teams to
ascertain if any performance deficiencies identified are the result of problems internal to specific teams.
This same measure will also pose validity to the incentives in place and be aligned holistically to the
results of surveys for Incentives gained with Level 1- Level 3 analyses.
In order to assess the impact of the training intervention the results of these analyses will be
compared to Team Leads who participated in Project Management Training and compared to the results
of teams where team leads opted out of training.
Analytic Sciences Intervention Recommendations
11
References
Kirkpatrick, D. L., & Kirkpatrick, J. D. (2006). Evaluation Training Programs 3rd Edition. San
Francisco, CA: Berrett-Koeher Pubushers.
Stolovitch, H. D., & Keeps, E. J. (2004). Training Ain't Performance. Alexandria, VA: ASTD
Press.
Appendix A: Human Performance Analyses Summaries
Table 5: Human Performance Analysis
Description Desired State Actual State Symptoms/Findings Probable root
cause(s) Type
(Gilberts Model) Interventions Type
Deliverables
for work
products are
consistently
extended
Deliverables
should start
and end on
schedule.
There is
consistency of
extended
deliverables
for various
contractor
support teams.
• Customer direction may be misguiding.
• Customer(s) consistently changing project scope and effecting delivery dates.
• Customers are very open minded and creative to evolving projects.
• (Communication issues) No mechanism to track customer directed changes and justify extended product deliverables to upper management
Resources
Information
Actors (Team Lead)
Development of
Electronic tracking
support tool for
product/project
changes and
reasons. This
tracking system is
necessary for
project
management
reports and
conveying printed
paper resources for
tracking product
changes with
customer
approval/signature
and
acknowledgement
of how change of
scope will effect
delivery date.
Performance
Aid (Job Aid)
Analytic Sciences Intervention Recommendations
13
Description Desired State Actual State Symptoms/Findings Probable root
cause(s) Type
(Gilberts Model) Interventions Type
• No stated guidelines for customer and contractor relative to project changes
Resources
Information The creation of
internal guidelines
and processes for
customer and
contractor relative
to project changes.
(Reference and self-
study)_
Process
Change
Contractor
support
teams are
consistently
working over
40 hours.
Contractor
support
teams should
be able to
complete
tasking during
a 40-hour
workweek.
Contractor
support
teams are
working an
average of
50-60 hours
per week.
• Impromptu customer demands/multiple customers requiring project support combined with too many meetings during normal business hours (1-3 per day).
• There is no
equitable
distribution of
Contractor
support staff
workload by
management
Information
Resources
Team Leads
(Participation in
company paid
Project
Management
Certification
Courses)
-Process developed
for allocation of
tasking to balance
workload and
minimize workload
among contractor
support staff (
information for self-
study):
-Policies and
practices are
consistently
implemented, to
ensure that
workloads are seen
as fairly and
equitably
Training
Environmental
(Process
Redesign)
Process
Redesign
Analytic Sciences Intervention Recommendations
14
Description Desired State Actual State Symptoms/Findings Probable root
cause(s) Type
(Gilberts Model) Interventions Type
• Contractor support staff takes on individual work that extends beyond a 40 hr work week with no approval or awareness from management
• Overtime for
salaried
employees
requires no
justification or
approval from
management
(allows for
awareness, task
planning and
Resources
distributed
- Managers provide
clear direction
regarding
organizational
priorities to assist
employees to
focus on the
highest-value tasks
to meet deadlines;
-Support from team
members should be
provided for those
workers coping with
high workload.
-High levels of
teamwork within
and across
organizational units
are emphasized
Process developed
for overtime
approval
Increased
Management
Support
Organizational
Redesign
Analytic Sciences Intervention Recommendations
15
Description Desired State Actual State Symptoms/Findings Probable root
cause(s) Type
(Gilberts Model) Interventions Type
• Customer underestimates time to task for support staff on the average of 30%
The organizational culture supports customer satisfaction and direct support. Performance across teams is rated on the overall satisfaction the customer assigned reflects in surveys and is tied to bonuses.
coordination).
• .Customers have
no awareness of
overtime hours
and costs for
required tasks
(awareness)
• Bonus and
corporate
culture tie into
the emotional
and behavioral
state of
contractor
support staff to
overextend
themselves to
meet customer
needs at the
expense of
uncompensated
work hours and
increased
workload
Information
/Communication
(cost awareness)
Incentives
Customers
supported are
provided a monthly
report of overtime
hours and cost for
additional man
hours as a result of
added tasking to
impromptu
deadline
In addition to existing Bonus incentives, contract support teams are assessed for overtime monthly. Those teams who consistently meet the 40-hour week requirement or keep approved overtime for the team under 5% and meet customer deadlines as a team are recognized for “smart team solutions” and smart task
Provision of
Information
Incentives
Analytic Sciences Intervention Recommendations
16
Description Desired State Actual State Symptoms/Findings Probable root
cause(s) Type
(Gilberts Model) Interventions Type
management and are awarded with one free paid extended team lunch from a selected choice of nearby restaurants. Time to participate is counted as “on the company clock” Intangible incentives also exist in less work hours and equitable workload.
Types based on Gilbert’s Engineering Model (Stolovich & Keeps (2004), pp. 38-39
Table 6: Additional Analysis Summary
Due to the delicate nature of client contractor relationship, analysis was conducted to assess the actual state vs.
the expected state between the customer and contractor support staff relative to overtime and meeting
deliverables. The analysis should shed insights relative to process improvements that improve performance
relative to maintaining deliverables and decrease workload issues that inspire overtime and influence
budgeting.
Performance gap:
Description
Technique/tool for
further investigation
What you hope to learn Results
Analytic Sciences Intervention Recommendations
17
Inability to consistently
meet deliverables
Observation By observing
customer/contractor
meetings, the Project
Manager will gain insights
to interactions taking
place relative to the
change of project scopes
and deliverables.
Ten contractor support
teams were observed for
weekly project planning
and status meetings for a
period of 4 weeks. .
During this observational
period, 6 projects were
logged as ongoing and 4
projects were started.
The observations noted
that 8 out 10 projects
consisted of 2 or more
major changes from their
original scope of
direction.
Customer Survey
Asking customers to
identify any issues with
project scope and
deliverables will assist in
gaining insights to
customer issues with
contractor support teams.
Survey questions:
Are you satisfied with
delivery schedules for
your projects?
All said yes.
Do you have any concerns
relative to project scope
planning or evolution?
Responses included: It
takes time to envision a
project to conclusion.
Sometimes you start a
project with just a basic
plan and then you learn
Analytic Sciences Intervention Recommendations
18
Consistency in needed
overtime to complete
customer tasking
as the project evolves or
more options come to
awareness and it changes
everything you initially
envisioned.
I really never know what I
want until I see it.
Some projects are simple
to envision, others are
complicated and it takes
time to configure a way
to simplify complex
information.
It does not matter to me
how many times I have
my support team start
from scratch. What
matters is if I like what I
get in the end.
Weekly Customer
Survey (4 weeks)
Asking individual
customers how many
weekly work hours they
deem contractor support
staff require to fulfill
assigned tasks will shed
light on discrepancies
between expectations and
the actual states of
Customers were provided
a task list of the team’s
tasking for the prior
weeks. Customers were
required to assess their
opinion of what each task
would take in man-hours.
The results were
compared with actual
Analytic Sciences Intervention Recommendations
19
required work hours. This
information can be used
to determine if task
planning and deliverables
are being scoped to 40-
hour workweeks or weeks
that will require more
time.
man-hours recorded by
the staff for daily tasking.
In most instances, there
was a scaled discrepancy
of 30% between the
customer’s expectations
and the actual time it
took to achieve a task.
Customers are therefore
underestimating time
requirements for tasking.
Staff Observation (4
week duration)
Having the staff complete
a list of daily tasking and
time spent on task, will
assist in gaining insights
to the current state of
workload for the
contractor support staff
and their assigned
customers. This tracking
will also assist to reveal
discrepancies between the
customers’ expectations
of time to task and the
actual time it takes to
complete a task.
These results are
compared to customer
expectations for work
hours noted in the
customer survey.
Analytic Sciences Intervention Recommendations
20
Appendix B: Intervention Materials
B1: Team Lead Training
Project Management Training Requirements
The learning content will consist of lessons encompassing the Project Management Life Cycle and utilization of Resources as noted
by the following curricula requirements:
The learner will understand the basic proponents of the Project Management Lifecycle to include: Project Initiation, Planning the
Project, Project Plan Execution and Closing the Project
Scope and Change Management
The learner will be able to apply Critical Path Management (CPM) techniques to Projects
The learner will engage in principles that apply to creating the scope of a project
The learner will be able to conduct an impact analysis for project changes
Scheduling
The learner will know the basic principles that apply to tracking a schedule and deliverables.
The learner will be able to create a deliverable or milestone focused schedule
The learner will know and apply the basic considerations and elements for Reporting of Present and Future Activities
Deliverables and Milestones
Analytic Sciences Intervention Recommendations
21
The learner will engage in methods to track project status based on identified objectives
The learner will know how to construct project baselines and actual reports
The learner will learn to practice incremental approaches to strategic planning for current operations that convey to user acceptance
Quality and Cost
The learner will utilize practice to track and report variances in cost of projects
The learner will be able to create deliverable review schedules that impact cost, budget & schedules
The learner will learn principles that apply to managing a project budget (cost) with current and expected needs (due to changes)
Building and Managing Teams
The learner will demonstrate practices that align with the PMBOK ® Guide, Fifth Edition, to learn to tools and techniques to
stakeholder management, planning, engagement and the discovery of stakeholder and team expectations.
http://www.pmi.org/PMBOK-Guide-and-Standards.aspx
The learner will demonstrate best practices in the following:
Delegating responsibilities and tasks
Motivating using a combination of intrinsic and extrinsic rewards
Developing the skills of team members
Coaching team members
Analytic Sciences Intervention Recommendations
22
B2: Change Request Performance Aid-Prototype
A user may enter a new project id for tracking or enter an existing project ID to retrieve a queue of changes.
Project Tracking System
Project Identifier: (Enter ID)
Add New Project: (Add New Project ID)
Figure 1: Access or create tracking
An individual can access the list (Figure 2) and select to enter staffing hours, which will generate an open window for the data entry
of staffing hours for the open project ID. This data is for informational purposes only.
An individual may click twice on a row to view or edit existing data on the Change Request Input form (Figure 5).
Project ID Queue
Project ID Number xxxxxx
Project Name xxxxxxxxxxxxxx
Document of Record for project change requests per Project ID
Allocated Staffing Budget
(The planned staffing Budget is
indicated here auto-populated
from the Financial Impact
column on the change request
form)
Current Staffing Costs
Billed to Project
(Over budget values
will appear in
parenthesis and be red
font.)
Enter Staffing Hours
(selecting this button
will generate a pop-up
window to input
staffing hours allocated
to project)
Change
Number
001 Start Date XX/XX/XXXX
Change
Number
002 Start Date XX/XX/XXXX
Change
Number
003 Start Date XX/XX/XXXX
Analytic Sciences Intervention Recommendations
23
Change
Number
004 Start Date XX/XX/XXXX
Figure 2 Tracking List for Projects
Input method of staffing hours for team lead relative to overtime
Project ID XXXX Staffing Report Document of Record for overtime hours charged to a project
(EMPLOYEE ID) Enter employee ID
Employee Name Appears Here xx/xx/xxxx enter date of overtime (OT)
Enter the number of Hours of OT worked for this date
Figure 3 Data entry screen or tracking overtime hours (Team Lead)
Analytic Sciences Intervention Recommendations
24
A printout of overtime hours for an object can be printed utilizing a date range and will appear in the following format to provide
information to the reader (customer) as noted by Figure 4.
Project ID XXXX Staffing Report Document of Record for overtime hours charged to a project
(EMPLOYEE ID) Enter employee ID
Employee Name Appears Here xx/xx/xxxx – xx/xx/xxxx Date range
Total Overtime Hour worked per Employee
001 Mike Smith 08/01/2015-08/06/2015 6
008 Hope Sails 08/01/2015-08/06/2015 1
0067 Don Smart 08/01/2015-08/06/2015 2
Figure 4 Prototype of Overtime Information allocated to each project for the customer (information awareness)
Change Request Form Document of Record for a project change request
Project Name XXXXXX Project ID Number Derived from original project planning document)
Contracted Entity Sponsor Requesting Change The name of the contractor sponsor requesting the change
Created by Person submitting the change on behalf of the sponsor approved customer ( Team Lead or Project Manager)
Change Number (auto populated by system)
001
Date of Request Change Start Date:
Change Request Detail - The Change Request form is the document of record for a change request.
Reason for Change Description of the change requested with rationale
Business Change or Need
Funding Source Plan Change
Scope Change or Need
Schedule Change
Analytic Sciences Intervention Recommendations
25
Error Correction
Regulatory Requirement
Other
Original Project Description (Derived from original project planning document)
Benefits of Proposed Changed The additional benefits the proposed change would have.
Impact Severity 1 - Critical Impact: Threatens the success of the program 2 - High Impact: Significant disruption to program schedule, cost, or quality 3 - Normal Impact: Progress disrupted with manageable extensions to short-term schedule and cost 4 - Low Impact: Exposure is slight
List of Impacted Deliverables List any associated deliverables (projects) internal/external that may be effected by this change request.
Current Allocated Funding (Enter the original proposed costs allocated to the project)
Staffing Costs:
Technical/Physical Resources:
Other:
Financial Impact ($) Estimated Cost of the Change(s) requested
(Include estimated cost of Added Staffing Hours) (specialty staff) that may be required as a result of the change
(cost of additional technical/physical resources)
(other anticipated costs Example: travel)
Revised Funding (Add the current allocated funding to the Financial Impact of the changes for each corresponding row)
Staffing Costs:
Technical/Physical Resources:
Other:
Schedule Impact (days) Time Required for the Change
Original Deliverable Date Date: Revised Deliverable Date Date:
Authorization
Who has the authority to approve this type of change? Business Office
Analytic Sciences Intervention Recommendations
26
Customer/Stakeholder
Other:
Fiscal Reviewed and Approved by If additional funding is sourced separately by a department
Date:
Authorizing Signature (Customer) Date:
Project Manager Signature Date:
Team Lead Signature Date:
Figure 5 – Prototype of input and printout data to record changes for a project.
Analytic Sciences Intervention Recommendations
27
Appendix C: Intervention Timeline – Gantt Chart
Activities Timeline (Months)
1 2 3 4 5 6 7 8 9 10 11 12
1. Project Management Training (Team Leads)
2. Process Redesign - Overtime
3. Increased Management Support - Overtime
4. Electronic Support Project Tracking System (Implementation)
5. Process Change Deliverables
6. Incentives – Contractor Support Staff
Analytic Sciences Intervention Recommendations
28
Appendix D: Evaluations
Table 7: Kirkpatrick’s Evaluation Levels for Interventions
Kirkpatrick
Level
Training
Intervention
(Team Lead)
Performance Aid
Intervention
(Team Lead,
Customer, Project
Manager, Contracting
Office, Funding
Office)
Environmental
Interventions
Support Staff,
Customer
Emotional
Intervention
Support Staff
Project Management Training
Electronic Project Change Tracking Support Tool
Process Redesign
Increased Management Support
Information (Customer)
Incentives
Level 1:
Reaction
Surveys and open-ended questions are used to assess Reactions to all interventions.
Details are provided per intervention category. Scaled ratings will range from (not at
all (1), somewhat (2), Very Much So (3).
Did the
training
prepare you
better for
daily
tasking?
How
confident do
you feel
using the
project
Do you find this
Project Change
Tracking tool
assistive to your
daily activities?
Comments:
Do you feel this
tool eases your
Workload?
Does the tool
provide the
information you
require?
Support Staff-
Overtime, Management
support
Do you feel
comfortable requesting
overtime?
Are you satisfied with
the current overtime
policy?
Do you have any
comments relative to
benefits or
improvements to the
overtime policy? Please
Are you satisfied
with the incentives
in place?
Comments:
Do you feel the
incentives assist in
your focus to meet
the goals aligned to
them?
Comments:
Do you feel your
team works to meet
incentive goals?
Comments:
Analytic Sciences Intervention Recommendations
29
Kirkpatrick
Level
Training
Intervention
(Team Lead)
Performance Aid
Intervention
(Team Lead,
Customer, Project
Manager, Contracting
Office, Funding
Office)
Environmental
Interventions
Support Staff,
Customer
Emotional
Intervention
Support Staff
Project Management Training
Electronic Project Change Tracking Support Tool
Process Redesign
Increased Management Support
Information (Customer)
Incentives
management
skills gained
through
training?
In your
opinion,
could the
training be
improved?
Do you have
any
comments
relative to the
overall
training
course?
Please list
these:
Comments:
Do you have any
comments or
suggestions to
improve the
performance aid
(open-ended
question)
include rationale:
Do you feel you can
achieve your tasking
effectively to customer
satisfaction in a 40-hour
workweek?
Comments:
Do you feel your
manager adequately
supports the needs of
your tasking?
Comments:
Do you feel your
manager fairly
distributes workload
among team members?
Comments:
Do you have any
comments or
Do you have any
comments or
suggestions relative
to improving
incentives? (open-
ended question)
Analytic Sciences Intervention Recommendations
30
Kirkpatrick
Level
Training
Intervention
(Team Lead)
Performance Aid
Intervention
(Team Lead,
Customer, Project
Manager, Contracting
Office, Funding
Office)
Environmental
Interventions
Support Staff,
Customer
Emotional
Intervention
Support Staff
Project Management Training
Electronic Project Change Tracking Support Tool
Process Redesign
Increased Management Support
Information (Customer)
Incentives
suggestions to improve
the process for
overtime? (open-ended
question)
Team Leads and
Customer/Deliverables-
Project Change
Tracking Process
Are you satisfied with
the project change
tracking process?
Do you have any
comments or
suggestions to improve
the process for tracking
project changes(open-
ended question)
Analytic Sciences Intervention Recommendations
31
Kirkpatrick
Level
Training
Intervention
(Team Lead)
Performance Aid
Intervention
(Team Lead,
Customer, Project
Manager, Contracting
Office, Funding
Office)
Environmental
Interventions
Support Staff,
Customer
Emotional
Intervention
Support Staff
Project Management Training
Electronic Project Change Tracking Support Tool
Process Redesign
Increased Management Support
Information (Customer)
Incentives
Level 2:
Learning
Refer to
Evaluation
Section in
document
Level 2-
Training
n/a n/a n/a
Level 3:
Behavior
Are you
actively
incorporating
what you
learned into
daily work
activities?
What specific
processes,
resources or
tools do you
actively
incorporate
into your
tasking and
how
frequently?
Refer to Evaluation
Section in
document
Level 3- Behavior.
Behavior is
supported through
processes that
require this
performance aid to
be used
. A survey may be
provided to the
project manager to
ask questions
relative to his staff
Refer to Evaluation
Section in document
Level 3- Behavior.
Establish a
quarterly average of
results within and
between teams
relative to meeting
the established
incentive
requirements. A
result of 75% or
higher across teams
will determine if the
incentives in place
are motivational to
the desired
behavior. Results
within teams that
demonstrate lower
results <75% may
Analytic Sciences Intervention Recommendations
32
Kirkpatrick
Level
Training
Intervention
(Team Lead)
Performance Aid
Intervention
(Team Lead,
Customer, Project
Manager, Contracting
Office, Funding
Office)
Environmental
Interventions
Support Staff,
Customer
Emotional
Intervention
Support Staff
Project Management Training
Electronic Project Change Tracking Support Tool
Process Redesign
Increased Management Support
Information (Customer)
Incentives
Comments:
of Team Leads:
Does your staff
appear to use this
tool correctly?
Does your staff
provide you the
required documents
realized through
this tool monthly?
Have you had any
customer
complaints relative
to the use of this
tool?
require deeper
analyses through
Project
management
intervention.
Level 4: Results Refer to
Evaluation
Section in
document
Level 2-
Learning &
Refer to Evaluation
Section in
document
Level 4- Results
Refer to Evaluation
Section in document
Level 4- Results
Refer to Evaluation
Section in
document
Level 4- Results
Analytic Sciences Intervention Recommendations
33
Kirkpatrick
Level
Training
Intervention
(Team Lead)
Performance Aid
Intervention
(Team Lead,
Customer, Project
Manager, Contracting
Office, Funding
Office)
Environmental
Interventions
Support Staff,
Customer
Emotional
Intervention
Support Staff
Project Management Training
Electronic Project Change Tracking Support Tool
Process Redesign
Increased Management Support
Information (Customer)
Incentives
Level 4-
Results