Johnson smith

1

The Chronicles of APEX: Experiences from the PM Development Frontier

Presented by

Ron Johnson, Chief of Ames’ Systems Management OfficeClaire Smith, APEX Deputy Program Manager

February 2007

2

• APEX Program Overview

• Top Ten Lessons-Being-Learned So Far

• Where We Go From Here

Topics

3

One year development program designed to demonstrate a measurable increase in the participants’ ability to execute project management and systems engineering

Two integrated pathwaysAPEX PM (10 project manager participants) and APEX SE (10 systems engineering participants)Participants selected by Center Management because of their leadership positions in critical ARC

projects, examples: CEV, LCROSS, SOFIA, Kepler, Air Traffic, NGATS, Columbia (Super Computing)Participants spend approximately 25% of their time on APEX activities (paid by assigned project/home

organization)

Development areas for both pathwaysTechnical Development: Agency-unique APEX Online tool created for PM and SE core competency

technical assessment and automatic creation of Individual Development Plan (IDP)Leadership Development: APPEL funded 4D Leadership model (workshop, assessments, and monthly

coaching)

Ames Project Excellence (APEX) Development Program

4

Profiles

Of the

Participants

Focus of APEX Pilot

Year

5

PM and SE Competencies used as foundation for the Program- Consistent with APPEL model

Project Managers1.1 New Business Development1.2 Bid and Proposal1.3 Requirements Development and Management1.4 Project Planning and Scheduling1.5 Acquisition Strategies1.6 Risk Management2.1 Budget and Full Cost Management3.1 Systems Engineering3.2 Design and Development3.3 Technology Assessment, Development, and Transfer3.4 Contract Management3.5 Stakeholder Management4.1 Project Control4.2 Project Review and Evaluation5.1 System Safety5.2 Mission Assurance6.1 Agency Structure and Mission6.2 NASA and Ames PM Procedures6.3 International Agreements7.1 Team Dynamics and Management7.2 Communication7.3 Problem Solving8.1 Knowledge Management8.2 Knowledge Sharing

Systems Engineers1.1 Trade Studies1.2 Proposal Development2.1 Baseline System Architecture2.2 Alternative Design Concept Development3.1 Requirements Development3.2 Requirements Management4.1 Margin Determination4.2 Technical Resource Allocation5.1 System Design5.2 System Analysis5.3 Uncertainty Analysis6.1 System Verification6.2 System Validation7.1 Risk Identification and Assessment7.2 Risk Mitigation7.3 Risk Tracking8.1 Review Planning8.2 Review Implementation and Support9.1 Baselining9.2 Change Control10.1 Systems Engineering Planning10.2 SEMP Implementation11.1 Team Dynamics and Management11.2 Communication11.3 Problem Solving

6

Sample

Critical Tasks identified for each Competency and Profile

7

Team Learning

On-The-Job Development Assignments

Classroom Training

One-On-One Mentoring

IDP

APEX DevelopmentCouncil

APEX Management Staff

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Key Program Components

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

8

Process for Demonstrating a Measurable Increase in Capability

Assessment(APEX Online)

Assessment(APEX Online)

- Participant- Supervisor- Mentor

EstablishBeginning

Capability Level(for each Critical

Task)

EstablishBeginning

Capability Level(for each Critical

Task)

- “Developing”- “Functional”- “Proficient”

IndividualDevelopment

Plan w/ Milestones

IndividualDevelopment

Plan w/ Milestones

- One on One Mentoring- Team Mentoring- On the Job Training- Training Classes- Knowledge Sharing

Achieving Next Higher Capability

Level - Graduation

Achieving Next Higher Capability

Level - Graduation

- Evaluated by Development Council based on APEX achievements

4 Month Rev.

8 Month Rev.

All focused on improving capability in a minimum of four critical tasks

9

- Participant- Supervisor- Mentor

- “Developing”- “Functional”- “Proficient”

- One on One Mentoring- Team Mentoring- On the Job Training- Training Classes- Knowledge Sharing

- Evaluated by Development Council based on APEX achievements

APEX Capability Levels

• Developing: Limited/No knowledge. Limited/No experience in executing task.

• Functional: Can define core concepts and practices, and has executed task in routine situations under guidance of expert practitioner.

• Proficient: Breadth of knowledge, skills and experience applied to executing task in complex environments.

APEX Capability Levels

• Developing: Limited/No knowledge. Limited/No experience in executing task.

• Functional: Can define core concepts and practices, and has executed task in routine situations under guidance of expert practitioner.

• Proficient: Breadth of knowledge, skills and experience applied to executing task in complex environments.

DevelopingFunctional

FunctionalProficient

Process for Demonstrating a Measurable Increase in Capability

Assessment(APEX Online)

Assessment(APEX Online)

EstablishBeginning

Capability Level(for each Critical

Task)

EstablishBeginning

Capability Level(for each Critical

Task)

IndividualDevelopment

Plan w/ Milestones

IndividualDevelopment

Plan w/ Milestones

Achieving Next Higher Capability

Level - Graduation

Achieving Next Higher Capability

Level - Graduation4 Month Rev.

8 Month Rev.

10

Example IDP from a Systems Engineer

11

APEX Graduation DocumentationParticipant’s Name:

APEX Pathway (PM or SE):

APEX Profile (Selected before beginning of APEX Online Assessment):

Immediate Supervisor:

One-On-One Mentor:

1. How have your APEX program activities (Mentoring, OJT, Courses, etc) improved your technical and leadership capabilities?

2. Please complete the following chart. Format allows you to be as detailed as necessary:

Participant Signature__________________________ Date:__/__/__

One-On-One Mentor:

Do you confirm that the participant has satisfied the APEX graduation criteria, I.e. accomplished the IDP Objectives listed above? __ Yes __No. Explain:

What are your recommendations for the participant's future learning and development?

Mentor Signature: __________________________ Date: __/__/__

APEX Focused Tasks

IDP Objective* *Enter Proficient

Ready if Applicable

Please provide a concise and detailed (as necessary) description of how you successfully satisfied or exceeded both the knowledge and performance criteria associated with this objective

1

2

3

4

12

Mar Apr FebJanMay Jun Nov DecOctSepAugJulAssessment

Individual Development Plan Living document

Team Mentoring Monthly

One-on-One Mentoring

OJT and Mentor-AssignedTasks

Milestone Reviews Final4-month 8-month

Leadership Development and Courses/Workshops

APEX 12 Month Timeline

13

Top Ten Lessons-Being-Learnedso far

14

1. Program Leadership Needs To Combine Technical and Learning Expertise

Pilot Year : APEX co-managed/led by Systems Management Office and Workforce Development BranchPilot Year : APEX co-managed/led by Systems Management Office and Workforce Development Branch

Key Strength: Combined best practices of both disciplines.Key Strength: Combined best practices of both disciplines.

Key Challenges: Differences in language, perspectives and approaches to technical development, and complications from splitting budgets (vs. embedded in one organization).

Key Challenges: Differences in language, perspectives and approaches to technical development, and complications from splitting budgets (vs. embedded in one organization).

Recommendation: Anticipate difficulties. Focus on collaboration vs. and commitment to shared vision.Recommendation: Anticipate difficulties. Focus on collaboration vs. and commitment to shared vision.

15

2. Learning and Development is a VoluntaryActivity, or

You Can Lead Them To Water But You Can’t Make Them Drink

Pilot Year : Candidates Nominated & Participants Selected by Center Senior LeadershipPilot Year : Candidates Nominated & Participants Selected by Center Senior Leadership

Key Strength: Participants were project practitioners working on mission critical programs and projects. Senior leadership nomination provided status and supported importance of program.

Key Strength: Participants were project practitioners working on mission critical programs and projects. Senior leadership nomination provided status and supported importance of program.

Key Challenge: Participants felt “conscripted” into program. Impacted attachment and commitment.Key Challenge: Participants felt “conscripted” into program. Impacted attachment and commitment.

Recommendation: Nomination Process should include Candidate Application indicating interest, commitment and self-defined development needs and objectives.

Recommendation: Nomination Process should include Candidate Application indicating interest, commitment and self-defined development needs and objectives.

16

3. Integrate Technical and Leadership Development In One Program

Pilot Year : Participants required to demonstrate increase in capability in both technical and leadership* skills.

* Utilizing 4D Leadership Assessments, Workshop and Monthly Coaching.

Pilot Year : Participants required to demonstrate increase in capability in both technical and leadership* skills.

* Utilizing 4D Leadership Assessments, Workshop and Monthly Coaching.

Key Strength: Leadership component gave participants opportunity to address all aspects of a successful project, e.g. people skills.Key Strength: Leadership component gave participants opportunity to address all aspects of a successful project, e.g. people skills.

Key Challenge: leadership development activities were “one more thing” added to participant’s plate. Note: Resistance decreased as value of coaching was proven.

Key Challenge: leadership development activities were “one more thing” added to participant’s plate. Note: Resistance decreased as value of coaching was proven.

Recommendation: Integrate technical and leadership components into any PM and/or SE development program.Recommendation: Integrate technical and leadership components into any PM and/or SE development program.

17

4. Focus Technical Development On Increasing Participants’ Ability to Execute PM and/or SE.

Pilot Year : APEX Competency Matrices (PM and SE) Defined Specific “Critical Tasks”* Associated with Each Competency.Pilot Year : APEX Competency Matrices (PM and SE) Defined Specific “Critical Tasks”* Associated with Each Competency.

Key Strength: Defined critical tasks offered specific guidance on what a PM or SE is supposed to do.Key Strength: Defined critical tasks offered specific guidance on what a PM or SE is supposed to do.

Key Challenges: Can only focus on 1-2 tasks* vs. all tasks, i.e. a few trees, not the forest. * Necessary But Not SufficientKey Challenges: Can only focus on 1-2 tasks* vs. all tasks, i.e. a few trees, not the forest. * Necessary But Not Sufficient

Recommendation: Each center should prioritize competencies based on mission, and define critical tasks for each, i.e. actionable items.Recommendation: Each center should prioritize competencies based on mission, and define critical tasks for each, i.e. actionable items.

18

5. Pave A Clear Development PathwayA Map From Here To There

Pilot Year : APEX Assessment Process Defined Beginning Capability Levels for each Critical Task. APEX Individual Development Plan (IDP) Defined Development Objective.

Pilot Year : APEX Assessment Process Defined Beginning Capability Levels for each Critical Task. APEX Individual Development Plan (IDP) Defined Development Objective.

Key Strength: Provided a model for observing and/or measuring actual learning product,i.e. increased capability.Key Strength: Provided a model for observing and/or measuring actual learning product,i.e. increased capability.

Key Challenges: Pilot year definitions of capability levels too vague, and, in some cases, unrealistic. Key Challenges: Pilot year definitions of capability levels too vague, and, in some cases, unrealistic.

Recommendation: Be willing to experiment with metric to measure development. Define stepping stones (improvements) that are attainable and traced to Center and Agency priorities.

Recommendation: Be willing to experiment with metric to measure development. Define stepping stones (improvements) that are attainable and traced to Center and Agency priorities.

19

6. Participants’ Program/Project Assignments The MostCritical Factor Promoting Actual Learning and Development

Pilot Year : Participants’ IDPs focused on “OJT” activities tied to existing project work. Where necessary, OJT assignment on external project.

Pilot Year : Participants’ IDPs focused on “OJT” activities tied to existing project work. Where necessary, OJT assignment on external project.

Key Strength: APEX activities (IDP, mentoring, etc) tailored and applied to participants’ current project assignments and responsibilities increasing chances of project success.

Key Strength: APEX activities (IDP, mentoring, etc) tailored and applied to participants’ current project assignments and responsibilities increasing chances of project success.

Key Challenge: Difficult (almost impossible) to take participants outside of project to develop an expertise.Key Challenge: Difficult (almost impossible) to take participants outside of project to develop an expertise.

Recommendation: Tailor program, as much as possible, to participants current project assignments.Recommendation: Tailor program, as much as possible, to participants current project assignments.

20

7. Mentoring is Second Most Critical Factor Influencing Actual Learning and Development.

Pilot Year : Participants matched with one-On-One Mentor and Team-based mentoring (PMs and SEs). Mentors and participants trained in mentoring partnerships. Mentoring Agreements initiated relationship, set direction.

Pilot Year : Participants matched with one-On-One Mentor and Team-based mentoring (PMs and SEs). Mentors and participants trained in mentoring partnerships. Mentoring Agreements initiated relationship, set direction.

Key Strength: Individual mentoring focused on participant’s immediate issues and executing IDP. Team mentoring helped build knowledge sharing network i.e. “community of practice”

Key Strength: Individual mentoring focused on participant’s immediate issues and executing IDP. Team mentoring helped build knowledge sharing network i.e. “community of practice”

Key Challenges: Making appropriate matches (changed two mentors). Mentor and protégé’s work commitments interfered with partnership.Key Challenges: Making appropriate matches (changed two mentors). Mentor and protégé’s work commitments interfered with partnership.

Recommendation: Define criteria for mentors. Provide mentor training. Involve mentors in reviews.Recommendation: Define criteria for mentors. Provide mentor training. Involve mentors in reviews.

21

8. A Healthy Program Has Regular Check-Ups

Pilot Year: Program activities included IDP reviews at 4 and 8 months, regular “House Calls” on participants and mentors and periodic brown-bags to discuss program issues.

Pilot Year: Program activities included IDP reviews at 4 and 8 months, regular “House Calls” on participants and mentors and periodic brown-bags to discuss program issues.

Key Strength: Activities provided opportunity to any required course corrections/improvements to program and/or participants’ IDPs.Activities increased sense of ownership of and commitment to program among participants and mentors.

Key Strength: Activities provided opportunity to any required course corrections/improvements to program and/or participants’ IDPs.Activities increased sense of ownership of and commitment to program among participants and mentors.

Key Challenge: Multiple, and sometimes conflicting needs and approaches to defining success, how to review progress, etc.Key Challenge: Multiple, and sometimes conflicting needs and approaches to defining success, how to review progress, etc.

Recommendation: Design and schedule reviews throughout program. Clarify expectations regarding focus and scope of reviews. Involve participants and mentors in program decision-making, including review design.

Recommendation: Design and schedule reviews throughout program. Clarify expectations regarding focus and scope of reviews. Involve participants and mentors in program decision-making, including review design.

22

9. Developing A Community Of Project Practitioners* is Easier Said Than Done * Vehicle for generating and sharing practical knowledge, lessons-learned and for fostering innovation.

Pilot Year : Activities designed to develop Community Of Practice, e.g. brown bags, 8th. Month review and team-based mentoring.Pilot Year : Activities designed to develop Community Of Practice, e.g. brown bags, 8th. Month review and team-based mentoring.

Key Strength: Participants and mentors have voice in program design and implementation = increased sense of ownership and commitment. Spontaneous eruptions of informal get-togethers.

Key Strength: Participants and mentors have voice in program design and implementation = increased sense of ownership and commitment. Spontaneous eruptions of informal get-togethers.

Key Challenge: Developing consensus around definition of COP, how to develop it and apply it to Agency and Center mission. Value of COP cannot be demonstrated, can only be discovered.

Key Challenge: Developing consensus around definition of COP, how to develop it and apply it to Agency and Center mission. Value of COP cannot be demonstrated, can only be discovered.

Recommendation: Define clear objectives for COP and trace objectives to mission. Try a lot of different things! Recommendation: Define clear objectives for COP and trace objectives to mission. Try a lot of different things!

23

10. First Build A Prototype, Then A Program

Pilot Year : APEX Pilot year defined as “experiment”, a prototype to test and discover best methods for implementing learning and development activities resulting in measurable improvements.

Pilot Year : APEX Pilot year defined as “experiment”, a prototype to test and discover best methods for implementing learning and development activities resulting in measurable improvements.

Key Strength: Actual experience and practice leads to improved, relevant and effective design and execution. Everyone is involved in the process of improvement, especially users.

Key Strength: Actual experience and practice leads to improved, relevant and effective design and execution. Everyone is involved in the process of improvement, especially users.

Key Challenges: Last 9 things mentioned - plus 100 others!Key Challenges: Last 9 things mentioned - plus 100 others!

Recommendation: Define Pilot year as prototype, test and improve design in real-time as you proceed (don’t wait until pilot year is over). Use participants and mentors experience, not theories, as guide for ongoing improvements.

Recommendation: Define Pilot year as prototype, test and improve design in real-time as you proceed (don’t wait until pilot year is over). Use participants and mentors experience, not theories, as guide for ongoing improvements.

24

Where We Go From HereGraduate our first class

Continue to support APEXers as the core of our growing community of practice– Continued access to assessment tool and mentors, assist in selection of

new APEXers and become their mentors, host knowledge sharing sessions

Capture final lessons from participants, mentors, Development Council, management staff, other stakeholders

Prioritize and make improvements to selected program elements– Include participants and mentors in design improvements

Plan for the next APEX class

Continue to share our experiences and make available our products and tools to APPEL and other Centers