Software Productivity Framework

Zinnov

Productivity Assessment

January, 2010

Framework

Zinnov 2

In this module we will define and analyze the various metrics that impact productivity and identify key existing challenges

Why do we need to track productivity?

Understand the throughput from the engineering resources, based on

To Identify the key productivity metrics to measure across organization

Setup and monitor the metrics Measure key productivity gaps across teams

Life cycle of the product Geographic location

2

Zinnov 3

We will leverage our existing framework and customize it to suit Client needs

3

Developing The Framework Data Collection Analysis Report &

Optimization

• Review Zinnov’s productivity framework Identify gaps in existing

metrics Finalize the metrics for

monitoring and measurements

• Identify the mechanism to collect the data for metrics

• Review existing artifacts (product and project management documents) to collect the data

• Interview key stakeholders to collect missing information and to validate the data collected through artifacts

• Review the data to understand the key gaps between the teams

• Capture qualitative and quantitative insights from the analysis

• Identify key risks impacting productivity across the organization

• Propose risk mitigation strategies

• Deliver a comprehensive report

Zinnov 4

Zinnov will define the productivity drivers and metrics using experiences from other engagements and through discussions with Client (1/2)

4

Productivity Drivers`

Organization Structure• Project ownership in India center• Technical ownership • Team alignments• Customer interaction• Level of autonomy

Infrastructure• Internet latency• Commute time• Phone/internet connectivity from

home• IT support• Lab availability

Communication• Travel between the centers• Number/Type of issue escalations• Status reports• Meeting effectiveness• Project transparency Human Resources

• Brand recognition• Team ramp up time• Attrition rate• Capability development• Resource allocation • Interview process efficiency• Employee motivation

Knowledge Transfer and Management

• Time spent on knowledge transfer• Mode of knowledge transfer• Average number of reviews• Effectiveness of KMSDevelopment Process

• Coding standard• Documentation• CMS process• Release processes• Risk planning• Requirement mgmt• Change management• Project initiation

Zinnov 5

Zinnov will define the productivity drivers and metrics using experiences from other engagements and through discussions with Client (1/2)

5

DEV

ELPO

MEN

T

A

QU

ALIT

Y AS

SURA

NCE

B

SUST

ENEN

CE

C

• % of milestone slippage• Estimation accuracy• Effort variance• Defect density• Field error rate• Defect removal efficiency

• Cost of quality• % of automation• Test case defect

detection efficiency• Bug type distribution• Test cycle distribution• Field error rate

• Defect removal efficiency• Maintenance rework• Response time• # open P1, P2, P3

Definition of Productivity Metrics for the Engineering Group

Key metrics required by Client to measure productivity will be determined and appropriate metrics will be identified Metrics could be based on factors such as center maturity, people maturity, product maturity or PDLC process

Zinnov

Define baselinesMap complexity vs. impact

Analyze impact on productivity

6

In addition will baseline and finalize the metrics and drivers

6

Impact

Com

plex

ity in

Col

lecti

ng D

ata

GL

R

C F

Q

H

P

EI

D

B

K

M

A

Productivity Drivers

O

M

Impact

Com

plex

ity in

Col

lecti

ng D

ata

Productivity Metrics

R

C

Q

H

DK

O

M

A

Analyze data collection complexity

Review with stakeholders

1 2 3 4 5

Metrics and Drivers baseline

Zinnov 7

An excel modeler will be developed that will assist in the collection, monitoring and tracking of the drivers and metrics

7

• Metrics and driver information will be inputted into the modeler

• The modeler will have the capability – To compare metrics between

projects– To plot historic trends of

metrics– To plot relationship between

the metrics and drivers• The modeler can also be used as

a dashboard to measure the health of the projects

Zinnov 8

Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8100

200

300

400

Planned hours Actual hours

23%

38%16%

19%

4% Design

Personal & Team reviews

Implementation

Unit test

System test

Other

0

10

20

30

40

50

60

P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 Fiery PPA0

20

40

60

80

100

120

80

100

6070

3528

IdentifiedFixedRe-Opened

Planned Vs. Actual Hrs Percentage Engineering Effort

Planned Vs. Actual features Details of Bugs re-opened

The productivity metrics will be represented as a dashboard to executive management team

Sample Output

8

Zinnov 9

Case Study 2: We assisted a industry leading product company in defining the productivity and quality measures and benchmarking them in their global centers

Challenge: Productivity measurements were frowned by the client stakeholders due to the complexities involved. Executives were frustrated with the level of details and assumptions provided

9

Key Challenges- Process and metrics should to be simple yet powerful- Ability to easily compare productivity across Business Units

and Geographies- Strong framework that would minimize assumptions

Created Objectives

Defined Frameworks

Measured Efforts

Output Metrics

Compared Productivity

Zinnov and the client worked to propose an objective mechanism to measure the productivity of Development and Quality teams

Framework was defined to measure the complexity of the product based on the architecture, feature set, functionality size, internal/external interfaces, data structure etc.

A simple mechanism was devised to calculate the efforts using the experience levels, size of the team, man hours for each activity etc.

For the quality engineering teams also, similar complexity metrics were defined using number of bugs captured, number of bugs not captures, percentage of automation etc.

Using the weighted averages of the complexity and the engineering efforts, we were able to get the directional productivity comparison between teams working across the organizations

Zinnov Solution

Zinnov 10

Case Study 2: We assisted a industry leading product company in defining the productivity and quality measures and benchmarking them in their global centers

Output: The questionnaire based framework along with the dashboard view provided key insights to the executives on the productivity and quality differences between the teams

10

KEY BENEFITS

Ability to measure the directional

measure productivity

without getting into complex calculations

Simple efforts from the clients to collect, measure and monitor the

metrics on an ongoing basis

Ability to measure metrics across business units,

product/feature based teams and

geo diversified teams

Normalizing the assumption to

minimize the error in measurement

Deliverables- Framework with the guidelines for the input- Comprehensive questionnaire with details on scoring mechanism- Excel based tool to capture the complexity of the team based on the User

input- Dash board view to objectively compare the differences between teams

Zinnov 11

Case Study 3: We engaged with a mid-sized product company to analyze the productivity of their product development teams

Output: Apples to Apples comparison between global teams were not feasible due to the maturity of the organizations. Only the engineering metrics were collected which resulted in dissatisfaction across teams

11

0 to 2 2 to 4 4 to 6 6+0

5

10

15

20

25

30

35

10

5

7

4

6 3

14

5

4

14

1 2

Fresher Product Engineering Vendor IT Services

Product Development Companies

• Average experience of the development team is about 18 months.

• Only one employee is from a product development background, with 3 – 4 years of experience which means most of the early hires were from service companies.

• There are only 6 engineers in the APPS group with an experience ranging from 3 – 4 years; they are the earliest employees

• The majority of the people working for the past 2 – 3 years are from service companies.

Num

ber o

f Soft

war

e D

evel

oper

s

Years of Experience

Companies Vs. Client Experience – Development team

31

18

11

6

Zinnov 12

Case Study 3: We also provided recommendations to enhance productivity

Output: On the basis of our analysis, the Client looked to resolve some constraints that impacted the productivity of the teams

12

• There seems to be a high level of threshold for non-performance in the organization. There were not many who had been let go due to non-performance.

• There has been no instances of successful initiation and implementation of change initiatives.

• Even the product managers of different products were not clear on the roadmap for those products. There were at least two scenarios where the engineering team spent months building features/add-ons that the product managers were not aware of or had consented to.

• There are number of projects still in a split team model. The projects carry an inherent overhead in terms of communication and management time from the US. It is also extremely difficult for the US managers to track the tasks executed by the local teams.

Organization cultural issues

Lack of well-defined product planning process

Project model with inbuilt overheads

• Most of the engineers have never met a customer or even seen a video of how the customer uses the products that are built from the India center. Only few engineers have been exposed to customers at trade shows.

Lack of domain exposure to India center

Zinnov 13

About ZinnovZinnov helps organizations globalize their business and improve people strategy. Our consulting solutions are based on rigorous research techniques, data analytics and communities

Zinnov Research: http://www.zinnov.com/white_papers.php

Zinnov Blog: http://zinnov.com/blog/

Zinnov Services: http://zinnov.com/services.html

Zinnov Events: http://www.zinnov.com/zn_events.php

Zinnov LinkedIn: http://www.linkedin.com/companies/30724

Zinnov Facebook: http://www.facebook.com/pages/Zinnov/111718952202627

@zinnov

For additional details on the topic, contact [email protected]

Zinnov

Thank You

69 "Prathiba Complex", 4th 'A' Cross, Koramangala Ind. Layout5th Block, KoramangalaBangalore – 560095Phone: +91-80-41127925/6

575 N. Pastoria AveSuite JSunnyvaleCA – 94085Phone: +1-408-716-8432

21, Waterway Ave, Suite 300The WoodlandsTX – 77380 Phone: +1-281-362-2773

[email protected]

www.zinnov.com

@zinnov