How to Build a Learning Organization

Janelle Kleinopenmastery.org @janellekz

Tour Speaker

How to Build a Learning Organization

(in 2016)

How to Measure the PAIN in Software Development

Janelle Klein

http://@openmastery

Why isn’t EVERYONE Talking About This Book?

Then I read this book!

“The Art of the Scientist” Lean Startup Workshop, Ash Maurya

Ash: “The goal isn’t learning, the goal is traction.”


Janelle Klein

==

2.0

Then I wrote my book…

Visibility

+

Lean Manufacturing

==

2.0

Then I wrote my book…

+

Traction

News Flash!For the first time in our industry,

we have all the tools necessary

to pull this off.

, Developer & Consultant for 15+ yearsSpecialized in Statistical Process Control (SPC) and Supply Chain Optimization from Lean Manufacturing (I <3 cool data)

Continuous Delivery infrastructure, software design & test automation strategies

Janelle Klein

Who Am I?


Janelle Klein

leanpub.com/ideaflow

Data-Driven Software Mastery

Tour Speaker

I’m a hobbyist Cognitive Scientist

CTO @

Founder @Industry Peer Mentorship Network

My Job

Why Should You Care?

source: Domics - “What’s a Shrug?”

Every few years we rewrite our software…

Start%Over%

Unmaintainable%So0ware%

Across the Industry

We Start with the Best of Intentions

High Quality Code

Low Technical Debt

Easy to Maintain

Good Code Coverage

Then This Happens!

Constant Urgency

Compromise Safety for Speed

Time%Pressure%

Compromise%Safety%for%

Speed%

Increase%Number%&%Severity%of%Hazards%

%

More%Pain%and%Higher%Task%Effort%

Constant'Urgency'

Cycle of Chaos High-Risk Decision Habits

Time%Pressure%


Speed%


%


Constant'Urgency'


Time%Pressure%


Speed%


%


Constant'Urgency'


Time%Pressure%


Speed%


%


Constant'Urgency'


Time%Pressure%


Speed%


%


Constant'Urgency'


PAIN

The Classic Story of Project Failure

Problems get deferred

Builds start breaking

Releases get chaotic

Productivity slows to a crawl

Developers begging for time

It’s never enough

Project Meltdown

Our software problems

Create culture problems

The Cultural Divide

Contempt

Hopelessness

Self-Deception

Fear Imagining the world we want to live in, instead of seeking truth

Feeling hurt by those who are preventing us from reducing our pain.

Undesirable Reality

Managers: Lack of Visibility

Developers: Lack of Control

Fake Happiness

Culture #1: Everything is Awesome!

Culture #2: ”Bitch-Bonding”

Validation of Hopelessness

This is POINT A!

We fail to communicate, make decisions in ignorance, and don’t adapt when our strategy is obviously failing.

Start%Over%

Unmaintainable%So0ware%

This is WHY It Matters:

From the Outside:

This is the pattern of a broken feedback loop.

Product(Requirements( So1ware(Task(

Possible(Decision(Paths(

So1ware(

Software is a Reflection of our Decisions


High%Risk%Decision%Habits%

So1ware((and(all(our(problems)(

(

RESET



High%Risk%Decision%Habits%

So1ware((and(all(our(problems)(

(

Technical Debt is just a Symptom of the Problem


Learn & Adapt

Just because we repair the car… doesn’t mean we know how to drive.

Learn & Adapt

How do we build a learning organization?

We want to drive our projects like THIS:

Make the PAIN Visible (see waste)

Step ONE:

Great Team Disciplined with Best Practices Constantly Working on Improvements+

Project FAILURE

About 8 years ago…

Technical Debt Mistakes

I thought the main obstacle was Technical Debt

?Most of our mistakes were in the

most well-written parts of the code.

Mistakes

We made significantly more mistakes in code that we didn’t write ourselves.

Lower Familiarity

More Mistakes=

There had to be more to the story...

Complex(So*ware(

PAIN

This is what I knew...

What made development feel painful?

Unexpected Behavior

Problem Resolved

Measure Painful Interaction with the Code (Friction)

Troubleshooting

Progress

5 hours and 18 minutes of troubleshooting...

PAINFUL

The amount of PAIN was caused by…

Likeliness(of((Unexpected(Behavior(

Cost(to(Troubleshoot(and(Repair(

High(Frequency(Low(Impact(

Low(Frequency(Low(Impact(

Low(Frequency(High(Impact(

PAIN(

What Causes Unexpected Behavior (likeliness)?

What Makes Troubleshooting Time-Consuming (impact)?

Semantic Mistakes

Stale Memory Mistakes

Association Mistakes

Bad Input Assumption

Tedious Change Mistakes

Copy-Edit Mistakes

Transposition Mistakes

Failed Refactor Mistakes

False Alarm

Non-Deterministic Behavior

Ambiguous Clues

Lots of Code Changes

Noisy Output

Cryptic Output

Long Execution Time

Environment Cleanup

Test Data Creation

Using Debugger

Most of the pain was caused by human factors.

What causes PAIN?




Ambiguous Clues


Noisy Output

Cryptic Output

Long Execution Time

Environment Cleanup

Test Data Creation

Using Debugger

What causes PAIN?


Semantic Mistakes





Copy-Edit Mistakes



False Alarm




Ambiguous Clues


Noisy Output

Cryptic Output

Long Execution Time

Environment Cleanup

Test Data Creation

Using Debugger

What causes PAIN?

Semantic Mistakes





Copy-Edit Mistakes



False Alarm





Ambiguous Clues


Noisy Output

Cryptic Output

Long Execution Time

Environment Cleanup

Test Data Creation

Using Debugger

What causes PAIN?

PAIN is a consequence of how we interact with the code.

Semantic Mistakes





Copy-Edit Mistakes



False Alarm

PAIN occurs during the process of understanding and extending the software

Complex(So*ware(

PAIN

Not the Code.

Optimize “Idea Flow”

My team spent tons of time working on improvements that didn’t make much difference.

We had tons of automation, but the automation didn’t catch our bugs.

My team spent tons of time working on improvements that didn’t make much difference.

We had well-modularized code,

but it was still extremely time-consuming to troubleshoot defects.

The hard part isn’t solving the problems it’s identifying the right problems to solve.

“What are the specific problems that are causing the team’s pain?”

measures the time spent on:

Idea Flow

x

Troubleshooting

x

Learning

x

Rework

Quality Risk Familiarity Risk Assumption Risk

Quality Risk (Troubleshooting)

Likelihood)of))Unexpected)Behavior)

Cost)to)Troubleshoot)and)Repair)

High)Frequency)Low)Impact)

Low)Frequency)Low)Impact)

Low)Frequency)High)Impact)

PAIN)

Familiarity Risk (Learning)

Likelihood)of))working)with)Unfamiliar)

Code)

Cost)to)Learn)

High)Frequency)Easy)to)Learn)

Low)Frequency)Easy)to)Learn)

Low)Frequency)Hard)to)Learn)

PAIN)

Assumption Risk (Rework)

Likelihood)of))making)a))

Bad)Assump4on)

Cost)to)Correct)Decisions)

High)Uncertainty)Low)Delay)

Low)Uncertainty)Low)Delay)

Low)Uncertainty)High)Delay)

PAIN)

Idea Flow Mapping Tools

(Open Source, Supported GA ~June 2016)github.com/ideaflow/tools

http://ideaflow.org/tools

Troubleshooting

Progress

Learning

7:070:00

0:00 19:52

12 year old project after all original developers left.

Case Study: Huge Mess with Great Team

70-90% of dev capacity on “friction”

The Team’s Improvement Focus: Increasing unit test coverage by 5%

Case Study: Huge Mess with Great Team

1. Test Data Generation

2. Merging Problems

3. Repairing False Alarms

1000 hours/month

The Biggest Problem: ~700 hours/month generating test data

18 months after a Micro-Services/Continuous Delivery rewrite.

Troubleshooting

Progress

Learning40-60% of dev capacity on “friction”

0:00 28:15

12:230:00

Case Study: From Monolith to Microservices

18 months after a Micro-Services/Continuous Delivery rewrite.

Troubleshooting

Progress

Learning40-60% of dev capacity on “friction”

0:00 28:15

12:230:00

Case Study: From Monolith to Microservices

The Cost of Chaos

0%

100%

Release 1 Release 2 Release 3

Troubleshooting

Progress

Learning

Percentage Capacity spent on Troubleshooting (red) and Learning (blue)

(extrapolated from samples)

0%

100%



Figure out what to do Learning is front-loaded

Troubleshooting

Progress

Learning

The Cost of Chaos

0%

100%



Rush Before the Deadline Validation is Deferred

Troubleshooting

Progress

Learning

The Cost of Chaos

0%

100%



Pain Builds Baseline friction keeps rising

Troubleshooting

Progress

Learning

The Cost of Chaos

0%

100%



Chaos Reigns Unpredictable work stops

fitting in the timebox

Troubleshooting

Progress

Learning

The Cost of Chaos

Step TWO:

Focus on the Biggest Pain (eliminate waste)

“What caused the pain in this case?”

Categorize the Problems with #HashTags

#ReportingEngine

#Hibernate

#MergeHell

1. Test Data Generation

2. Merging Problems

3. Repairing False Alarms

1000 hours/month

Add up the Pain by Category

Friction % by Area of Code

What’s the biggest problem to solve?

Friction % versus Upcoming Demand



Friction % Grouped by Familiar vs Unfamiliar

Added Complexity: Human Relationships

14:230:00

George’s Painful Experience

Added Complexity: Human Relationships

What made troubleshooting take so long?

This is scary as hell.

Step up and lead, and those you lead will

reflect your feelings toward them.

The Reflection Principle

Ideas from:

Culture is an Emergent Property of a Complex Adaptive System

Culture is an Emergent Property of a Complex Adaptive System

There is only “I” who chooses.

The only thing we actually control:

Personal Mindset

Lead with a C.L.E.A.R. MindsetA mindset is a principle of continuous choice.

Courage.

Leadership.

Empathy.Authenticity.

Respect.

14:230:00


When it’s scary,

You do it anyway.

Lead with a C.L.E.A.R. Mindset

14:230:00


When you see pain,

You teach.


14:230:00


When you have an idea,

You share it.


14:230:00


When you hear an idea,

You try to understand it.


14:230:00


When you see a problem with no owner,

You take responsibility.


14:230:00


If you see a struggling leader,

You help them to be a better leader.


14:230:00


“I am human.” “I am human.”


14:230:00


“I am valuable.” “I am valuable.”


14:230:00


“You are human.” “You are human.”


14:230:00


“You are valuable.” “You are valuable.”

(Respect)


Lead with a C.L.E.A.R. MindsetA mindset is a principle of continuous choice.

Courage.

Leadership.

Empathy.Authenticity.

Respect.

These Energies Repel Each Other

Contempt

Hopelessness

Self-Deception

Fear

Contempt

Hopelessness

Self-Deception

FearC.L.E.A.R.

C.L.E.A.R.

Seeking Truth

Repair the Broken Feedback Loop

Step THREE:

(systems thinking)

Learn & Adapt

The Challenge: Decision-Making is Distributed

We want to drive our projects like THIS:

What if our organization was a robot?

Fire x1

Dev Team

Management

Nothing’s happening…

Pain Sensor


Fire x1

Dev Team

Management

Fire x10

Pain Sensor

Management

Dev Team

Fire x10


If communication breaks down, we can’t adapt.

Pain Sensor

Learn & Adapt

Role

Decision(Type(

Required(Knowledge(

Visibility and Decisions Coupled

Visibility and Decisions De-coupled

Role A

Required(Knowledge(

Role B

Decision(Type(

Broken Feedback Loop is Baked into the Design

Manager

Alloca&on(Decisions(

Knowledge(of(Risks(

Risk(Mgmt(Decisions(

Developer

Communication Breakdown

Broken Feedback Loop (Manager Role)

Developer Product Owner

Actual'Risks'

interacts''with'

Actual'Customers'

interacts''with'

Knowledge'of'Customers'

Trade9off'Decisions'

depend'on'depend'on'Knowledge'of'Risks'

Communication Breakdown

Broken Feedback Loop (Product Owner Role)

Risk Translator

Engineering(Execution)

Management(Coordination)

Risk is the bridge language.

We have to Repair the Broken Feedback Loop

Manager


Knowledge(of(Risks(


Developer

RiskTranslator

Risk(Summary(

Refactor Step 1: Risk Translator

Manager


Knowledge(of(Risks(


Developer

RiskTranslator

Risk(Summary(

Refactor Step 1: Risk Translator

The Risk Translator provides monthly

“Project Visibility Updates”

In the language of:

Quality Risk

Familiarity Risk

Assumption Risk

So#ware(Task(

Mi.gate(the(Risks(

Product(Development(

Product(Work(Queue(

Risk(Mgmt(Work(Queue(

Product(Owner(

Product(Decisions(

Knowledge(of(Customers(

Dev$Team$Capacity$

Alloca.on(Decisions(

Manager2Translator$Partnership$


Knowledge(of(Risks(

Refactor Step 2: Partnership

Dedicated resources (1 or 2 developers)

Translator partners with manager to identify and prioritize highest leverage improvement opportunities

Continue to share Project Visibility Updates each month

3-Month Improvement Trial

Now we can steer!!

So#ware(Task(

Mi.gate(the(Risks(

Product(Development(

Product(Work(Queue(

Risk(Mgmt(Work(Queue(

Product(Owner(

Product(Decisions(

Knowledge(of(Customers(

Technical(Risk(Manager(


Knowledge(of(Risks(

Dev$Team$Capacity$

Manager

Alloca.on(Decisions(

Refactor Step 3: Technical Risk Manager

Optimize for Safety

Step FOUR:

(continuous improvement)

Quality Target

Lower Control Limit

Upper Control Limit

X"

This is “Out of Control”

Lower Variability = Better Control

Steering Wheel = Process Control

“Pain Control” in Software Development

Optimal Friction

Upper Control Limit X"

“Out of Control”

20min

0m

Average Pain per Incident

This is Better

Target

Control Limit

Categorize the Pain Type

Time%Pressure%


Speed%


%


Constant'Urgency'


0%

100%



Chaos Reigns Unpredictable work stops

fitting in the timebox

Troubleshooting

Progress

Learning

Gambling Habit —> Gradual Loss of CONTROL

Fewer%Problems%to%

Fix%

Stop%%and%Think%

Mi8gate%the%Risk%

Increased%Produc8vity%

and%Innova8on%

Safety'

Cycle of Safety Low-Risk Decision Habits

Fewer%Problems%to%

Fix%

Stop%%and%Think%

Mi8gate%the%Risk%


and%Innova8on%

Safety'


Fewer%Problems%to%

Fix%

Stop%%and%Think%

Mi8gate%the%Risk%


and%Innova8on%

Safety'


Fewer%Problems%to%

Fix%

Stop%%and%Think%

Mi8gate%the%Risk%


and%Innova8on%

Safety'


Input: Decision Constraints

Target: Optimize the Rate of Idea Flow

short-term loop long-term loop

1. Visibility

2. Clarity

3. Awareness

F ocus!

Output: “Friction” in Idea Flow

Improve Quality of Decisions

Steering Wheel = Pain Control System




1. Visibility

2. Clarity

3. Awareness

F ocus!


Target - The direction of “better”






1. Visibility

2. Clarity

3. Awareness

F ocus!


Input - The constraints that limit our short-term choices…





1. Visibility

2. Clarity

3. Awareness

F ocus!


Output - The pain signal we’re trying to improve





1. Visibility

2. Clarity

3. Awareness

F ocus!


Focus on the biggest pain…

F ocus!





1. Visibility

2. Clarity

3. Awareness

F ocus!


1. Visibility - Identify the specific patterns

1. Visibility





1. Visibility

2. Clarity

3. Awareness

F ocus!


2. Clarity - Understand cause and effect

2. Clarity





1. Visibility

2. Clarity

3. Awareness

F ocus!


3. Awareness

3. Awareness - Stop and think to adjust habits





1. Visibility

2. Clarity

3. Awareness

F ocus!


Run Experiments to Learn What Works





1. Visibility

2. Clarity

3. Awareness

F ocus!


Systematically Improve the Quality of Decisions


What’s a Decision Principle?

Big Pain

Little Pain

Trade-off Decision

Answers two questions:

How do I evaluate my situation?

What should I optimize for?

Explicit “Know How”

The Haystack Principle

Lots of unvalidated changes

Easier to find the needle.

Example: The Haystack Principle (batch size)

Optimize for small manageable haystacks.

Optimal Friction



20min

0m

Average Friction per Incident

Target Rate

Control Limit

Decision Principles give us the capability to do this:

High Friction

Min Friction

Process Control is the Discipline of the Feedback Loop(six-sigma)

Optimal Friction



20min

0m

Average Pain per Incident

This is Mastery

Target Rate

Control Limit

This is the Process of Mastery

Categorize the Pain TypeHigh Pain

Min Pain

Optimal Friction



20min

0m

Average Friction per Customer

This is Traction

This is the Process of Lean Startup

High Friction

Min Friction

“No” Threshold

Customer Happiness

Categorize the “No” Type

Optimize Traction Along Three Dimensions Three Dimensions of Organizational Knowledge

Discovery (Expansion)

Peer Mentorship (Diffusion)

Optimize the Whole (Acceleration)

“Decision Principles” (models)

Decisions are managed by a distributed Network of Open Mastery Circles

The Open Mastery Circle

Circle Leader

Circle Member

Focus: What’s the problem to solve?

Observe: Ask questions about the facts

Conclude: What’s causing the pain?

Optimize: How do we reduce pain?

Learn: What questions does this raise?

Observation Questions

Make a F.O.C.O.L Point!

Summarize ideas in a #HashTagged Blog.

(problem-solving focus meetings)

Discovery (Expansion)

Peer Mentorship (Diffusion)

Optimize the Whole (Acceleration)

Priority 1

Priority 2

Priority 3

Priority 1

Priority 2

Priority 3

Priority 1

Priority 2

Priority 3

Risk Management Becomes an Investment Decision

Biggest Pains

$$Biggest Skill Gaps

$Limiting Constraints

$$$$

4. Optimize the Whole

2. Design the Strategy

1. Identify Obstacles (Focus)

3. Learn What Works (Traction)

Steering Wheel

Create a Steering Wheel to Drive the Organization!

$$ $$

4. Optimize the Whole

2. Design the Strategy

1. Identify Obstacles (Focus)

3. Learn What Works (Traction)

Steering Wheel

$$ $$“know how”

Create a Steering Wheel to Drive the Organization!

Focus: Quality of Improvement Decisions

Focus Traction

$$$ $$

Focus: Quality of Investment Decisions

Risk Translator

(interface)

Management(Coordination)

Engineering(Execution)

Open Mastery Learning Framework

Idea Flow Pain Sensor


Janelle Klein

These are the Blueprints!

++

We have the opportunity to take our organizations to a whole new level of effectiveness

If we choose to do it.

I Came Here to Say:

Why? Because this REALLY Sucks.

The problem: Our blueprints are broken!

93% of problems are caused by the system - Deming

LEARN YOUR WAY TO AWESOME.

Here’s Your Personal Invite:

The Conversation is on Slack

We’re debating the patterns and principles of Idea Flow in Wikipedia

It’s a Developer Party!

We’re sharing our ideas over Blogs

(and managers are invited)

Industry Peer Mentorship Network

Companies

Community Groups

“Pay it Forward”

HQ in AustinOpen Mastery

Austin (meetup.com)

Let’s Make the PAIN Visible!


Janelle Klein

+

Running Lean Ash Maurya

Scaling Lean Ash Maurya

The Lean Startup Eric Reis

…then Learn our Way to AWESOME!

…and have a conversation about THIS:

Personal Mastery

Mental Models

Shared Vision

Team Learning

Systems Thinking

As an industry.

Join the Conversation


THIS PRINCIPLE

if you BELIEVE IN

Open Mastery is a roadmap that takes us from where we are to where we want to be

Point A Point AWESOME.

AS AN INDUSTRY.

(Why? Because Open Mastery is a Learning Organization!)

You’re invited to the party! openmastery.org


http://openmastery.org

openmastery.org

@janellekz

How to Build a Learning Organization

Business