Experimentation Mindset

#adchi / #ExperimentationMindset / @DocOnDev

The Experimentation Mindset

Doc Norton, CEO

doc@wearecto2.com

@DocOnDev

@adchi / #ExperimentationMindset / @DocOnDev

Mindset

A simple experiment

Two Groups

Small Difference; Big Result

Mindset

the ideas and attitudes with which a person approaches a situation

fixed growth

Dreyfus Model of skills acquisition

Novice

Advanced Beginner

Competent

Proficient

Expert

Where are you?

Most think they’re Expert

most stay at Proficient

From Implementation

Implementation Mindset get it right

to experimentation

experimentation Mindset explore different ways

but in a typical company…

telephone sales

Follow the script

This will

great!

It wasn’t Great

Ptolemy Janet Raphael Susan Kathy Tommy Doc Chris P.

Weekly Sales Rankings

Watch The Mastermy name is ____________________ …Hello (sir/madam),

Watch The Mastermy name is ____________________ …Pat MichaelsGood Day!

Experiment

experiment

top in sales!

Doc Ptolemy Janet Kathy Tommy Raphael Susan Andy

Weekly Sales Rankings

8 week review

#adchi / #ExperimentationMindset / @DocOnDevnot the feedback I expected

My last day

Best Practices

Implementation Mindset get it right

Single Loop

Assumptions Actions Outcomes

Get it Right

The Dark Side

double Loop

Get it Right

Explore Different Ways

best practices

leading practices

from implementation to experimentation

experimentation

split (a/B) + Multivariate

Cumulative Flow

velocity trends

code quality

employee engagement

team joy

Retrospectives

refine their process

Orientation + On-Boarding

Interest Leagues

hackathons

mentorships

360 / peer reviews

managers institutionalize procedure

- Gabe Newell

being open about [pay] breeds trust

- Leo Widrich

managers [are] mentors- Kevin Goldsmith

four things

know your purpose; challenge assumptions

make failure acceptable

– Henry Ford

“Failure is simply the opportunity to begin again, this time more intelligently.”

think big… start small… .

be intolerant of mediocrity

keep experimenting

Get it Right

Explore Different Ways

Thank You!

The Experimentation Mindset

• Mad Scientist - https://www.flickr.com/photos/dzingeek/4587871752/sizes/o/

• Stages of Growth - http://superkoopatroopa.deviantart.com/art/Terri-Growth-chart-125875917

• Elephant and Mother - http://upload.wikimedia.org/wikipedia/commons/7/75/Baby_asian_elephant_(with_his_mother)_born_in_the_Lao_Elephant_Conservation_Center_in_Sayaboury.jpg Phone Script - http://www.docstoc.com/docs/6629075/Sample-Sales-Script

• Phones -

• http://www.enumclawsda.org/uploads/1/3/3/9/13391741/1971539_orig.gif?119

• http://4.bp.blogspot.com/-qASUt2T7C7o/ULvJ8HVU3LI/AAAAAAAAACw/LufW5HYhCyM/s1600/Tel%C3%A8fon+fixe+antic.png

• https://c2.staticflickr.com/8/7443/8758001086_2692f91620_h.jpg

• http://cdn2.hubspot.net/hub/147894/file-270530761-jpg/images/phone-large.jpg

• Mob Programming - https://techblog.betclicgroup.com/wp-content/uploads/2014/09/mob-programming-1-1024.jpg

• Spotify Matrix - http://blog.kevingoldsmith.com/wp-content/uploads/2014/03/Screen-Shot-2013-11-09-at-7.30.08-AM-1024x644.png

• a/b | multivariate testing - https://www.dynamicyield.com/wp-content/uploads/2014/05/testing.jpg

• Hackathon - https://m2.behance.net/rendition/pm/22070355/disp/f4d62818fcb09d43722a945486ca5300.png

• Things - http://41.media.tumblr.com/d721f204c69de1b5a6a4899429430c9e/tumblr_n52o0matip1snioclo1_1280.jpg, http://cdn.instructables.com/FIT/POMF/HKJTCKRQ/FITPOMFHKJTCKRQ.LARGE.jpg, http://shadowgrail.deviantart.com/art/Rock-of-Ages-83993992, http://vo5.deviantart.com/art/Fantastic-Four-163610854

Images

• Students Test - https://img.washingtonpost.com/rf/image_1484w/2010-2019/WashingtonPost/2014/09/17/Others/Images/2014-09-17/DSC_54171410973389.jpg?uuid=h6VsJD6MEeSkMLgqPme3Yg

• Woody Zuill - https://pbs.twimg.com/media/A8NKQ9lCQAAdA7h.jpg

• Code Quality (WTF) - http://www.sonarqube.org/wp-content/uploads/2009/09/wtfm.jpg

• 360 Feedback - http://screencrush.com/442/files/2012/07/360-review.jpg

• Test Tube - http://fc09.deviantart.net/fs70/i/2010/018/2/1/Vector_Test_Tube_by_Algerianoo.jpg

• No Estimates - https://twitter.com/NoEstimates

• Tesla - https://c2.staticflickr.com/4/3006/2932363100_5bf7ff5820_b.jpg

• Smiley - https://blogs.dal.ca/csc/files/2012/07/MP900443129.jpg

• Dilbert - http://www.stephenshapiro.com/images/dilbert-bp.gif

• Manager’s Office - https://www.flickr.com/photos/islespunkfan/8281372547/sizes/l

• Bad review - http://media.benefitspro.com/benefitspro/article/2014/02/06/negative%20reviews.jpg

• Captain my Captain - http://startupeers.co/wp-content/uploads/2014/08/o-captain-e1407919367905.jpg

• Vader in Suit - http://fc06.deviantart.net/fs71/f/2011/179/a/a/darth_vader_by_iamthedarthvader-d3k9n2h.jpg

• League - https://oliverwillis.com/wp-content/uploads/2015/08/league-of-legends.jpg

• Mentorship - http://humancapitalist.com/wordpress/wp-content/uploads/2013/11/canstockphoto15167410.jpg

more images

• Dreyfus Model of Skills Acquisition - http://en.wikipedia.org/wiki/Dreyfus_model_of_skill_acquisition, http://files.eric.ed.gov/fulltext/ED384695.pdf#page=130, https://www.dropbox.com/s/px1d5orrtjm5d5i/ADA084551.pdf?dl=0

• Learning through experimentation - http://lifehacker.com/5985565/how-and-why-to-intentionally-set-yourself-up-for-failure

• HBR: Collective Genius - http://hbr.org/2014/06/collective-genius/ar/1

• How Organizations Learn - http://www.areteadventures.com/articles/the_challenge_of_entering_the_green_room.pdf

• Argyris: Teaching Smart People how to Learn - http://hbr.org/1991/05/teaching-smart-people-how-to-learn/ar/1, http://pds8.egloos.com/pds/200805/20/87/chris_argyris_learning.pdf

• Argyris: Double Loop Learning in Organizations - http://hbr.org/1977/09/double-loop-learning-in-organizations/ar/3

• A/B Testing and the Experimentation Culture - https://hbr.org/2014/02/ab-testing-and-the-benefits-of-an-experimentation-culture/

• Enlightened Experimentation - https://hbr.org/2001/02/enlightened-experimentation-the-new-imperative-for-innovation/ar/4

• Argyris: Single-Loop and Double-Loop Learning in Research on Decision Making - https://093#1c5e28761e25b79cfc10de03f5a188778a0.googledrive.com/host/0B_RO2ClD0V7AZHpWRS1mNUc2d1k/argyris.pdf

• Ed Batista Blog - http://www.edbatista.com/2008/05/double-loop.html

Reference Material

• Lean Change - http://www.lean.org/LeanPost/Posting.cfm?LeanPostId=225

• Scaling Up Excellence - http://www.amazon.com/Scaling-up-Excellence-Hayagreeva-Rao-ebook/dp/B00EGMQIDG

• Start With Why - http://www.amazon.com/Start-Why-Leaders-Inspire-Everyone-ebook/dp/B002Q6XUE4/

• Leading Change - http://www.amazon.com/gp/product/0875847471/

• Woody Zuill - https://twitter.com/woodyzuill, http://zuill.us/WoodyZuill/

• Mob Programming and No Estimates - http://www.infoq.com/interviews/zuill-mob-programming

• Democratizing Experimentation - https://hbr.org/2011/04/8-ways-to-democratize-experime/

• Failing by Design - https://hbr.org/2011/04/failing-by-design/ar/1

• Smart Business Experiments - https://hbr.org/2011/03/a-step-by-step-guide-to-smart-business-experiments/ar/1

• Evidence Based Management - https://hbr.org/2006/01/evidence-based-management/ar/5

more reference material

• Buffer Transparency - http://www.fastcompany.com/3024306/bottom-line/why-this-startup-made-their-salaries-radically-transparent

• Buffer Transparency Fail - http://www.fastcompany.com/3050682/lessons-learned/why-our-transparent-feedback-experiment-failed

• Valve - https://blog.inf.ed.ac.uk/sapm/2014/03/14/management-at-valve-an-analysis-of-management-practices-at-valve-software/

• Valve Handbook - https://www.valvesoftware.com/company/Valve_Handbook_LowRes.pdf

• Spotify White Paper - https://dl.dropboxusercontent.com/u/1018963/Articles/SpotifyScaling.pdf

• Kevin Goldsmith on Spotify - http://blog.kevingoldsmith.com/2014/03/14/thoughts-on-emulating-spotifys-matrix-organization-in-other-companies/

yet more reference material

Experimentation Mindset

Leadership & Management

Aperture Experimentation

Anmal Experimentation

Animal Experimentation in Research - DFG - …...

Experimentation intervention

Layout Experimentation

Beyond Experimentation

RANSAC experimentation

Digipak experimentation

AMN Experimentation

Mindset- Growth Mindset V Fixed Mindset

Managing experimentation

Fixed Mindset and Growth Mindset

Innovation starts with Culture · 9. Adopt an...

DIGITAL FIRST - chicagohistory.org · 21/06/2018 ·...

Bringing an Experimentation Mindset to Personalization

Objectives: Materialsbetween a Fixed Mindset and a Growth...