The Pivotal Engineering Dojo: Earning Your Black Belt in Cloud Foundry Engineering (Cloud Foundry Summit 2014)

CF dojo experience earning your black belt in CF engineering

dr.max @maximilien julz friedmanibm cloud labs v0.5.0

June 11, 2014

photo credit: http://urbosquid.com

2

photo credit: The Matrix, Warner Bros.

overview of this talk

• how do I contribute to Cloud Foundry?

‣ yourself

‣ your company

• why the dojo program?

• is agile the same everywhere?

• our experiences?3

4

photo credit: The Matrix, Warner Bros.

It Works!

photo credit: The Matrix, Warner Bros.

In theory, there is no difference between theory and practice, however, in practice there is.

Big company agile vs. Pivotal agile

some differences

11

• Single Coder

• Sometimes a long time to come up to speed

• Knowledge sharing by training sessions, some on-the-job training, documentation..

• Big difference in productivity between junior and senior programmers

• Rely on code review, QA etc.

@Big Company Single Coder

12

• Mentally Taxing!

• Very impressive team, quick working, have to be on your game to keep up

• Reliance on high test coverage, code quality

• Pairing is synchronous code review

@Pivotal Pair programming

.. probably not for everyone

14

• Quite agile, but adapted to the realities of the business (Distributed Programmers, Project Management, Detailed Feature Tracking..)

• Sometimes uses Agile terms without actually being all that agile.

• Some test-first development. But not nearly pervasive enough. credit: http://kasperowski.com/2010/09/

hardening-sprints-sorry-youre-not-agile.html

@Big Company “agile” methodology

15

• HEAVY reliance on test-first development, refactoring & communication

• Huge opportunity to learn

• e.g. Limited documentation, lightweight stories, minimal up-front architecture and design, aggressive refactoring + wicked fast feedback cycle.

• Very effective BUT reliance on being in the room.

• How to make it scale for a distributed project? (will come back to this)

@Pivotal XP-style methodology

16

photo credit: The Matrix, Warner Bros.

our dojo experiences

• spent 7-8 weeks at Pivotal SF working with CF team

• tried to work with as many teams as possible

• primary goals were to:

1. learn CF codebase, lean how to debug, learn how to contribute

2. meet as many members of CF team as possible, build relationships

3. gain credibility for myself and IBM

• overall experience positive - achieved most goals

17

standups

18

pairing

19

runtime team

• maybe the most important CF teams (Diego + regular runtime)

• runtime integrates all pieces. Runs and manages: Tabasco, A1, and Production Pivotal CF environments

• typically seen as chaotic - especially while I was there (syslog issue)

• always under lots of pressure, however, can be rewarding

• not the ideal place to learn CF tools or CLI or CF itself, but great to learn internals of cloud-controller and DEA components

20

bosh team

• under less pressure than runtime but nonetheless critical

• vision for BOSH evolving to be a tool that can be used outside of CF

• some “star” Pivotal engineers are on BOSH

• has its own “language” (release, jobs, packages, spec, etc)

• not the place to learn how to use BOSH, however, great to learn internals and its goals and directions

• BOSH is great once you understand, learn its language, and learn its key goals

21

miscellaneous

• got a chance to pair and meet docs team

• got to meet, but not paired with, services team

• got to meet and indirectly-pair with other team Pivotal product team members (especially Tempest, PHD, CLI)

• good recap with Rob Mee and directors

• got to provide feedback to Pivotal culture (good and bad)

22

photo credit: The Matrix, Warner Bros.

thoughts - positive

• Pivotal has welcoming and “be nice” culture, easy to fit, but must contribute

• striking mismatch between Pivotal culture and IBM’s (emails and meetings)

• decisions are super fast since directors, PMs, and engineers are in close proximity

• Pivotal is one of few companies doing XP with high-level of "discipline" and in particular Pair Programming and TDD

• entire floor of two-floor Pivotal Labs in SF dedicated to CF (rotations all the time)

• significant IBM contributions to CF will require embedding into that culture

• no “schedules”, no dates, works get done when done... Pivotal tracker (similar to online RTC) is used to track work... GitHub for all code

24

thoughts - not so positive

• lack of slack => engineers have hard time innovating (“story blinders”)

• decisions while fast are not necessarily data-driven

• pairing helps with feedback and checks, but => less external documentation

• “fanatic” move to Go-lang seems to be generally welcomed but I think Go while good for some system-level things is not necessarily a panacea; Ruby will soon be missed :)

• “Big ball of mud” happens for all software (Go, Ruby, etc). Second implementation helps with learning and getting better

• TDD gets abused, sometimes - especially with overuse of mocks

25

thoughts (cont.)

• highly recommended to all engineers wanting to contribute to CF

• read and practice TDD and pairing and basic data structures

• be ready to embed yourself into culture, be open, be nice

• very few places do full agile, like Pivotal, so this by itself is a great learning experience

• continue adding to you and company’s credibility

• helps put a human face to the Github IDs or vcap-dev names

26

27

photo credit: The Matrix, Warner Bros.

Thank you Q & As

28

Backup

photo credit: The Matrix, Warner Bros.