Dependencies, distributed code and engineering velocity

Dependencies,Distributed codeand engineering velocity

Mike McGarr@SonOfGarr© J. Michael McGarr, 2017

I have a problem...

© J. Michael McGarr, 2017

...that impacts every engineer at Netflix.


Affects you too.


I don't know the solution.


direction


So what is this problem?


SHAREDCODE© J. Michael McGarr, 2017

cloud microservices




client libraries



transitive dependencies



consuming librariesshould beeasy


software isn't static


semantic versions



(*) 3.1.133.1.143.1.153.1.163.2.03.2.13.2.23.2.3

4.0.0-RC1


Which version is good?



(*) 3.1.13 ❌3.1.14 ❌3.1.15 ❌3.1.16 ✔3.2.0 ❌3.2.1 ❌3.2.2 ✔3.2.3 ❌

4.0.0-RC1 ❓



Semantic versioning is insufficient




Which version of guava will you get?




Pinning versions is technical debt


Just give me a version thatworks!


consuming librariesis actuallyhard


Well, at leastpublishing

is the easy, right?© J. Michael McGarr, 2017

Need to change foobar-client:3.2.2→ Bug fix? 3.2.3

→ Behavior change? 3.2.0→ API change? 4.0.0


Who will I break?


Who is consumingmy library?


Who is using thisAPI

that I want to change?© J. Michael McGarr, 2017

Publishers lack

organization-widevisibility


There are solutions out

there...© J. Michael McGarr, 2017

Option #1:

Just deal with pain


Option #2:

Share nothing


Option #2:

Share nothing little


Option #3:

Monorepo


→ all code in a single repository→ source integration→ head is production

→ no versions→ atomic API changes

→ lock step deployments


constraints


monorepo depends on gates



Our approach...© J. Michael McGarr, 2017

Can we get the benefitsof a monorepo,

without the constraints?© J. Michael McGarr, 2017

Publisher feedbackManaged source

Distributed refactoring© J. Michael McGarr, 2017

Publisher feedback


AstridNiagara





I now know who I will break!


Managed source© J. Michael McGarr, 2017

eliminate reduceconsumer pain




Shorten adoption curve


Give me a version thatworks!


Distributed refactoring


Gradle Lint


How do IChange my API

across hundreds of repos?


1. Find all repositories2. Find all API calls

3. Refactor all API calls4. Issue pull requests en masse


experimental


Publisher feedbackManaged source

Distributed refactoring© J. Michael McGarr, 2017

"Eventually consistent distributed monorepo"-- Avi Bryant (Stripe)


Will this work?


Thank youMike McGarr@SonOfGarr

[email protected]© J. Michael McGarr, 2017

http://www.twitter.com/SonOfGarr

mailto:mmcgarr@netflix

Dependencies, distributed code and engineering velocity

Technology