Dependency Management at Netflix

Dependency Management at Netflix

Dependency management in a vast microservice landscape

Thin Services, Fat Clients

Freedom and Responsibility

Responsible people thrive on freedom and are worthy of

freedom

The Binary Dependency Choice

In principle provides the most freedom of action per team

Each team selects a build tool that matches their style

Version SkewHow many versions am I

behind the latest available version?

Skew leads to runtime execution paths that were not

unit tested!

StabilityBuild repeatability

Don't make me change until I'm ready!

Nebula Dependency LockingCaptures results of dependency resolution now

so the same results can be reused later./gradlew generateLock./gradlew -PdependencyLock.useGeneratedLock=true test./gradlew saveLock commitLock

Nebula Dependency LockingTo force every project in a multimodule project to

agree on dependency versions:./gradlew generateGlobalLock./gradlew -PdependencyLock.useGeneratedLock=true test./gradlew saveGlobalLock

Avoidance

Astrid

AdepthubUses a version compatibility matrix to calculate a

known valid solution

ModularizationShading involves package relocating dependencies

makes them globally uniqueRuntime modularization like OSGi and JBoss Modules are too constraining for us, work

well in some contexts

Gradle Shadow PluginRequires action on the part of the dependency

producershadowJar { dependencies { include(dependency('com.google.guava:guava:18.0')) } relocate 'com.google', 'shaded.com.google'}

(roughly equivalent to the Maven Shade Plugin)

Gradle Shadow Pluginpublic class NameAgeClient { Multimap<String, Integer> agesByName = HashMultimap.create();

public void addAll(Multimap<String, Integer> agesByName) { this.agesByName.putAll(agesByName); }

public Integer maxAge(String name) { return agesByName.get(name).stream().max(Integer::max).orElse(0); }}

Gradle Shadow PluginShaded transitive dependencies are leaked to

dependency consumer@Test public void demonstrateUnshadedSeam() { shaded.com.google.common.collect.Multimap<String, Integer> nameAges = HashMultimap.create(); nameAges.put("jon", 10);

NameAgeClient client = new NameAgeClient(); client.addAll(nameAges);

assertThat(client.maxAge("jon"), equalTo(10));}

Project NemoA seam exists at the points

where your project interacts with the public API of its first

order dependenciesThe goal is for this seam to

always refer to only your first-order dependencies

Project NemoJust-in-time shading on the dependency

consumer side

Project Nemorepositories { maven { url 'https://nemo.netflix.com' }}

dependencies { // appending _module causes JIT shading compile 'commons-configuration:commons-configuration_module:1.10'}

Project NemoShaded artifacts are generated on the fly and

differentiated by SHA1

Project NemoShaded transitive dependencies are no longer

leaked@Testpublic void demonstrateUnshadedSeam() { Multimap<String, Integer> nameAges = HashMultimap.create(); nameAges.put("jon", 10);

NameAgeClient client = new NameAgeClient(); client.addAll(nameAges);

assertThat(client.maxAge("jon"), equalTo(10));}

Thanks!— we are hiring