From a Small Club to a Giant Stadium - SC2 · Spotify Project Title 00.00.2015 Proprietary & Confidential 1 From a Small Club to a Giant Stadium Scaling Engineering at Spoti! Wouter

Project Title 00.00.2015 Proprietary & Confidential 1Spotify

From a Small Club to a Giant StadiumScaling Engineering at Spotify Wouter de Bie - Director of Engineering wouter@spotify.com

2Project Title 00.00.2015 Proprietary & ConfidentialSpotify

How did we go from this..

.. to this?

00.00.2015 4Spotify

What is our scale?

00.00.2015 5Spotify

Spotify was founded in 2006. First launched in Sweden in 2008

Spotify went from 1 to 60+ countries in 9 years

Spotify went from 0 to 150 Million users in 9 years

00.00.2015 6Spotify

Currently we’re at about 3500 employees, of which 1700 are in R&D

Spotify consists of 400+ micro services

We have 60+ PB of data under management

00.00.2015 7Spotify

Scaling Spotify • The organization • Our technology platform

I’ll be talking about the two parts that we scaled over the years: our org and tech platform.

00.00.2015 8Spotify

Tech organization

We started with an oldskool tech org, as seen in many organizations.

Project Title 00.00.2015 Proprietary & ConfidentialSpotify

Backend Frontend Data QA

Different team doing different things. The Backend Team does backend, the frontend team does frontend, etc

But all these teams are dependent on each other. And as the org grows, those dependencies become bigger and bigger. Also, these teams don’t move in lock-step and have different priorities and eventually there is the chance of a dead-lock since they are all waiting for each other. Since we need to stay ahead in the game, we need to move fast.

00.00.2015 11Spotify

The Matrix organization

Because of this we designed the Matrix Org

00.00.2015 12Spotify

Autonomy is the word…

It’s all about autonomy. We believe that through autonomy (aka cutting down on dependencies), we achieve speed.

Instead of a team per competence, we create a team that can take on a task or mission from A to Z with all the competence needed.

We call this a squad

Agile Coach

Product Manager

And we add a product manager and an Agile Coach

Who’s the Boss?

Problem: who’s the boss of the squad? PM? AC?

Chapter

We organize hierarchically in Chapters

Chapter

Chapter Lead

And a Chapter Lead is a first line engineering manager

VP Engineering

And those chapter leads reported to the VP of Engineering, who reported to the CTO

“What if we have multiple $competence chapters?”

Say that we have a bunch of chapters

And some of them have the same competence (e.g. backend)

We create a Guild where engineers with that competence come together

00.00.2015 25Spotify

Tribes

VP Engineering

Initially our org looked like this. All CLs reporting to the VP of Eng

VP Engineering

But this group grew too big, so we had to do something

We split up different chapters and let them report to multiple VPs.

Tribe Tribe

And the sub organization we call a tribe.

“What if we have multiple $competence chapters in different

tribes?”

00.00.2015 31Spotify

Still one guild for one competence. Just a little bigger..

CTO Head of Product

At the time we did this, our product org was separate and had PMs “embedded” in squads. Product reported to a Head of Product

00.00.2015 33Spotify

Missions

After a while, the amount of tribes became to big for the CTO, so we had to come up with something else.

Head of R&D

We split the tribes into groups

Head of R&D

Mission

And called it a mission

Creator User Engagement

Core Experience Revenue Growth

Opportunities

Technology Platform

Design / Product Insights

Now we have 8 mission in total and we make a distinction between vertical an horizontal missions. Here are the vertical missions

Opportunities

Technology Platform

And the horizontal missions that support the vertical ones

Opportunities

Technology Platform

00.00.2015 40Spotify

A Typical Mision • One or more Engineering Tribes • Product organization • People Operations • (sometimes other mission specific sub-orgs)

VP Data

Engineering Operations Research Solutions Design Product

My mission

“What if we have multiple $competence chapters in different tribes

in different missions?”

00.00.2015 43Spotify

Still one guild for one competence. Just a little bigger..

00.00.2015 44Spotify

Organizational Learnings

Some learnings that we have had over the years.

00.00.2015 45Spotify

No Org Model is Perfect Spotify is optimized for speed

through autonomy, Other things don’t come easy and

have to be mitigated for

00.00.2015 46Spotify

Our org model seems complex

On-boarding is sometimes a little hard because of terms

and internal roles used

00.00.2015 47Spotify

Dependencies move up the stack

First there were team dependencies, then tribe dependencies and now Mission dependencies

00.00.2015 48Spotify

Scaling Technology

The only stereotypical stock photo image. Apart from the org there is tech

00.00.2015 49Spotify

Micro Services vs

Monolith And have a standard protocol

What does this accomplish?

Autonomy

00.00.2015 51Spotify

Ops in squads a.k.a. DevOps

Autonomy

00.00.2015 53Spotify

Transition to the Cloud Google Cloud Platform in our case

Autonomy … and also because other companies are better at running a data center

00.00.2015 55Spotify

Trying to prevent multi-tenant solutions … think about a 3500 node cluster with 1500 users

Autonomy

00.00.2015 57Spotify

Trying to prevent multi-tenant solutions … think about a 3500 node cluster with 1500 users

00.00.2015 58Spotify

The downside No organizational model is perfect.

• Autonomy requires a lot of coordination.

• Some things need centralized management like data governance or quality

Obviously there is a downside too..

That’s it! Questions?

From a Small Club to a Giant Stadium - SC2 · Spotify Project Title 00.00.2015 Proprietary & Confidential 1 From a Small Club to a Giant Stadium Scaling Engineering at Spoti! Wouter

Documents

Giant Cell Tumor / Giant Cell Myeloma / Osteoclastoma

GENDER ISSUES IN GIANT FRESHWATER: GIANT … ISSUES IN GIANT...

Giant Steps -...

Cape Town Stadium Durban StadiumPolokwane StadiumRustenburg....

MANUAL OF RESPONSIBILITIES - Blue Giant - Blue Giant ...

MetLife Stadium 1 MetLife Stadium Drive East Rutherford...

Durban Stadium Polokwane Stadium Rustenburg Stadium...

New Meadowlands Stadium Stadium

MetLife Stadium (formerly New Meadowlands Stadium...

Hollett Little Giant Stadium (14,000) - Wabash College at...

Dynamic assessment of wind sensitive stadium · PDF...

Stadium Series contract for TCF Bank Stadium

NEW MEADOWLANDS STADIUM PROJECT - New Jersey ·...

New National Stadium International Concept Design...

Stadium: Jesse Owens Stadium Officials: Referee: Nabil ...

This Giant Stadium Roof - Sexton...