Conway’s Revenge: How ignoring org setup backfires in architecture, and vice versa

Conway’s revenge

How ignoring org setup

backfires in architecture,

and vice versa

How do you keep quality

in 1-man project?

Test Driven Development?Following best practices?Not being stupid?


in 1-man project?

Self-discipline


in a 1 team project?

Simple: hire only smart people!



So smart engineers will come to the same conclusions, right?



So smart engineers will come to the same conclusions, right?

But when we agree on unifying, it will be alright. How do you

keep quality in a 1 team

project?

USB: UNIVERSAL Service Bus



USB: UNIVERSAL Service Bus





How was this problem being solved before?Siloses, jars, maven/gradle modules



Problems:truck factor = 1final solution looks like this



Current solutions

Pull requests + code reviewPair programmingSharing good practicesMentoring

Self-disciplineTeam-disciplineLast resort: EnforcementHow do you

keep quality in a 1 team

project?


in a 50 people

project?


in a 50 people

project?

Discipline?


in a 50 people

project?

Discipline?In a creative, passionate crowd?Have you ever seen a rock concert?


in a 50 people

project?

Engineers be like:Static code analysis? Pipeline + Sonar?Inquisition!Purge the heretics!


in a 50 people

project?

Managers be like:Laws, rules, committees, permission control. Control! Moar control! MOAR!!1!


in a 50 people

project?

Productivity and creativityrequires freedom

Fine balance between chaos and order


in a 50 people

project?

Solutions so far:Shared responsibility Education & MentoringCorporate...


in a 50 people

project?

Solutions so far:Shared responsibility Education & MentoringCorporate… Hacker...


in a 50 people

project?

What they had in common was mainly love of excellence and programming. They wanted to make their programs that they used be as good as they could. [Richard Stallman explains about hackers who program]


in a 50 people

project?

Culture is hard to make rightHackers hate corporationsHackers like freedom

How to solve people’s

problem with technology?







Requires:DevOps as in “developers operating their systems in production”Distributed monitoringDistributed securityGood knowledge of distributed systems, CAP theorem, etc.Lots of infrastructure setup, code, libs...



But it’s mostly a technical problemDevelopers are good at solving technical problems



Let’s see where we can fail

1Too many products

Orchestration

You have a distributed system (eCommerce) that has to go from single product for single country, into 5 slightly different products for 20 slightly different countries (5x20 matrix)

How do you do it?

Orchestration

Orchestration

Options:1. Country/product specific

modules per microservice

Orchestration

Country/product specific modules per microservice

Modules

If you have country/product specific modules per microservice

If this service grows (lots of country/product specific features) it will become the new monolith.

Difficult to maintain.

Modules

Except not really, because you can have self-discipline/quality as long as you only have one team for that service

But what if requirements grow so much, that you need to add more teams to it? Then it’s a mess again

Orchestration


modules per microservice2. Proxy-microservice in front,

with country/product specific logic

Proxy-microservice, with country/product specific logic

Proxy

If you have proxy-microservice in front, with country/product specific logic

It may not always be possible (proxy transforms input/output data), and you may need to write a new instance of a microservice, just to change a few things inside

And hundreds of services are hard to maintain

Orchestration


modules per microservice2. Proxy-microservice in front,

with country/product specific logic

3. One orchestration service to rule country/product flow, services sharing many entry points

One orchestration service to rule country/product flow, services sharing many entry points

Orchestration

If you have an orchestration service, either:

There is a team responsible for it, therefore creating a huge bottleneck (everybody waits for them)

Nobody is responsible for it, therefore it’s a total mess, impossible to maintain

Conclusion

If for each of your services you choose between modules and proxies, you will- stay within reasonable

number of services- not copy&paste to change 2

lines of a service inside- not use modules, when the

service has a chance of growing to >1 teams

Conclusion

Orchestration, however, is the worst solution for this company

Because of lack of ownership/self-discipline

Conclusion

Unless every team has one guy from this project(2D team system)

But that complicates a lot and is risky (may not work in the long run). Is it worth it?

2Cool

technologies, boring people

People on the market

Scala + Akka

So you want a cool technologySmall country~100 Scala developers~30 Akka developersCan get ~10% from the marketBusiness success, needs to hire ~30 devs


Scala + Akka

Options:- accept slowing down, recruit

juniors, train them- become a 100% remote

working company (big culture change)

- remove Akka, drop Scala


Scala + Akka

Accept slowing down, recruit juniors, train themManagement doesn’t want to slow down

Become a 100% remote working company (big culture change)Management doesn’t trust developers enough

Remove Akka, drop ScalaManagement doesn’t give a fuck


Scala + Akka

Lesson:

Ignore management, organization, structure, and your architecture will perish

3Cool people,

boring technologies


Delphi

Year 2003. Business analyzes local University. They still teach Turbo Pascal

Enterprise architects + management decide to build new platform with Delphi (Object Pascal) because it’s close to what is at the University


Delphi

They chose Kylix (Linux version)

Good business - Linux: free- Students: almost free, educated- Borland Kylix: cheap


Delphi

Students were learning Turbo Pascal, because the University had old teachers, that couldn’t teach anything else.

In 2003 not a single developer would like to work in Kylix

Students treated this job as a start to go somewhere else

EnterpriseService

Bus

An architect wanted to introduce ESB in 2014, for internal distributed system

3 months later, his team rebelled


Delphi

Lessons:

Ignore fashion, needs of developers, and you will end up without developers

4How much

abuse can your architecture

take?

Resistant architecture

DDD + CQRS, for clean design

Company hires lots of new devs

Big ball of mud (all entities have relation to each other, no boundaries)


Does your architecture fit the people who are going to use it?

How do you communicate constraints?

How do you verify you architecture?


Integration Tests for architectureVeripacksSeparate modulesSeparate DBs for bounded contextsMake it hard to break architectureWiki, documentation

5Two styles of management

Two style of management

How do sell your microservice architecture to management?


“I’ve also noticed that different countries and cultures place different values on control.

Some (e.g. the UK) value control and the restraint that it brings whereas others (e.g. Scandinavia)value empowerment and motivation.”

[Software Architecture for Developers; Simon Brown; Leanpub 2014]


For Scandinavia:- emphasise productivity- invest in tools to get people

up to speed- get to production fast- show how much devs like this

architecture


For UK: - create a service visualization

tool (who talks with whom)- emphasise control, and

visibility- create a reporting service- get metrics working first- use magic word: SOA

(English people love three letter acronyms)

6Why big

systems go bad

Conway’s Rationale

1Managers realize the system will be large, so they throw too many people at the design

2many people = too many communication paths = zero productivity

so organizations limit communication by creating design subgroups

3

large organization can understand only tree structure with single superior + 7 subordinates.

so design subgroups are organized this way

this limits communication channels to this structure, hence: miscommunication

4relationship between the graph structure of a design organization and the graph structure of the system it designs is 1:1

5so the final design also has the wrong structure, and is build on miscommunication

Conclusion

Conclusion

Look at your architecture from communication perspective

Will communication be efficient?

Can your organization handle this architecture?

Conclusion

Even big systems should be designed only in a small group

What a small group cannot handle, a big one will fuck up even more

Conclusion

Design architecture AND organization

World has no boundariesBoundaries are in our mindsAll systems are connectedAll systems interact

And read Conway’s paper:http://www.melconway.com/Home/Committees_Paper.html

Thanks!

[email protected]@jnabrdaliksolidcraft.eu

Conway’s Revenge: How ignoring org setup backfires in architecture, and vice versa

Technology