Decompose that WAR? A pattern language for microservices (@QCON @QCONSP)

@crichardson

Decompose that WAR! A pattern language for

microservicesChris Richardson

Author of POJOs in Action Founder of the original CloudFoundry.com

@crichardson [email protected] http://plainoldobjects.com http://microservices.io

@crichardson

Presentation goal

Why patterns and pattern languages?

A pattern language for microservices

@crichardson

About Chris

@crichardson

About Chris

Founder of a startup that’s creating a platform for developing

event-driven microservices

@crichardson

For more information

https://github.com/cer/event-sourcing-examples

http://microservices.io

http://plainoldobjects.com/

https://twitter.com/crichardson

@crichardson

Agenda

Why a pattern language for microservices?

Core patterns

Deployment patterns

Communication patterns

@crichardson

In 1986…

http://en.wikipedia.org/wiki/Fred_Brooks

@crichardson

Yet almost 30 years later developers are still

passionately arguing over “silver bullets”

@crichardson

Suck/Rock Dichotomy

Spring vs. Java EE

JavaScript vs. Java

Functional programming vs. Object-oriented

http://nealford.com/memeagora/2009/08/05/suck-rock-dichotomy.html

Containers vs. Virtual Machines

@crichardson

Gartner Hype Cycle

http://upload.wikimedia.org/wikipedia/commons/b/bf/Hype-Cycle-General.png

It’s awesome

It’s not awesome

Trade-offs understood

@crichardson

How we make decisions

Decide using

emotions

Rationalize with our intellect

http://en.wikipedia.org/wiki/Mahout

@crichardson

We need a better way to discuss and think about technology

@crichardson

What’s a pattern?

Reusable solution to a problem

occurring in a particular context

@crichardson

The structure of a pattern

=

Great framework for discussing and thinking about technology

@crichardson

The structure of a pattern

Resulting context

aka the situation

Name

Context

Problem

Related patterns

(conflicting) issues etc to address Forces

Solution

@crichardson

Resulting context

Benefits

Drawbacks

Issues to resolve

@crichardson

Related patterns

Alternative solutions

Solutions to problems introduced by this pattern

Pattern languageA collection of related patterns that solve problems in a particular domain

Relationships

Pattern A results in a context that has a problem solved by Pattern B

Patterns A and B solve the same problem

Pattern A is a specialization of pattern B

http://en.wikipedia.org/wiki/A_Pattern_Language

Access to Water

Promenade

Local townhall

Intimacy gradient

Light on two sides

@crichardson

Meta-pattern

Problem: How to talk/reason about technology?

Solution: Use the pattern format

Benefit: More objective

Drawback: Less exciting

Context: Emotional software development culture

Related patterns: It’s awesome!

@crichardson

Monolithic architecture

Microservice architecture

API gateway

Client-side discovery Server-side discovery

Service registry

Self registration 3rd party registration

Multiple Services per host

Single Service per Host

Service-per-Container

Deployment

Discovery

Core

Communication

Service-per-VM

Partitioning

Messaging Remote ProcedureInvocation

Style

MotivatingPattern

SolutionPattern

Solution A Solution B

General Specific

Work in

progress

@crichardson

Agenda


Core patterns

Deployment patterns


@crichardson

@crichardson

Let’s imagine you are building an online store

Browser/Client

SQL Database

Review Service

Product Info Service

Recommendation Service

StoreFrontUI

Order Service

HTML

REST/JSON

@crichardson

Problem: what’s the deployment architecture?

@crichardson

Forces

There is a team of developers that must be productive

The application must be easy to understand and modify

Do continuous deployment

Run multiple instances for scalability and availability

Use emerging technologies (frameworks, programming languages, etc)

@crichardson

Tomcat

Pattern: Monolithic architecture

Browser/Client

WAR/EAR

MySQL Database

Review Service

Product Info Service


StoreFrontUI

Order Service

HTMLREST/JSON

develop test

deploy

Simple to

scale

@crichardson

Examples everywhere

@crichardson

But when the application is large …

@crichardson

Intimidates developers

@crichardson

Obstacle to frequent deployments

Need to redeploy everything to change one component

Interrupts long running background (e.g. Quartz) jobs

Increases risk of failure

Fear of change

Updates will happen less often - really long QA cycles

e.g. Makes A/B testing UI really difficult

Eggs in one basket

@crichardson

Overloads your IDE and container

Slows down development

@crichardson

Lots of coordination and communication required

Obstacle to scaling development

I want to update the UI

But the backend is not working

yet!

@crichardson

Requires long-term commitment to a technology stack

@crichardson

Pattern: Microservice architecture

@crichardson

Apply functional decomposition

X axis - horizontal duplication

Z axis

- data

partit

ioning

Y axis - functional

decomposition

Scale b

y split

ting s

imilar

thing

s

Scale by splitting

different things

@crichardson

Product Info

Microservice architectureProduct Info

Service


Review Service

Order Service

Browse Products UI

Checkout UI

Order management UI

Account management UI

Apply X-axis and Z-axis scaling to each service independently

@crichardson

Examples

http://highscalability.com/amazon-architecture

http://techblog.netflix.com/

http://www.addsimplicity.com/downloads/eBaySDForum2006-11-29.pdf

http://queue.acm.org/detail.cfm?id=1394128

~600 services

100-150 services to build a page

@crichardson

BenefitsSmaller, simpler apps

Easier to understand and develop

Less jar/classpath hell - who needs OSGI?

Faster to build and deploy

Scales development: develop, deploy and scale each service independently

Improves fault isolation

Eliminates long-term commitment to a single technology stack

System level architecture vs. service level architecture

Easily and safely experiment with new technologies

@crichardson

DrawbacksComplexity of developing a distributed system

Implementing inter-process communication

Handling partial failures

Implementing business transactions that span multiple databases (without 2PC)

Complexity of testing a distributed system

Complexity of deploying and operating a distributed system

Managing the development and deployment of features that span multiple services

Fortunately solutions exists

@crichardson

The benefits typically outweigh the drawbacks

for large, complex applications

@crichardson

Issues to address

How to deploy the services?

How do the services communicate?

How do clients of the application communicate with the services?

How to partition the system into services?

….

@crichardson

Agenda


Core patterns

Deployment patterns


@crichardson

We have applied the microservices pattern:

How to deploy the services?

@crichardson

ForcesServices are written using a variety of languages, frameworks, and framework versions

Each service consists of multiple service instances for throughput and availability

Building and deploying a service must be fast

Service must be deployed and scaled independently

Service instances need to be isolated

Resources consumed by a service must be constrained

Deployment must be cost-effective

@crichardson

@crichardson

Pattern: Multiple service instances per host

Host (Physical or VM)

Service-A Instance-1

Service-B Instance-2

Service-C Instance-2

Process WAR

OSGI bundle

Benefits and drawbacksBenefits

Efficient resource utilization

Fast deployment

Drawbacks

Poor/Terrible isolation

Poor visibility (with WAR/OSGI deployment)

Difficult to limit resource utilization

Risk of dependency version conflicts

Poor encapsulation of implementation technology

@crichardson

Pattern: Service instance per host

@crichardson

Pattern: Service per VM host

Service VM image

VM

Service

VM

Service

VM

Service

packaged as

deployed as

@crichardson

Examplehttp://techblog.netflix.com/~600 services

packer.io is a great tool


Great isolation

Great manageability

VM encapsulates implementation technology

Leverage AWS infrastructure for Autoscaling/Load balancing

Drawbacks

Less efficient resource utilization

Slow deployment

@crichardson

VM

VM

Pattern: Service per Container host

Service Container image

Container

Service

Container

Service

Container

Service

packaged as

deployed as

@crichardson

Examples


Great isolation

Great manageability

Container encapsulates implementation technology

Efficient resource utilization

Fast deployment

Drawbacks

Immature infrastructure for deploying containers

@crichardson

Agenda


Core patterns

Deployment patterns


@crichardson

Communication issues

System Client

Service A

Service B

Service C

The SystemHow do clients of the

system interact with the services?

How do services within the system

interact?

@crichardson

@crichardson

How do clients of the system interact with the services?

@crichardson

Forces

Mismatch between the fine-grained microservices and needs of the clients

Different clients need different data

LAN vs. WAN vs. Mobile network performance

The number of service instances and their locations (host+port) is changes dynamically

Partitioning into services can change over time and should be hidden from clients

@crichardson

Directly connecting the front-end to the backend

Model

View ControllerProduct Info

service


Review service

REST

REST

Thrift

Model

View Controller

Browser/Native App

Traditional server-side web application

Chatty API

Web unfriendly protocols

What’s the host/port??

@crichardson

Pattern: API gateway

Model

View ControllerProduct Info

service


Review service

REST

REST

Thrift

API Gateway

Model

View Controller

Browser/Native App

Single entry point

Client specific APIs

Protocol translation

Traditional server-side web application

@crichardson

Example: Netflix API

http://techblog.netflix.com/2013/01/optimizing-netflix-api.html

Device specific end points

@crichardson

Benefits of the API gatewayInsulates the clients from the partitioning

Insulates the clients from the problem of discovery

Provides the optimal API for each client

Reduces the number of requests/roundtrips

Simplifies the client by moving logic for calling multiple services from the client to API gateway

Gateway translates between web-unfriendly protocols and HTTP/WebSockets

@crichardson

Drawbacks of the API gateway

Increased complexity - the API gateway is yet another highly available component that must be developed, deployed and managed

Increased response time due to the additional network hop through the API gateway

@crichardson

Benefits and drawbacks of messaging

Benefits

Decouples client from services

Message broker buffers messages

Supports a variety of communication patterns

Drawbacks

Additional complexity of message broker

Request/reply-style communication is more complex

Client needs to discover location of message broker

Benefits and drawbacks of RPC

Benefits

Simple and familiar

Request/reply is easy

No intermediate broker

Drawbacks

Only supports request/reply

Service must be available

Client needs to discover locations of service instances

@crichardson

@crichardson

The problem of discovery

Service Client

Service Instance A

Service Instance B

Service Instance C

?10.4.3.1:8756

10.4.3.99:4545

10.4.3.20:333

Client or API gateway

Dynamically changing

Service Instance A

Service Instance B

Service Instance C

10.4.3.1:8756

10.4.3.99:4545

10.4.3.20:333

How to load balance?

Dynamically assigned

@crichardson

Pattern: Client-side discovery

Service Client

Registry-aware HTTP Client

Service Registry

Service Instance A

Service Instance B

Service Instance C

10.4.3.1:8756

10.4.3.99:4545

10.4.3.20:333

registerquery

request

load balance

@crichardson

Example: Netflix Eureka and Ribbon

https://github.com/Netflix/eureka/wiki/Eureka-at-a-glancehttps://github.com/Netflix/ribbon


Flexible, application-specific load balancing

Fewer network hops and moving parts compared to Server-side discovery

Drawbacks

Couples the client to the Service Registry

Need implement client-side discovery and load balancing logic in multiple languages/frameworks

Service Registry is yet another moving part to setup and operate - highly available

@crichardson

Pattern: Server-side discovery

Service Client

Router

Service Registry

Service Instance A

Service Instance B

Service Instance C

10.4.3.1:8756

10.4.3.99:4545

10.4.3.20:333

registerquery

requestrequest

load balance

@crichardson

Example

http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/vpc-loadbalancer-types.html

Internal ELB Internal

ELB

Public ELB


Simpler client code

Built-in to some cloud/container environments, e.g. AWS ELB, Kubernetes, Marathon

Drawbacks

Limited to (generic) load balancing algorithms provided by router

More network hops

Router and Service Registry is yet another moving part to setup and operate - highly available

@crichardson

The problem of registration

Service Registry

Service Instance A

Service Instance B

Service Instance C

10.4.3.1:8756

10.4.3.99:4545

10.4.3.20:333

register

How does registration happen?

@crichardson

@crichardson

Forces

Service instances must be

registered with the service registry on startup

unregistered on shutdown

Service instances that crash must be unregistered from the service registry

Service instances that are running but incapable of handling requests must be unregistered from the service registry

@crichardson

Pattern: Self registration

Service Registry

Service Instance A

10.4.3.1:8756

registerheart-beatunregister

@crichardson

ExamplesNetflix Eureka Java client

API for registering/unregistering with Eureka server

Specify state: STARTING, UP, DOWN, OUT_OF_SERVICE

Register a periodically invoked health check callback

Zookeeper-based Java applications

Service = znode

Service instance = ephemeral child of znode


Simple - no extra components

Service can implement a more sophisticated state model

Drawbacks

Couples the service to service registry

Must implement registration logic in multiple languages/frameworks

Service might lack the self-awareness to unregister itself

@crichardson

Pattern: 3rd party registration

Service Registry

Service Instance A

10.4.3.1:8756

registerheart-beatunregister

Registrarhealthcheck

@crichardson

ExamplesAWS Autoscaling groups

Automatically register/unregister EC2 instance with ELB

Registrator

Registers and unregisters Docker containers with service registry

Kubernetes/Marathon

Automatically register/unregister services

Netflix Eureka Prana

Sidecar application for non-Java clients

Registers/unregisters service with Eureka

Performs health checks


Simpler service

Registrar can perform health checks

Some cloud/container environments provide a Registrar

Drawbacks

Registrar is yet another component that must be setup/operated. Must be highly available

@crichardson

Summary: Patterns and pattern languages are a great way to …

Think about technology

Discuss technology

Apply technology

@crichardson

Summary: The Microservices pattern language is a great way to …

Think about microservices

Discuss microservices

Apply microservices (or not)

@crichardson

@crichardson [email protected]

http://plainoldobjects.com http://microservices.io

Decompose that WAR? A pattern language for microservices (@QCON @QCONSP)

Software