Patterns & Practices of Microservices

PATTERNS & PRACTICES

MICROSERVICES

Proxy / Load Balancer

Web Server

Database Server

Application Server

Web Server

~4 YEARS


Web Server

Database Server

Application Server

Web Server Web Server

Application Server

~3 YEARS


Web Server

Database Server

Application Server


Application Server


Web Server

Application Server


Application Server

Hot

Cold

Database Server

~2 YEARS


Web Server

Database Server

Application Server


Application Server


Web Server

Application Server


Application Server

Hot

Cold

GLB / DNS

Reporting Database Server

~2 YEARS


Web Server

Database Server

Application Server


Application Server


Web Server

Application Server


Application Server


Hot

Hot

Database Server

GLB / DNS

~2 YEARS

X18

x5

~2 YEARS

~2 YEARS

x2

2 YEARS

W E S L E Y R E I S Z

San Francisco / London / New York

CHAIR QCON

“Loosely coupled Service Oriented Architecture with Bounded Contexts”

-Adrian Cockcroft, Microservices Thoughtleader

IF EVERY SERVICE HAS TO BE

UPDATED AT THE SAME TIME

IT’S NOT LOOSELY COUPLED

IF YOU HAVE TO KNOW TOO

MUCH ABOUT SURROUNDING

SERVICES YOU DON’T HAVE A

BOUNDED CONTEXT. SEE THE

DOMAIN DRIVEN DESIGN BOOK

BY ERIC EVANS.

Why Microservices?

CONS

PROS

• Monitoring/Logging becomes more important than ever.

• You trade code simplicity for operational complexity.

• Easy to chase the latest shiny technology / mix new languages & tech in.

• Another dimension to scale the application.

• Easier to have different services usedata persistence most appropriate for the use case.

• Easier to reason about moving applications to the cloud.

• Easier to have a developer understandthe portion of the system and contribute

• Easier to test & deploy a single service.

• Cascading failures can cause problems.

• Orchestration can become complex

FIRST PRINCIPLES DevOps Culture,

Evolutionary Architecture, Scale Cube & CAP Theorem

MICROSERVICESPATTERNS & PRACTICES

PATTERNS & PRACTICESTechnical, Operational, Cultural

DEVOPS CULTURE IS A MUST

• Rapid provisioning

• Basic monitoring

• Rapid Application Deployment

http://martinfowler.com/bliki/MicroservicePrerequisites.html

“Signing up for Microservices is signing up for evolutionary architecture. There’s no point where you’re just done.” -Josh Evans, Director of Operations Engineering @Netflix


Web Server

Database Server

Application Server


Application Server


Web Server

Application Server


Application Server


Hot

Hot

Database Server

GLB / DNS

https://www.infoq.com/presentations/netflix-failure-multiple-regions


SCALE CUBE

X axis - horizontal Scaling (Scale by Cloning)

Z ax

is - D

ata P

artiti

oning

(Sca

le by

Sha

rding

)Y axis - Functional Decomposition

(Scale by Splitting)

The CAP theorem is the idea that a distributed computing system is not able to provide partition tolerance, consistency and availability at

the same time.


AVAILABILITY/PARTITION TOLERANCE

CONSISTENCY/PARTITION TOLERANCE CP: Choose Consistency over Availability when your business

requirements dictate atomic reads and writes. Wait for a response from the partitioned node which could result in a timeout error. The

system can also choose to return an error, depending on the scenario you desire.

AP: Choose Availability over Consistency when your business requirements allow for some flexibility around when the data in the

system synchronizes. Return the most recent version of the data you have, which could be stale. This system state will also accept writes that can be processed later when the partition is resolved. Availability is also

a compelling option when the system needs to continue to function in spite of external errors (shopping carts, etc.)


Web Server

Database Server

Application Server


Application Server


Web Server

Application Server


Application Server


Hot

Hot

Database Server

GLB / DNS

Evolu&onary Architecture

Scale Cube

CAP Theorem

Microservices is a architecture that conFnues to change

and adapt

Scale Cube consists of an X, Y and Z axes – each

addressing a different approach to scale a service.

The idea that a distributed compuFng system is not able

to provide parFFon tolerance, consistency and availability

at the same Fme.

FIRST PRINCIPLESThings we need to

understand about as we plan/scale systems

DevOps is a MustMicroservices is the first architecture post devops culture

shiP

TECHNICAL Tools, Event Sourcing, CQRS,

Circuit Breaker, Retrys

OPERATIONALObservability, Containers, Polyglot

CULTURALDevOps Culture, Conway’s Law


123

TOOLS

“Everything is a tradeoff just be intentional about them”

SPINE MODEL

• Needs: We are here to satisfy ________

• Values before PrinciplesWe optimize for ________

• Principles before Practices We leverage _____ to change the system

• Practices before tools We do _____ to create value

• We have always been Tool users and makers. Tools should answer the question: We use ____ to get our work done.

http://spinemodel.info/explanation/introduction/


Web Server

Database Server

Application Server


Application Server


Web Server

Application Server


Application Server


Hot

Hot

Database Server

GLB / DNS

Hot

Hot

GLB / DNS

User Facing AvailabilityConcerns

Internal Consistency Concerns

User Facing AvailabilityConcerns

Internal Consistency Concerns

Hot

Hot

GLB / DNS

High Reads

High Writes

High Reads

High Writes

Hot

Hot

GLB / DNS

High Reads

High Writes

High Reads

High Writes

https://www.infoq.com/presentations/wix-microservices-devops

EVENT SOURCING

“Yes, I think we should talk about this again.”

Student Enroll In Workshop Security Module Student Module Workshop Module

reserveSeat() isAuthorized()

isEligible()

isAvailable()

updateSchedule()

updateSchedule()

ALL SYNCHRONOUS

MUTIABLE STATE!

The fundamental idea of Event Sourcing is that of ensuring every change to the state of an application is captured in an event object, and that these event objects are themselves stored in the sequence they were applied for the same lifetime as the application

state itself.- M Fowler

Student Enroll In Workshop Security Module Student Module Workshop ModuleQueue/Topic

subscribe()subscribe()

subscribe()

reserveSeat() isAuthorized()

reserveSeat()

ASYNCHRONOUS

IMMUTIABLE STATE!

updateSchedule()

updateSchedule()

Par

Immutable Events

Recreate the exact state

Increase in complexity

Performance / Load tes&ng

Provides a model to think about scalability

Because you have all of the state changes, you have all of

the changes that affected state

No longer doing CRUD. You have calculate the result.

Leverage things like snapshots

You an reply the events across the Enterprise

EVENT SOURCINGThe fundamental idea of Event

Sourcing is that of ensuring every change to the state of an

application is captured in an event object, and that these

event objects are themselves stored in the sequence they

were applied for the same lifetime as the application state

itself.

CQRS

“Decouple queries from Commands to scale each independently.”

Presentation

Business Logic

DAO Data Storage

Updates / Queries

Command and Query Responsibility Segregation (CQRS) is a pattern that segregates the operations

that read data (Queries) from the operations that update data (Commands) by using separate

interfaces.

Presentation

Commands

Data Persistance

Data Storage

Updates (Writes)Domain Logic

Queries (Reads)

Presentation

Presentation

Commands

Data Persistance

Data Storage

Updates (Writes)Domain Logic

Queries (Reads)

Presentation

Presentation

Presentation

Presentation

Commands

Data Persistance

Data Storage

Updates (Writes)

Domain Logic

Queries (Reads)

Presentation

Presentation

Presentation

Presentation

Presentation

Presentation

Read Data StoreWrite Data Store

EVENTUAL

CONSISTENCY?

Domain Specific Models

Scale

Tradeoff

Performance

Clean Domain models queried

Scale the reads above the writes

Increased the operaFonal complexity of the data stores

Tune different datastore for different performance needs

CQRSCommand-Query-Responsibility-

Separation (CQRS) separates query processing from command

processing, which updates entities using business logic that's

commonly implemented using event sourcing.

SecurityUpdates to the model controlled different from the reads

CIRCUIT BREAKER

“Decouple queries from Commands to scale each independently.”

Service A called B.

TRIPPED

Service B is Sick, let’s flag is and

try back in 5min

Service C

Different Instance of Service B

Service B is sick. Don’t send traffic there for 5

minute (Tripped)

Service A called B.

OPEN

Service B is still sick

Service C

Different Instance of Service B

Service B is still sick.

Isolate it and remove from

ELB

Service responds normally.

Service C

After 5 minutes. Traffic back to service B.

Closed

set the flag to closed. Nothing

else to do.service B

responded correctly,

Prevents Doomed to Fail

Three states

Prevents Cascading Failures

purpose is to detect external service failures and prevent

the applicaFon from trying to perform an acFon that is

doomed to fail.

Closed (Normal), Open(Tripped), & Half-Open state

Helps prevent cascading failures and improves resiliency.

CIRCUIT BREAKER

Circuit breaker is used to detect failures and

encapsulates logic of preventing a failure to reoccur

constantly (during maintenance, temporary external system failure or

unexpected system difficulties). Part of Hysterix

Part of Ne\lix OSS library for latency & fault tolerance

RETRY

“Upstream timeout must always be longer than total downstream

timeout * retries delay”-Adrian Cockcroft

Bad Service

Good Service

Edge Service

3s

1s 1s

Fails fast @ 2s

“Upstream timeout must always be longer than total downstream timeout * retries delay” -Adrian Cockcroft

Budgeted timeout, one retry

Tools

Resiliency PaIerns

Architectural PaIerns to Consider

Don’t start with the tool.

Circuit Breaker, Retry

Event Sourcing & CQRS

TECHNICAL

These are some interesting technical considerations

with Microservices

TECHNICAL Getting started, Event Sourcing, CQRS


CULTURALDevOps Culture, Conway’s Law,


123

OBSERVABILITY

In control theory, observability is a measure for how well internal states of a system can be inferred by knowledge of

its external outputs.

At a minimum setup log stash, look at going an extra step to setup

ELK.

OBSERVABILITY: THINGS TO THINK ABOUT

LOGS

• Need consistent, structured logging

• Logs are not for humans

• Logstash, Elasticsearch, Kibania works great


PROTOCOLS/TRANSPORT

RPC/HTTP vs

JSON vs Binary

RPC vs HTTP

JSON vs Binary

• Speed/Performance vs Readability

• Measure serialization, transmission, deserialization costs


How do you Recreate a failure in a distributed resilient

environment?

TRACING

Look into:

• Zipkin

• Prometheus

Best way to understand Fanout

https://www.infoq.com/presentations/uber-scalability-services

CONTAINERS

“What do you feel is the most disruptive (or at least potentially disruptive technology)?”

POLYGLOT

More challenges than they thought…

Observability

Polyglot

Containers

Logging, Service CommunicaFon, Tracking

Containers makes it easier, but observability, logging,

tracking can be more challenging

Low fricFon deployment, Finer-grained execuFon

environments, speed, isolaFon

OPERATIONAL

Microservices simplifies code. It trades code

complexity for operational complexity

TECHNICAL Getting started, Event Sourcing, CQRS


CULTURALDevOps Culture, Conway’s Law


123

CONWAY'S LAW

“Organizations which design systems ... are constrained to produce designs which are

copies of the communication structures of these organizations”

-Conway

Model

One size does not fit all

Tradeoff

Organiza&onal Lifecycle Stages

Provides a model to think about scalability

Microservices (funcFon decomposiFon) is AN approach

Each approach has different tradeoffs (For example: as

we’ll find with Y-axis scaling, you may trade code

complexity for organizaFonal complexity)

Different needs at different points of an org’s lifecycle

CONWAY’S LAW

Organizations which design systems ... are constrained to produce designs which

are copies of the communication structures of

these organizations

So what do I wish I knew before starting the push to Microservices?


Web Server

Database Server

Application Server


Application Server


Web Server

Application Server


Application Server


Hot

Hot

Database Server

GLB / DNS

Document Repo

Recruiting

Processing

Boards

Applicants

Recruiting

Processing

Document Repo

Applicants

Applicants

API Gateway

API Gateway

Applicants

Boards



FIRST PRINCIPLES


• Have capabilities in your org to do things like rapidly provision, monitor, and use rapid deployment. (DevOPs Culture is a must)

• Accept that the service going to evolve (Evolutionary Architecture)

• Handle their own data needs (Consistency/Availablity Tradeoffs)

• Be able to scale based on it’s needs (Scale Cube)

Processing

DevOps Culture, Evolutionary Architecture,

Scale Cube & CAP Theorem

TECHNICAL Spine Model

Event Sourcing/CQRS Circuit Breaker

Retry

OPERATIONALObservability

Containers Polyglot

CULTURALConway’s Law

MICROSERVICESPATTERNS & PRACTICES Processing

• Have a decision process(Spine Model)

• Consider using events where it makes sense and how you might scale commands & queries(Event Sourcing / CQRS)

• Protect against cascading failures(Circuit Breaker, Bulkhead)

• “Everything is a tradeoff, do it intentionally” Matt Ranney (RPC/HTTP, Containers, Polyglot)

• Bootstrap services with organizational capabilities (Logs, Tracing)

• Beware of Conway’s Law

CONS

PROS

• Monitoring/Logging becomes more important than ever.

• You trade code complexity for operational complexity.

• Easy to chase the latest shiny technology / mix new languages & tech in.

• Another dimension to scale the application.

• Easier to have different services usedata persistence most appropriate for the use case.

• Easier to reason about moving applications to the cloud.

• Easier to have a developer understandthe portion of the system and contribute

• Easier to test & deploy a single service.

• Cascading failures can cause problems.

• Orchestration can become complex

“If you don’t end up regretting your early technology decisions, you probably over engineered.” - Randy Shoup, Engineering Director @StitchFix


Wesley Reisz@wesreisz

[email protected]

Patterns & Practices of Microservices

Technology