architecting for failure building fault-tolerant systems Jakub Derda Warsaw, 2015
4Developers 2015: Designing for failure - architecting fault-tolerant system - Jakub Derda
Jul 16, 2015
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
architecting for failure building fault-tolerant systems
Jakub Derda
Warsaw, 2015
‘Tree’ component – overview
‘Tree’ component – detailed view
‘Tree’ component – detailed view
client
network connection
sever
‘Tree’ component – detailed view
human factor software client library
ISP protocol stack network
load balancers OS power source
client
network connection
sever
Your component – detailed viewWhat is a fault?
What is not a fault?
Service is not working
on our side*
* Caused by e.g. technical failures, outages, corrupted data, attacks
What is a fault?
The real fault is when we don’t
deliver valueto customers.
Value delivering without working system
Bring your own wine, we’re waiting for license.Last election in Poland
What fault-tolerance is not?
It’s NOT making sure your system
never goes down.
It (eventually) will.
What is a fault-tolerance?
It’s making sure that system can
quickly recover and/or
client is not impacted.
How to solve it?
Solving – redundancy
Hot/warm replicas
Caches
Geographical distribution, CDNs
Hardware redundancy
Alternative systems and procedures
Solving – design
Stateless
Auditing
Idempotent requests
Uniqueness / randomness
Asynchronous and decoupling
EIPs
Commands, not data
Break the rules
Solving – procedures
Backup creation, cleanup and restore
QA & potential problems
Continuous integration
Deployment
Solving – observe
Dive deep, post-mortems
Identify bottlenecks
Observe key metrics
Verify assumptions
Predict traffic
Tradeoffs - simple
1/scope
QUALITY
Tradeoffs - real
cost
durability
time
consistency
trust
audit (traceability)
complexity
security
scalability
functionalitystability
reliability
extensibility
performance
maintainability
manageability
Summary
Learn to live with
crashes
Summary
Automate
procedures
Summary
Don’t be afraid to
cross the line
Fault tolerance is not a
property of a design,
it’s a process.
Related Documents
