Scaling Railsa system-wide approach to performance optimization
February 2010
FEUP
Exploratory Case Study: http://escolinhas.pt
< Problem >
Scaling Rails: a system-wide approach to performance optimization2
Web 2.0
Scaling Rails: a system-wide approach to performance optimization3
Better web
Better tools
Scaling Rails: a system-wide approach to performance optimization4
Ruby on Rails
Scaling Rails: a system-wide approach to performance optimization5
Popularity brings USERS
USERS need scalability
Scaling Rails: a system-wide approach to performance optimization6
Rails doesn’t scale
Scaling Rails: a system-wide approach to performance optimization7
Scaling Rails: a system-wide approach to performance optimization8
Scaling Rails: a system-wide approach to performance optimization9
Twitter abandoning Rails?
Scaling Rails: a system-wide approach to performance optimization10
“FWIW: Twitter currently has no plans to
abandon RoR. Lots of our code is not in
RoR, already, though. Maybe that's why
people are confused.”
Scaling Rails: a system-wide approach to performance optimization11
- Evan Williams, Twitter CEO
YellowPages
Scaling Rails: a system-wide approach to performance optimization12
More users?
“Just throw hardware at it.”
Scaling Rails: a system-wide approach to performance optimization13
Rails doesn’t scale
Scaling Rails: a system-wide approach to performance optimization14
Hulu
Scaling Rails: a system-wide approach to performance optimization15
Scribd
Scaling Rails: a system-wide approach to performance optimization16
They scaled
They gave keynotes on
HOW
Scaling Rails: a system-wide approach to performance optimization17
What’s missing here?
Scaling Rails: a system-wide approach to performance optimization18
No general approaches to scaling
Scaling Rails: a system-wide approach to performance optimization19
Problem Overview
Scaling Rails: a system-wide approach to performance optimization20
Rails doesn’t scale?
YellowPages
Rails doesn’t scale?
Hulu
Scribd
General approaches to scaling?
Scaling Rails: a system-wide approach to performance optimization21
< Problem Overview >
< Motivation >
Scaling Rails: a system-wide approach to performance optimization22
WEB
Scaling Rails: a system-wide approach to performance optimization23
WEB
WEB
WEB
WEB
USERS
Scaling Rails: a system-wide approach to performance optimization24
USERS
USERS
USERS
USERS
IMPORTANCE
Scaling Rails: a system-wide approach to performance optimization25
IMPORTANCE
IMPORTANCE
IMPORTANCE
IMPORTANCE
Better web
Scaling Rails: a system-wide approach to performance optimization26
Better tools
Scaling Rails: a system-wide approach to performance optimization27
UX involves reasonable response
times and service uptime
Scaling Rails: a system-wide approach to performance optimization28
Most recent tools are great
Scaling Rails: a system-wide approach to performance optimization29
“Ruby on Rails is a breakthrough in lowering the
barriers of entry to programming. Powerful web
applications that formerly might have taken
weeks or months to develop can be produced in
a matter of days.”
Scaling Rails: a system-wide approach to performance optimization30
- Tim O'Reilly, Founder of O'Reilly Media
Everything can be improved
There is no perfect tool
There are always pitfalls
Scaling Rails: a system-wide approach to performance optimization31
What’s really motivating?
Scaling Rails: a system-wide approach to performance optimization32
Fixing pitfalls on great tools
Improving UX for everyone
Scaling Rails: a system-wide approach to performance optimization33
Motivation Overview
Scaling Rails: a system-wide approach to performance optimization34
Web keeps growing
More users, importance
Web 2.0
User Experience
Also dependant on:
Response times and availability
Recent tools are great
There are always pitfalls
Fixing pitfalls on great tools
Improving UX for everyone
Scaling Rails: a system-wide approach to performance optimization35
< Motivation Overview >
< Goals >
Scaling Rails: a system-wide approach to performance optimization36
What about scaling?
Scaling Rails: a system-wide approach to performance optimization37
Everything is easy
Scaling should also be easy
Scaling Rails: a system-wide approach to performance optimization38
Creation of
Scaling Rails: a system-wide approach to performance optimization39
Guidelines and conventions
for high performance
Rails applications
Scaling Rails: a system-wide approach to performance optimization40
Optimal Rails-oriented
configurations for every involved
component
Scaling Rails: a system-wide approach to performance optimization41
Patches to improve the
components performance from a
Rails-perspective
Scaling Rails: a system-wide approach to performance optimization42
Objectives Overview
Scaling Rails: a system-wide approach to performance optimization43
Scaling should be easy
Creation of:
Generic guidelines and conventions
Optimal configurations
Patches for all components
All from a performance-oriented philosophy centered
in Ruby on Rails!
Scaling Rails: a system-wide approach to performance optimization44
< Objectives Overview >
< State of the Art >
Scaling Rails: a system-wide approach to performance optimization45
< Operating Systems >
Scaling Rails: a system-wide approach to performance optimization46
Most used OS in servers: Linux
Scaling Rails: a system-wide approach to performance optimization47
BSD faster than Linux
Scaling Rails: a system-wide approach to performance optimization48
Linux faster than BSD
Scaling Rails: a system-wide approach to performance optimization49
Linux Ruby 1.8 2x faster
than Windows Ruby 1.8
Scaling Rails: a system-wide approach to performance optimization50
Linux Ruby 1.9 70% faster
than Windows Ruby 1.9
Scaling Rails: a system-wide approach to performance optimization51
Rails widely deployed in Linux
Scaling Rails: a system-wide approach to performance optimization52
< Ruby >
Scaling Rails: a system-wide approach to performance optimization53
Ruby 1.8 is the most widely used
Scaling Rails: a system-wide approach to performance optimization54
JRuby is the fastest implementation
Scaling Rails: a system-wide approach to performance optimization55
Ruby 1.8 will soon be outdated
Scaling Rails: a system-wide approach to performance optimization56
Ruby 1.9 will soon be outdated
Scaling Rails: a system-wide approach to performance optimization57
Ruby 1.9 has only one interpreter
Scaling Rails: a system-wide approach to performance optimization58
Ruby MRI 1.9
Scaling Rails: a system-wide approach to performance optimization59
Ruby MRI 1.9
Rails bottleneck: GC
Scaling Rails: a system-wide approach to performance optimization60
< Web servers >
Scaling Rails: a system-wide approach to performance optimization61
Webrick
Scaling Rails: a system-wide approach to performance optimization62
Webrick
Pioneer, slow
Scaling Rails: a system-wide approach to performance optimization63
Mongrel
Scaling Rails: a system-wide approach to performance optimization64
Mongrel
Much faster, excellent HTML parser
Scaling Rails: a system-wide approach to performance optimization65
Thin
Scaling Rails: a system-wide approach to performance optimization66
Thin
Mongrel’s HTML parser
Scaling Rails: a system-wide approach to performance optimization67
Thin
New I/O back-end: EventMachine
Scaling Rails: a system-wide approach to performance optimization68
Thin
Faster than those before
Scaling Rails: a system-wide approach to performance optimization69
Passenger
Scaling Rails: a system-wide approach to performance optimization70
Passenger
Not self-contained
Scaling Rails: a system-wide approach to performance optimization71
Passenger
Module for Apache/Nginx
Scaling Rails: a system-wide approach to performance optimization72
Passenger
Good performance, great at scaling
Scaling Rails: a system-wide approach to performance optimization73
Thin is the fastest
Passenger scales better
Scaling Rails: a system-wide approach to performance optimization74
< Database >
Scaling Rails: a system-wide approach to performance optimization75
MySQL
Scaling Rails: a system-wide approach to performance optimization76
PostgreSQL
Scaling Rails: a system-wide approach to performance optimization77
MySQL is faster and more popular
Scaling Rails: a system-wide approach to performance optimization78
Schema-less databases
Scaling Rails: a system-wide approach to performance optimization79
Schema-less databases
Said to have better performance than SQL
Scaling Rails: a system-wide approach to performance optimization80
MongoDB
Scaling Rails: a system-wide approach to performance optimization81
CouchDB
Scaling Rails: a system-wide approach to performance optimization82
Benchmarks are scarce
Scaling Rails: a system-wide approach to performance optimization83
Major improvement in all DB:
Caching, caching, caching!
Scaling Rails: a system-wide approach to performance optimization84
< Rails >
Scaling Rails: a system-wide approach to performance optimization85
Rails 2.3
Scaling Rails: a system-wide approach to performance optimization86
Many performance pitfalls
improved in Rails 3
Scaling Rails: a system-wide approach to performance optimization87
Rails 3
Beta release in February
Scaling Rails: a system-wide approach to performance optimization88
Immediate need: form helpers, i18n
Scaling Rails: a system-wide approach to performance optimization89
< Application >
Scaling Rails: a system-wide approach to performance optimization90
Porting to Ruby 1.9 (from 1.8)
Scaling Rails: a system-wide approach to performance optimization91
Porting to Rails 3 (from 2.3)
Scaling Rails: a system-wide approach to performance optimization92
Check code for common pitfalls
Eager loading
Fetching records in batches
Database Indexes
Caching
Scaling Rails: a system-wide approach to performance optimization93
< Methodology >
Scaling Rails: a system-wide approach to performance optimization94
State of the Art narrows options down
Scaling Rails: a system-wide approach to performance optimization95
Benchmarking
Controlled tests to choose the best
software for each component
Scaling Rails: a system-wide approach to performance optimization96
Tweaking
Try out different configurations and
architectures
Scaling Rails: a system-wide approach to performance optimization97
Testing
Measuring of results, finding bottlenecks
Scaling Rails: a system-wide approach to performance optimization98
Developing
Improving over bottlenecks
Scaling Rails: a system-wide approach to performance optimization99
Scaling Rails: a system-wide approach to performance optimization100
Benchmarking
Tweaking
Testing
Developing
and
Improving
Scaling Rails: a system-wide approach to performance optimization101
Recipe: repeat for all components
Scaling Rails: a system-wide approach to performance optimization102
Rails perspective!
Scaling Rails: a system-wide approach to performance optimization103
Escolinhas
Scaling Rails: a system-wide approach to performance optimization104
Scaling Rails: a system-wide approach to performance optimization105
Butterfly Effect
Scaling Rails: a system-wide approach to performance optimization106
Scaling Rails: a system-wide approach to performance optimization107
Consider sensitive dependencies
< Work schedule >
Scaling Rails: a system-wide approach to performance optimization108
Scaling Rails: a system-wide approach to performance optimization109
Scaling Rails: a system-wide approach to performance optimization110
Activity Start Date Duration (weeks)
OS testing and choosing 15/02/2010 1
OS kernel testing and tuning 22/02/2010 1
Web server testing and bottleneck research 01/03/2010 1
Web server bottleneck improvements 08/03/2010 1
Ruby interpreter testing and bottleneck research 15/03/2010 1
Ruby interpreter bottleneck improvements 22/03/2010 1.5
DB testing and bottleneck research 31/03/2010 1
DB bottleneck improvements 07/04/2010 1
Rails bottleneck research 14/04/2010 1
Rails bottleneck improvements 21/04/2010 2.5
Escolinhas analysis and bottleneck research 10/05/2010 1
Escolinhas improvements 07/05/2010 3
Writing of dissertation report 07/06/2010 4
< Thank you >
http://snaprails.tumblr.com
Gonçalo S. SilvaAdvisor: Ademar Aguiar
http://goncalossilva.comhttp://twitter.com/goncalossilva