Scaling Magento for the Global Market Kian Gould Founder & CEO
Scaling Magento for the Global Market
Kian Gould Founder & CEO
Angry Birds
Angry Birds
Angry Birds
Angry Birds
DRM Server
Mail Service
Giftcards
Braintree
Warehouse Inventory
3 Storefronts (US, EU, ASIA)
Custom Reporting
Splitshipping
Warehouse / Inventory
Management Fullfillment
Banner Management
Custom Tax, Custom Shipping
Catalog Rules
…
The Angry Birds Magento Universe
High Level Requirements
• Innovative, highly-scalable e-commerce systems
• High availability
• Unusually high peaks (20x and more)
• High Speed
• Backend server independent from frontend servers
• No down-times, ever.
Solution components
Cloud Services
RightScale Services
Amazon AWS Console
Cloud- Front
Elasti-Cache
S3 EC2 RDS ELB Route 53
Management
Deployments
Auto-Scaling
Monitoring
Logging
Scripting API
Magento in the Cloud
Share nothing?
Code
Cache Session
Storage
Assets Product images,
CMS images,
Scaled images
Database
JS/CSS
Bundles
Server
S3 Assets Storage
Varnish Array EC2
Frontend Array
CloudFront CDN for Skin
RDS MySQL
S3 Deployment
ElastiCache Cache Backend
ELB Load Balancer
CloudFront CDN for Assets
Monitoring
Backend Array EC2 EC2
EC2
Worker Array EC2
Braintree
Mail DRM
Giftcards
Logging
EC2
Route 53 DNS-Service
shop.angrybirds.com
S3 Deployment
Route 53 DNS-Service
shop.angrybirds.com
RDS MySQL
Varnish Array EC2
Frontend Array Backend Array EC2 EC2
EC2
Worker Array EC2 EC2
Deployment X
ELB Load Balancer
Deployment X+1 Release Y
S3 Deployment
Route 53 DNS-Service
shop.angrybirds.com
Varnish Array EC2
Frontend Array
ELB Load Balancer
Backend Array EC2 EC2
EC2
Worker Array EC2 EC2
Deployment X
Varnish Array EC2
Frontend Array Backend Array EC2 Worker Array EC2
ELB Load Balancer
EC2
EC2 EC2 EC2
Cache
warming
Direct Access
through
Local Host entry
RDS MySQL
Deployment X+1 Release Y
Varnish Array EC2
Frontend Array
ELB Load Balancer
Backend Array EC2 EC2
EC2
Worker Array EC2 EC2
Deployment X
Varnish Array EC2
Frontend Array Backend Array EC2 Worker Array EC2
ELB Load Balancer
EC2
EC2 EC2 EC2
Deployment / Rollback
Route 53 DNS-Service
shop.angrybirds.com
Downtime during the deployment:
All Basket Content, active Logins and Sessions are carried through each
Deployment / Rollback
Magento, faster!
Magento Frontend Caching
Areas of optimization
Frontend-Optimization
JavaScript / CSS
• Bundling (Magento)
• Minifying (with YUI Compressor)
• Gzip-Compression (Apache)
• Timestamp in filename
• Send Cache-Header
CDN (Cloudfront)
• Product images (+ scaled versions), CMS Images etc.,...
• Skins
• JS/CSS
Aoe_JsCssTstamp
Magento-Optimizations
Reduce Logging as much as possible
Deactivate non-cacheable features wherever possible
Caching
• Asynchronous Cache Clearing Queue
• Delete old Cache-entries
Cookie Storage for Username and Basket content
Ajax-Basket
and... Profiling, Profiling, Profiling,… (xdebug + kcachegrind, Newrelic, xhprof)
Aoe_AsyncCache
Aoe_CacheCleaner
95% of all hits are cacheable!
5% dynamic content
Homepage
Categories
Products
CMS
Basket
Checkout
Profile
Caching
Varnish Reverse Proxy
Magento
Caching
Solutions
Aoe_Static
Caching +
Handling of
dynamic Content
Magneto_Varnish
Purging
Lessons learned
• 404s are very expensive => cache or redirect
• Cloudfront with S3 Backend doesn’t support gzip => deploy time bundeling/compression
• Don’t run reports on live database!
• You can’t plan for everything => but you can be ready to solve anything quickly. How?
=> continuous deployment
Continuous Integration und Deployment
Don’t be afraid of deploying to Live!
Magento Installationpackage
Database
tables
Files*
‣htdocs [Magento Source]
‣.modman [Custom Packages]
Installer
Settings
‣URLs
‣DB-Parameters
‣…
Files
‣media
ChangeLog
System-
storage SVN
GIT
Settings
Performance Tests
Acceptance Tests
Unit-Tests
PHPUnit
Installation on
„Latest“ System
Build
Deployment Pipeline
Cloud Deployment
Production
Integration Tests
AOE + Rovio
Cloud Deployment
Staging Copy to S3
Deployment Pipeline
Installation on
Deployment System
Deployment of new Release
Space game Launch
Array scales up
USA wakes up
Launch!
In real life…
visitors/day
(real) concurrent users!
orders per second at peak!
Thank You!
www.aoemedia.com [email protected]
AOE media Inc. 251 Rhode Island Street San Francisco, CA 94103 USA Tel.: +1 415 230-0690 Fax: +1 415 230-0699
@aoemedia