Best Practices for Identifying Web Performance Issues Before Your Customers Do A Gomez/Dollar Thrifty Web Performance Testing Case Study: Imad Mouline, CTO, Gomez Emanuel Daniele, Sr. Solution Engineer, Gomez Jim Arrowood, DTAG, Director, Web Development and Architecture
36
Embed
Best Practices for Identifying Web Performance Issues Before Your Customers Do- A Gomez/Dollar Thrifty Web Performance Testing Case Study
Dollar Thrifty Automotive Group, Inc. (DTG) customers are increasingly choosing the Internet as the primary way to rent vehicles from the Dollar Rent A Car and Thrifty Car Rental Web sites. Recently DTG undertook a significant redesign initiative for its two Web sites to optimize customer experience ahead of its busiest summer travel season and used Gomez’s web load and performance testing solution to validate their efforts. Attendees of this hands-on Webinar will see a Gomez Reality Load product demonstration and learn about the steps DTG took to validate peak performance for all internal and external components including Content Delivery Networks (CDNs), ads, analytics and ecommerce platforms, delivered across the Internet to its customers’ browsers. This Webinar will cover:
•How Dollar Thrifty geared up for their peak summer season. •How a new style of load testing enables organizations to “walk in their customer’s shoes” and find problems before end-users find them. •Best practices for identifying and resolving Web performance issues across the entire Web application delivery chain, inside and outside the firewall. •Testing approaches that don’t require costly hardware or software investments. •How to uncover geographical response time discrepancies that may surface under load.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Best Practices for Identifying Web Performance Issues Before Your Customers Do
A Gomez/Dollar Thrifty Web Performance Testing Case Study:
Imad Mouline, CTO, GomezEmanuel Daniele, Sr. Solution Engineer, GomezJim Arrowood, DTAG, Director, Web Development and Architecture
Agenda
Reality Load Overview – Imad Mouline
Dollar Thrifty Case Study – Jim Arrowood
Product Demo – Emanuel Daniele
Internet
Content DeliveryNetworks
MajorISP
Local ISP
Mobile Carrier
3rd Party/Cloud Services
Browsers and devices Users
The Challenge of Delivering Quality Web Experiences
Storage
Load Balancing
Web Servers
App Servers
DB Servers
Mobile Components
Storage
Load Balancing
Web Servers
App Servers
DB Servers
Mobile Components
Users
The Web Application Delivery Chain
Traditional zone of control
Traditional testing: “OK” …user is
NOT happy
Internet
Content DeliveryNetworks
MajorISP
Local ISP
Mobile Carrier
Storage
Load Balancing
Web Servers
App Servers
DB Servers
Mobile Components
Users3rd Party/
Cloud ServicesBrowsers
and devices
The Web Application Delivery Chain
• Network peering problems
• Outages
• Inconsistent geo performance• Bad performance under load• Blocking content delivery• Incorrect geo-targeted content
SaaS with no investment or maintenance costs and rapid paybackSelf-service, full turnkey solution, or tailored to meet your needs with Gomez Professional Services offerings
Tests from an “Outside-in” customer point of view, with drill down to all web application components
Full desktop browser testing across globally distributed geographies
Expanded Load Generation
• Additional Geographies for Based Load high volume testing
• More Last Mile peer populations
Simplified Test Provisioning
• More easily schedule complex test scenarios
• Save and re-use test scenarios
• Last Mile UI scheduling improvements
Enhanced Reporting
• Detailed reporting for complex scenarios
• Filter results by load origin or time
• Streamlined data and chart exports
New in July 2009
Dollar Thrifty Case Study
Jim Arrowood, DTAGDirector -Web Development and Architecture
8
Who is Dollar Thrifty?
• Driven by the mission “Value Every Time,” the Company’s brands, Dollar Rent A Car and Thrifty Car Rental, serve value-conscious travelers in over 70 countries.
• Dollar and Thrifty have over 600 corporate and franchised locations in the United States and Canada, operating in virtually all of the top U.S. and Canadian airport markets.
• The Company’s approximately 6,400 employees are located mainly in North America, but global service capabilities exist through an expanding international franchise network.
9
Dollar and Thrifty.com
10
The Dollar Thrifty Environment
Dollar.comMiddle-Tier
Thrifty.comMiddle-Tier
Rate EngineMiddle Tier
Reservation Middle Tier
Database
11
The Problem
• In a highly, horizontally scaled environment, development, test, and staging environments rarely 100% match production
• Dollar Thrifty significantly redesigned both of our websites in 2008 leaving question as to how the sites will perform in the peak of the 2009 summer
• Internal load tests using traditional methods in test and staging environments did not provide the needed confidence
12
The Gomez Relationship
• Dollar Thrifty has utilized the Gomez ActiveXF platform for several years– Synthetic Tests of Key Business Processes
• Monitors for response time (Home Page Load & Reservation Process)
• Monitors for successful execution (Home Page & Reservation Process)
13
Gomez Reality Load
• In Early 2009, Dollar Thrifty teamed with Gomez to launch the first external load test of the company’s infrastructure and key software platforms utilizing Gomez Reality Load
14
The Goals
• Regardless of how much load, where is our weakest point?
• Can we handle our previous year’s peak +25% traffic?
15
Identify Core Business Processes
• How do the vast majority of the consumers interact with the site?
• Use 80-20 rule– Shop for Rates– Make Reservation– Modify Reservation– Cancel Reservation
80%
20%
Core Business ProcessesOther Processes
16
Distribute Core Business Processes
• By percentage, how do consumers interact with the site?– Shop for Rates – 70%*– Make Reservation – 15%*– Modify Reservation – 10%*– Cancel Reservation – 5%*
70%
15%
10%5%
Shop for RatesMake ReservationModify ReservationCancel Reservation
*Hypothetical 17
Forecast Visitors
• Based on historical web analytic data and the goal of the test, compute the intended visitor capacity of the site within an hour– 10,000 visitors per hour at peak historically*– If we intend to support peak +25% then our visitor
forecast is 12,500 visitors
*Hypothetical
0
2000
4000
6000
8000
10000
12000
Jan Feb Mar-Apr May-Jun Jul-Aug Sep-Oct Nov-Dec
Vis
ito
rs P
er
Ho
ur
18
Identify the Needed Scripts
• Throughput Tests – Where are we weakest?– One script per core business process with no
“think-time”• Capacity / Load Tests – How much load can we
legitimately handle?– One script per core business process with “think-
time”
19
Develop Needed Scripts
• Gomez Script Recorder allows the client to create of the script, inject look-up data, and replay the script
• RealityLoad utilizes the same scripting engine as ActiveXF
• The base of our scripts existed as we already had them built to execute in the ActiveXF platform– Added logic to pull from look-up data – Added think time
• Once, complete the tests are loaded into the Gomez portal
20
Configure Tests
• Using the Gomez portal, tests are configured and scheduled for execution
21
Execute
• Execute tests at off peak time – We chose 2 AM to 4 AM CST
• Establish war room with key system owners• Establish bridge line with key infrastructure and
monitoring personnel• As tests are executing ensure all internal monitoring
tools in all tiers of the application are capturing as much detail as possible without skewing results– CPU, Memory, Disk IO, Network Throughput, etc.
22
Analyze Results
• Each system owner and infrastructure owner collects key metrics and findings and submits to the test leader
23
Discuss Key Findings
• Thrifty.com front-end servers ran hot while the middle-tier was cold and overpowered
• Dollar.com front-end servers ran cool while the middle-tier was running on target
• Discovered excessive calls for validation to legacy reservation system
• Thrifty.com middle-tier had sticky sessions enabled causing load balancing to become less ineffective
24
Make Corrective Actions & Retest
• Mitigate the discovered issues and risks– We were able make a simple configuration change to
resolve the “sticky sessions” issue– We were able to move Thrifty middle-tier boxes into
the Thrifty.com front-end and Dollar.com middle-tier to better distribute our load with no additional expense
• When time and budget allows, retest to ensure mitigating actions resolve the issue and ensure no additional bottlenecks have appeared