Horizon 3 Shell Horizon 3 Performance Testing GFN Performance Test Volumetrics Customer :Shell Contract number : 11/0330/GLPL Business/project number : 4014/EC231690 Project Manager : Robert McMurdo Project/document reference : H3.15.03.01.02 Issue :0.1 Issue date : 21/08/2012 Status : Draft Distribution : Prepared by : Max Hunt Performance Testing Consultant Reviewed by : Robert McMurdo Performance Testing Team Lead Date: John Quinn Technical Architect Date: Checked by : Robert McMurdo Performance Testing Team Lead Date: Approved (Logica) : Authorised (Logica) : document.docx page 1 of 16
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Horizon 3
Shell Horizon 3 Performance Testing
GFN Performance Test Volumetrics
Customer : ShellContract number : 11/0330/GLPLBusiness/project number : 4014/EC231690Project Manager : Robert McMurdoProject/document reference : H3.15.03.01.02Issue : 0.1Issue date : 20/08/2012
Status : DraftDistribution :
Prepared by : Max HuntPerformance Testing Consultant
Reviewed by : Robert McMurdoPerformance Testing Team LeadDate:
John Quinn Technical ArchitectDate:
Checked by : Robert McMurdoPerformance Testing Team LeadDate:
2.1 Objectives of this document..............................................................................52.2 Audience for this document..............................................................................52.3 Scope of this document.....................................................................................52.4 Source material.................................................................................................5
3 Peak GFN Transaction per hour estimation................................................................6
3.1 Overview...........................................................................................................63.1.1 Summary of findings.......................................................................63.1.2 Analysis method..............................................................................63.1.3 Analysis for the Philippines..............................................................83.1.4 Analysis of Global throughput.........................................................8
Appendix A - Appendix......................................................................................................12
The Horizon3 (H3) Technical Solution Description (Ref 1) and High-Level Design (Ref 2) documents give further information regarding the business background and an introduction to the new system (H3 IT Platform).
Within the new IT Platform, the core application system is Global Fleet Net (GFN). GFN has its own GUI used by Shell and Logica staff who administer the Fuel Cards service.
A dedicated front-end web based application, Shell Cards OnLine (SCOL) allows Shell customers, i.e users of their Fuel Cards, to do self-service account maintenance operations, for example, view bills and request new cards.
An existing Shell application SuperSiras is to be replaced by GFN. A new version of SCOL is under development to work with GFN, and it has been radically changed for H3. It is critically dependent on web services provided by GFN. GFN is generally immature and the web services used by SCOL or GFN are not believed to have had any performance testing. Once rolled out globally, GFN will be used by the call centre users. The hardware server resources available to GFN and SCOL have not been specifically sized for the anticipated loading, although the GFN servers are powerful.
Altogether, the performance of the SCOL online system in combination with GFN is viewed as one of the higher risk areas within the H3 IT Platform and therefore a priority for performance testing. Given the relatively low user base in the Philippines, the risk for PH is low, but the risk will increase as the rollout progresses and the user numbers increase.
This document is the analysis work that defines the volume requirements for performance testing of GFN.
This document specifies the estimated volumes to undertake performance testing of the GFN application.
This document should give enough detail so that the offshore testing team can perform the tests without resorting to other documentation.
2.2 Audience for this document
The audience for this document is the performance testing team and interested parties within Logica H3 programme. They will use this document to help define performance tests scenarios to use with Loadrunner, or any other testing tool against the GFN application.
2.3 Scope of this document
The scope of this document is to detail the assumptions and methodology used in defining the volumetrics for performance testing the GFN application. It details the business transactions per hour for several countries and an estimate of the global throughput for GFN.
2.4 Source material
The main source material is Reference 3 (Cards Activities Guilaine v 2). This material has several features to note:
It is data from 2010, and only covers 10 months
It is only concerned with substantial transactions ie. not just a search but a more complex transaction that will involve a search and editing data.
It only covers 16 countries.
These features were borne in mind and taken account of during the analysis.
The following section show the analysis methods used to estimate the projected transactions per hour that will be experienced by GFN.
The results in this section are derived from analysis of spreadsheets of calls received in call centres and rollout estimation spreadsheets. This information was imported into Excel, analysed and refined, adding contingency to achieve the final figure.
3.1.1 Summary of findings
The throughput requirements for the performance testing of the perceived global GFN
Global throughput: 2325 business transactions per hour
Philippines throughput: 130 business transactions per hour
Hong Kong throughput: 70 business transactions per hour
3.1.2 Analysis method
The analysis was done in two stages.1. Estimates for the Philippines and Hong Kong were based on real call centre
data available for 16 countries.2. The global throughput figure for the 32 countries expected to use GFN was
extrapolated from the data used in Stage 1.
The main source of call centre data was contained in Reference 3. This Excel spreadsheet gives actual volumes of calls to call centres, split by country, call type and type of transaction. It gives both voice and non-voice received ‘calls’ or transactions for 10 months in 2010. The summary sheet is shown in table 1.
1. The peak month for PH was March 2010 with 18579 ‘tasks’ (highlighted). This was rounded up to 19000.
2. To estimate the tasks per day 19000 is divided by 30. This equals 633 tasks.3. 633 was rounded up to 650This subtotal gives us the average transactions per day of 650 on a busy month. However we want to estimate the peak hour throughput.
Based on analysis done for SCOL v3 using IIS logs of SCOL v2 logs(ref 6) peak hour throughput rates are approximately 50% higher than the average rate per hour in normal working hours.
4. The average tasks per hour 650/8 = 81.25. Round up to 85.5. Therefore the peak tasks per hour = 85 * 150% = 128. 6. Round this up to 130.
The estimated peak hour throughput for the Philippines is 130 transactions per hour.Using the same rounding up and estimation technique the figure for Hong Kong is 70 business transactions per hour.
3.1.4 Analysis of Global throughput
The source data only covered 16 countries. GFN is due to be implemented in over 30 countries. Therefore the ‘missing’ countries needed to be identified and their predicted use of the system needed to be estimated.
The method used was :1. Identify the missing countries2. Gauge the relative size of each missing country3. Estimate the average transaction per month for each missing country size4. Add those transactions to the global figure 5. Use the same formula as stage 1 to estimate the potential peak hour on average day in a peak month.
1. Using Reference (n) Rollout Volumes and cross checking with reference Nn) the contract 20 countries were identified to have GFN but were not in the source data. These countries were:Malaysia, Czech Republic, Poland, Hungary, Slovakia, Spain, SingaporeBulgaria, Romania, Slovenia, Greece, Russia, Croatia, Lithuania, UkraineMacau, Latvia, Gibraltar, Portugal, Estonia
2. The ‘Size Indicator’ column from Reference x (the ‘Territory Stats Spreadsheet) was used to assign the missing countries into 3 for largest, 2 for medium and 1 for smallest. Of the 20, 4 are medium; Malaysia, Czech, Poland, Thailand. The other 17 came out as small. This categorisation compared to transactions roughly aligned to Largest = >1.5 Mill txn/month, Medium = 0.5Mil to 1.5Mil txn/month and Small = <0.5Mil txn/month from the figures in the ‘Rollout Volumes’.
3. This sizing was then mapped to Guilaines spreadsheet so the original 16 countries listed in the spreadsheet had a 1, 2, or 3 assigned to them, based on their ‘transactions per month figures in Reference x. I then needed to estimate the average transactions per month for medium and small countries in Guilaines Spreadsheet.Ignoring some anomalies I then averaged the ‘transactions/month’ figures from Guilaines sheet for each Sizing group.
document.docx page 8 of 12
Horizon 3 - H3.15.03.01.02 / Issue 0.1
20/08/2012
This gave averages in Guilaines spreadsheet of 23k/month for large, 13k/month for medium and 3.5k/month for small countries. This gives the business transaction information needed to be added to the ‘global’ figures in Guilanes spreadsheet.
4. I then ‘added’ 4 x medium 13k/month entries and 16 x small 3.5k/month entries to Guilanes spreadsheet corresponding to the sizing estimates for the 20 other countries.
The result of adding this information to the original spreadsheet are shown in Table 2. The blue font shows the additional information added. There are two extra rows (“extra 2’s” and “extras 3’s”) that show the extra transactions estimated from the analysis above. They add approximately 108K/month transactions to the worst month, March 2010, giving a figure of 363k/month as a global total for that month.
5. Using the same methods as before , for stage 1 the monthly figure is then divided by 30 for a per day figure, divided by 8 for an average hour in that worst month and then multiplied by 150% for a peak hour uplift, rounded up slightly throughout.
Which gives a figure per peak hour for global ‘substantial’ GFN transactions of 2325/hr.
Therefore the extra 20 smaller countries have added 825 transactions/hr to the previous estimate which was estimated at 1500/hr for the primarily largest 16 countries.