Benchmark Testing Results: Telvent ArcFM Solution Running on Microsoft SQL Server · 2017-01-30 · Applies to: Microsoft SQL Server 2008 R2 Enterprise Abstract In May and June 2011,

BenchmarkTestingResults:TelventArcFM

SolutionRunningonMicrosoftSQLServer Benchmark testing confirms the superior performance and scalability of Telvent ArcFM Solution

10.0.1 SP1 running on SQL Server 2008 R2 Enterprise and Windows Server 2008 R2 Enterprise

Technical White Paper Published: October 2011 Applies to: Microsoft SQL Server 2008 R2 Enterprise

Abstract

In May and June 2011, Telvent and Microsoft worked together to conduct a series of performance and

scalability tests for Telvent ArcFM Solution 10.0.1 service pack 1 (SP1) running on Microsoft SQL Server

2008 R2 Enterprise and Windows Server 2008 R2 Enterprise.

The benchmark results validate that ArcFM Solution is a highly scalable system on SQL Server under real-

world conditions simulating a large utility with 10 million customers. The ArcFM geographic information

systems (GIS) functionality of ArcFM Solution running on SQL Server demonstrated excellent performance

and scalability under a heavy load. The Responder functionality also performed very well under a heavy

call load, handling 70,000 incoming calls per hour for 15 hours.

The benchmark results confirm that ArcFM Solution on SQL Server can scale and perform to meet the

needs of the world’s largest utilities. Utilities can be confident that ArcFM Solution 10.0.1 SP1, deployed

on SQL Server 2008 R2 Enterprise and Windows Server 2008 R2 Enterprise, provides a reliable foundation

for a smart grid solution with a leading price-to-performance ratio.

Benchmark Testing Results: Telvent ArcFM Solution and Microsoft SQL Server

2

©2011 Microsoft Corporation. All rights reserved. This document is provided “as-is.” Information and

views expressed in this document, including URL and other Internet Web site references, may change

without notice. You bear the risk of using it.

This document does not provide you with any legal rights to any intellectual property in any Microsoft

product. You may copy and use this document for your internal, reference purposes.


3

Table of Contents

Introduction ............................................................................................................................................. 4

Brief Overview of Telvent ArcFM Solution .................................................................................................. 5

ArcFM..................................................................................................................................................... 6

Responder .............................................................................................................................................. 6

Benefits of SQL Server for ArcFM Solution ................................................................................................. 7

Benchmark Testing Confirms Performance and Scalability of ArcFM Solution on SQL Server ................... 9

Benchmark Testing Goals ......................................................................................................................... 9

Benchmarking Test Scenarios................................................................................................................. 10

Test Database .......................................................................................................................................... 10

ArcFM Testing Process ............................................................................................................................. 11

Responder Testing Process ....................................................................................................................... 12

Benchmark Testing Setup ...................................................................................................................... 13

Database and Application Servers ........................................................................................................... 13

Initial Setup and Tuning ........................................................................................................................... 14

Benchmark Testing Results .................................................................................................................... 14

Summary of Results for ArcFM ................................................................................................................. 15

Summary of Results for Responder .......................................................................................................... 16

Conclusion ............................................................................................................................................. 16

Additional Information .......................................................................................................................... 17

About Telvent ........................................................................................................................................... 17

About the Microsoft Worldwide Utilities Group ....................................................................................... 17


4

Introduction Today’s utilities face the challenges of tighter IT budgets, reduced in-house

development staff, and the need to quickly deliver business benefits through

applications. Utilities also need to adapt to the rapid changes in smart grid technology,

the electricity distribution system of the future.

Telvent and Microsoft work together to provide integrated solutions that can help

utilities meet these challenges and better use their distribution assets for maintenance

and growth. By working together to integrate key aspects of their solutions—including

Telvent ArcFM Solution, Microsoft SQL Server, and Windows Server—Telvent and

Microsoft address the most pressing needs of the smart grid age, including performance

metrics, response times, and security.

ArcFM Solution running on SQL Server and Windows Server provides a complete, out-of-

the-box solution for facilities management designed to meet the needs of utility

businesses. The solution offers a map-centric, intuitive way to model, design, maintain,

and manage facility and land–based information.

In May and June 2011, Telvent, a Microsoft Gold Certified Partner, joined with Microsoft

to conduct a series of benchmarking tests to verify the performance and scalability of

ArcFM Solution 10.0.1 SP1 running on SQL Server 2008 R2 Enterprise and Windows

Server 2008 R2 Enterprise.

The results of the benchmark testing were impressive. Highlights of the results include:

• ArcFM Solution on SQL Server was able to support the requirements of a typical

10 million–customer utility for all processes tested.

• The Responder functionality successfully processed 70,000 calls per hour for 15

consecutive hours (with no user load simulated during the testing).

• The ArcFM Solution user experience in the benchmark test was comparable to

the user experience in tests run locally on significantly smaller datasets and with

a smaller load.

• ArcFM Solution running on SQL Server performed equally well on a large system

(32 cores, 512 GB RAM) and a smaller system (8 cores, 24 GB RAM).

This white paper presents an overview of ArcFM Solution and of SQL Server, and details

the results of the benchmark testing. It also includes links for further information.


5

Brief Overview of Telvent ArcFM Solution Telvent ArcFM Solution is an extension of the ESRI ArcGIS platform, a market-leading

geographic information systems (GIS) software platform. ArcFM Solution consists of a

family of models and a set of sophisticated tools that provide a comprehensive

application for electric, gas, water, and wastewater utilities, and for telecommunication

providers. ArcFM Solution enables maintenance and data viewing using company-

specific business rules, and includes applications to configure the data models and tools

to work together.

Telvent applications gain leverage for enterprise GIS through the use of open

technologies. Key components of this strategy include an industry-standard, open-

development environment, open databases, and Component Object Model (COM)

architecture. All Telvent application software takes advantage of this object-oriented

architecture and provides for fully networked solutions in a client-server environment.

The ArcFM Solution suite was developed using Visual Basic 6 (VB6), C++, and C#; these

languages can also be used for customization and extension of the applications.

Figure 1 shows the ArcFM Solution architecture for utilities.

Figure 1. ArcFM Solution architecture for utilities

For more information, see ArcFM Solution Overview.


6

Microsoft and Telvent conducted benchmark testing on two core elements of the ArcFM

Solution—ArcFM and Responder—which together provide a solid foundation for the

smart grid.

ArcFM

ArcFM is a complete GIS utility solution for modeling, editing, maintaining, and

managing facility asset data both in the field and on the desktop. ArcFM includes tracing

tasks to automate utility operations, and an extensive set of editing tools such as

automated snapping and attribute inheritance. ArcFM also includes tools to quickly

define and recall map sheet collections for map production. Users can create and edit a

session in the field, then send it back to the enterprise to be used for editing the

database, or geodatabase, using the Session Manager (see Figure 2).

Figure 2. ArcFM Session Manager

For more information about ArcFM, see Telvent Smart Grid Solutions Suite: ArcFM

Enterprise GIS.

Responder

Responder is a GIS-hosted outage management system (OMS) that takes advantage of

Microsoft .NET and ASP technologies to enable trouble call and outage incident

management in a web-based, scalable, and configurable desktop environment. The

Responder Trouble Call Analysis (TCA) Engine uses a sophisticated and iterative

prediction algorithm to determine which interruptible network device caused an

outage. High-performance GIS display capabilities provide utility personnel with a spatial

view of trouble call locations, enabling faster analysis of outages and immediate


7

dispatch of crews (see Figure 3). The application also supports historical archiving and

performance indices reporting.

Figure 3. Responder

For more information, see Smart Grid Solution Suite: Responder.

Benefits of SQL Server for ArcFM Solution SQL Server 2008 R2 Enterprise provides a reliable, cost-effective, low-maintenance

database framework for ArcFM Solution. SQL Server is a comprehensive, integrated, and

enterprise-ready data management software that can support the largest and most

process-intensive deployments.

Using SQL Server, Telvent customers can take advantage of reduced licensing, hardware,

administration, and support fees, in addition to enterprise-class performance and

scalability. Hundreds of organizations are currently running 10 terabyte (TB) and larger

transactional databases on SQL Server.1

1 http://www.microsoft.com/presspass/itanalyst/docs/06-30-09EnterpriseDatabaseManagementSystems.pdf


8

Running ArcFM Solution on SQL Server provides customers with many benefits. These

benefits include:

• Scalability and performance.

SQL Server includes many new features that help Telvent customers scale up

and scale out as their businesses grow.2

• Lower hardware costs.

SQL Server runs on standard commodity server hardware, providing

dramatically lower total cost of ownership (TCO) for utilities compared to

mainframe computers.

• Lower software costs.

With SQL Server, customers can typically enjoy a 3:1 reduction over the largest

competitor in licensing costs.3

• Simpler systems management and lower staffing costs.4

SQL Server database administrators (DBAs) can typically manage three times as

many physical databases as a competitor’s DBAs.5

• Lower maintenance costs.6

SQL Server first-year maintenance costs are up to 77 percent less than those of

the largest competitor solutions.

• Fewer security vulnerabilities.

SQL Server has consistently had fewer security vulnerabilities than the largest

competitor solutions.7

With SQL Server, Telvent customers can save with reduced licensing, hardware,

administration, and support fees, which translate into substantially lower costs over the

life of the system.

For more information, see the SQL Server home page.

2 http://www.microsoft.com/sqlserver/2008/en/us/performance-scale.aspx

3 http://www.microsoft.com/sqlserver/2008/en/us/value-calc.aspx

4 http://www.microsoft.com/sqlserver/2008/en/us/compare-oracle.aspx

5 http://www.alinean.com/PDFs/Microsoft_SQL_Server_and_Oracle-Alinean_TCA_Study_2010.pdf

6 http://www.microsoft.com/sqlserver/2008/en/us/compare-oracle-calc.aspx

7 NIST National Vulnerability Database, http://nvd.nist.gov/


9

Benchmark Testing Confirms Performance and

Scalability of ArcFM Solution on SQL Server As part of their ongoing commitment to provide utilities with truly scalable GIS systems,

Telvent and Microsoft conducted scalability and performance testing of the ArcFM and

Responder functionalities of ArcFM Solution running on SQL Server 2008 R2 Enterprise

and Windows Sever 2008 R2 Enterprise. The benchmark testing confirmed that

Telvent customers can choose SQL Server as a database platform for ArcFM Solution

with confidence for several reasons:

• The lower cost of using SQL Server does not translate to a compromise in

performance.

Some of the largest and smallest utilities in the world use ArcFM Solution. The

benchmark testing results demonstrate that SQL Server can scale easily across

the spectrum of usage patterns without any significant performance

degradation.

• Geometric network operations are not affected by system load or server

capacity.

In the benchmark testing, a geodatabase powered by ArcFM Solution and

managed by SQL Server showed no significant changes in performance as loads

were increased from very small to very large. Performance remained consistent

from a single user to 400 concurrent users.

• ArcFM functionality scales well as system usage increases over time.

The benchmark testing looked at the effects of a rapidly growing dataset when

multiple users added features to the geometric model over time. It confirmed

that with SQL Server as a database, users will not experience a significant

decrease in performance in creating, editing, and saving ArcFM sessions as their

dataset grows.

Benchmark Testing Goals The goal of the benchmark testing was to confirm that ArcFM and Responder running on the

Microsoft platform can meet the requirements of the largest Telvent utility customers.

To test the ArcFM functionality, Microsoft and Telvent simulated the most common GIS

activities typically conducted at Telvent customer sites. Table 1 lists the goals for ArcFM.


10

Table 1. Goals for ArcFM benchmark testing

Panning and zooming

within a map

Verify that performance for these processes does not degrade

as the database size and load increase

Tracing the geometric

network

Creating and completing

edit sessions

To test Responder, Microsoft and Telvent used the custom load–simulation tool

TroubleMaker to simulate trouble or outage calls. Table 2 lists the goals for the Responder

benchmark testing.

Table 2 Goals for Responder benchmark testing

Prediction

performance testing

(call processing)

Process at least 50,000 trouble calls/trouble events per

hour

User experience

testing

Verify that as the database size and load increase,

performance does not degrade and remains comparable to

that of SQL Server competitors

Note that Telvent and Microsoft also set the goal of identifying and taking advantage of

any SQL Server 2008 R2 Enterprise performance improvement opportunities.

Benchmarking Test Scenarios To ensure that the testing was as realistic as possible, Telvent and Microsoft used test

scenarios that simulated a very large utility with approximately 10 million customers.

Test Database One of the goals of the benchmark testing was to verify that performance remained

constant as the database size and load increased. Two full database configurations were

therefore used in the testing:

• 32 cores and 512 GB of RAM

• 8 cores and 24 GB of RAM


11

The geodatabase was designed to emulate the characteristics of a large city:

• 10 million customers

• 3.0 million service points

• 0.9 million transformers

• 1.9 million primary conductors, overhead (OH)

• 1.3 million primary conductors, underground (UG)

• 2.7 million secondary conductors, OH

• 2.6 million secondary conductors, UG

ArcFM Testing Process To test ArcFM, Telvent used Citrix EdgeSight to simulate common client activities

running concurrently for 30 minutes:

• Panning and zooming within the map (400 simultaneous users for 30 minutes)

• Tracing (400 simultaneous users for 30 minutes)

• Editing (200 simultaneous users for 30 minutes)

Note: The number of simultaneous users used in the benchmark testing is twice the

known number of concurrent users at any Telvent customer site.

Panning and Zooming

The Pan and Zoom test simulated 400 simultaneous users panning and zooming within a

map and using the following procedure:

1. Set minimum and maximum XY coordinates to establish a working area in the

map.

2. Select a random XY coordinate within the designated area and pan to it.

3. Select a scale between 1:1,000 and 1:100,000.

Tracing

The Trace test simulated 400 simultaneous users performing several tracing operations

on Dynamic Protective Device (DPD) features using the following procedure:

1. Open the Locator tool and perform an attribute search for all DPD features.

2. Enter a number in the ObjectID field to locate a random DPD feature.

3. Zoom to the located DPD feature and center it in the map.

4. Select the Downstream Trace tool and click on the DPD feature to initiate a

trace.


12

5. Clear the results and perform a Downstream Protective Device trace on the

same feature.

6. Clear the results and return to the Locator tool to search for another random

DPD feature.

Editing

The Edit test simulated 200 simultaneous users completing an editing session using the

following procedure:

1. Open Process Framework and create a new session. Name it based on the

current time.

2. Start the edit session in ArcMap, and zoom to a specific location on the map.

3. Sketch a number of poles (approximately 50).

4. Sketch a primary overhead conductor along the placed poles.

5. Save and close the session.

Responder Testing Process To test Responder, Microsoft and Telvent used Citrix EdgeSight and TroubleMaker to

simulate 70,000 trouble or outage calls per hour. These calls were submitted and

processed through the Responder Prediction Engine for 15 consecutive hours. This load

is more than double the load Telvent has observed at their largest existing customer

sites.

Two types of Responder tests were conducted:

• Prediction performance

• User experience

Prediction Performance Testing

The benchmark test used the TroubleMaker event simulation agent to submit calls at

various rates (between 25,000 calls/hour and 100,000 calls/hour). Telvent reviewed the

accuracy of the outage-prediction metrics (based on calls). Telvent also confirmed that the

number of outage groups created from the predictions (referred to as “incidents” in

Responder) was the expected number given the prediction thresholds for the areas.

User Experience Testing

The benchmark test calculated the time to perform various client activities such as cancel,

create, confirm, and restore an incident/outage. Activities were performed with various

system loads, and calls were submitted concurrently.


13

Benchmark Testing Setup The benchmarking test was conducted in the Microsoft Enterprise Engineering Center (EEC)

labs in Redmond, Washington, using standard, commodity servers and storage to confirm a

realistic cost of ownership for the solution.

Database and Application Servers For the ArcFM testing, the environment consisted of one database server and 12 application

servers in a Citrix farm (using Citrix XenApp 5.0).

For the Responder testing, the environment consisted of three business logic servers and

one Responder message-router server running the Responder business logic. Testing also

included a Responder web server and the Citrix EdgeSight server farm to provide the

simulated user load.

Figure 4 shows the hardware configuration used.

Figure 4. Benchmark testing environment

Table 3 shows details of the hardware used in the benchmark testing.


14

Table 3. Details of hardware used

Backend tier Database server

32 cores, 512 GB RAM;

also reduced to

8 cores, 24 GB RAM

Middle tier

Responder business logic servers (3) 8 cores, 64 GB RAM

Responder web servers (1) 8 cores, 64 GB RAM

Responder message router server (1) 8 cores, 64 GB RAM

License server (1) 2 cores, 4 GB RAM

Domain controller (1) 2 cores, 4 GB RAM

Frontend tier

Citrix farm servers (12) 8 cores, 48 GB RAM

Citrix EdgeSight simulation (5) 8 cores, 32 GB RAM

The database server was connected to a XioTech Magnitude 3D 3000 storage area

network (SAN) with one hundred and ten 146 GB-15K-FC drives.

Initial Setup and Tuning The databases and application servers were set up according to the ArcFM and

Responder configuration guides. Telvent installed Citrix XenApp 5.0 (an application

management system that virtualizes, centralizes, and manages applications) along with

Citrix EdgeSight, and configured both to support up to 500 concurrent users.

Note that the testing included a large number of domain users to ensure real-world

scenarios. Each user profile was set to “roaming” to ensure the environments were

consistent across the Citrix server farm.

Benchmark Testing Results The benchmark testing results confirm that Telvent ArcFM and Responder running on

SQL Server and Windows Server are highly scalable systems that provide the

performance required by large Telvent customers.


15

Summary of Results for ArcFM The benchmark testing showed that ArcFM is capable of supporting more than 400

users simultaneously panning different portions of the map without degradation of the

redraw performance.

Table 4 summarizes the results for the various ArcFM processes tested:

Table 4. Summary of results for ArcFM

Pan

and

zoom

• Pan and zoom scaled successfully up to 400 simultaneous users

(there was no degradation in system performance between a single

user and 400 users).

• In addition, even with 400 users, there was no performance

degradation between the large, scaled-up database configuration

(32 cores, 512 GB RAM) and the scaled-down configuration (8 cores,

24 GB RAM). This confirms that smaller hardware configurations can

provide high performance and scalability.

• The time of operation for pan and zoom in all scenarios remained at

0.5 seconds.

Tracing

• Tracing scaled successfully up to 400 simultaneous users (there was

minimal degradation in system performance between a single user

and 400 users).

• In addition, even with 400 users, there was no performance

degradation between the large, scaled-up database configuration

(32 cores, 512 GB RAM) and the scaled-down configuration (8 cores,

24 GB RAM). This confirms that smaller hardware configurations can

provide high performance and scalability.

• Time of operation for tracing functions varied by only milliseconds.

Edit

session

• The time of operation for opening and closing edit sessions

increased by less than 5 seconds when scaling from one user to 200

concurrent users.

• The time of operation for saving and closing a session grew by less

than 5 seconds when scaling from one user to 200 concurrent users.


16

Summary of Results for Responder The benchmark testing showed that the ArcFM Solution Responder functionality is able

to support high call volumes, with a maximum call processing of 70,000 calls per hour

over the course of 15 hours.

Table 5 summarizes the results for the Responder processes tested:

Table 5. Summary of results for Responder

Prediction

performance

testing (call

processing)

• Responder processed 70,000 calls per hour for an extended

period (approximately 15 hours, no user load).

User

experience

testing

• User experience metrics were comparable with tests run

locally on significantly smaller datasets with less of a load and

a smaller geodatabase size.

• User experience time measures for create, confirm, restore,

complete, edit, and close an incident/outage tasks were

comparable with tests run locally on significantly smaller

datasets with less of a load and a smaller geodatabase size.

• Note that the user experience measures for cancel,

consolidate, and move upstream an incident/outage tasks

were slow and/or timed out. The performance issues for these

particular operations under significant load and scale were

identified as issues with the Responder solution code, and will

be addressed in future versions.

Conclusion The benchmark testing results confirm that ArcFM and Responder are highly scalable

systems capable of supporting a geodatabase containing 10 million customers on SQL

Server 2008 R2 Enterprise and Windows Server 2008 R2 Enterprise.

The benchmark testing showed that the cost effectiveness of the Microsoft platform

does not mean a compromise in performance. ArcFM Solution running on SQL Server

showed no significant performance degradation as loads were increased from a single

user to 400 concurrent users. For ArcFM functionality, the testing showed that

performance remained consistent between a larger hardware configuration and a


17

smaller, less powerful system. For Responder, the user experience for most tasks was

comparable to when run locally on significantly smaller datasets with a smaller load.

Highlights of the benchmark testing include the following results:

• Both ArcFM and Responder were able to support the real-world processes for a

typical 10 million–customer utility within target time windows for all processes

tested.

• Responder successfully processed 70,000 calls per hour through the Responder

Prediction Engine for 15 consecutive hours. This demonstrates that Responder can

successfully handle an unusually high call volume for an extended period of time.

• ArcFM functionality showed zero degradation when scaling to a significantly larger

database with more load and on a less powerful server. This proves the ability of

ArcFM running on SQL Server to scale while maintaining performance.

Telvent customers can be confident that ArcFM Solution running on the Microsoft

platform can meet their performance and scalability requirements.

Additional Information The following references provide more information about Telvent and Microsoft.

About Telvent

Telvent (NASDAQ: TLVT) is a global IT solutions and business

information services provider dedicated to helping improve efficiency, safety, and security

for the world’s leading companies. Telvent serves markets that are critical to the

sustainability of the planet, including the energy, transportation, agricultural, and

environmental sectors. For more information, visit www.telvent.com.

About the Microsoft Worldwide

Utilities Group The Microsoft Worldwide Utilities Group offers platform and partner solutions that

empower utilities to thrive in today’s market-driven environment through optimized

business operations in customer care, generation, tracing and risk management,

transmission and distribution, regulatory compliance, and enterprise services. For more

information, contact your local Microsoft sales representative, or visit

www.microsoft.com/utilities.

Benchmark Testing Results: Telvent ArcFM Solution Running on Microsoft SQL Server · 2017-01-30 · Applies to: Microsoft SQL Server 2008 R2 Enterprise Abstract In May and June 2011,

Documents