Improving Performance of IBM DB2 ICE Clusters with Voltaire ...

Improving Performance of IBM DB2 ICE Clusters

with Voltaire InfiniBand Solutions

________________

An IBM DB2 and Voltaire Performance Study October 2003

Executive Summary Information Technology managers are under increasing pressure to run their IT and database infrastructures with fewer financial resources without sacrificing performance and reliability. Many are turning to open operating systems, low cost, commodity hardware and clustering technologies to face these challenges.

• Today, IBM offers a powerful database solution that is optimized for these new low cost computing environments. IBM DB2 ICE is a new Linux database clustering solution that helps businesses achieve the performance and scalability typically associated with large SMP servers.

• To achieve SMP-class performance, database clusters require a high speed

interconnect that is standards-based. IBM recommends InfiniBand rather than TCP/IP as the preferred interconnect for DB2 ICE Clusters.

• Recent tests by IBM and Voltaire, a leading provider of high performance

InfiniBand solutions, reveal that IBM DB2 ICE clusters leveraging Voltaire InfiniBand solutions provide a 50% transaction performance increase when compared to TCP/IP as the cluster interconnect.

The performance test results described in this paper demonstrate that InfiniBand solutions provide a winning alternative to TCP/IP for IT managers deploying DB2 ICE clusters.

Improving DB2 ICE Cluster Performance The success of Linux on Intel architecture-based servers has dramatically reduced the cost of server infrastructure. IBM has taken advantage of this trend and introduced powerful database software that is optimized for these cost effective servers. In July 2003, IBM unveiled IBM DB2 ICE, a new Linux database cluster solution that helps businesses of any size implement a low-cost, high-performance data center based on IBM Universal Database for Linux and IBM eServers. Designed to generate a fast return on investment and lower the total cost of computing, DB2 ICE provides businesses with the essential building blocks for highly available and scalable Linux database clustering. These clusters are already handling large database applications and will scale even more going forward. Problems with TCP/IP At the applications level, processing the business logic software, application servers communicate on the front-end with the Web-tier servers and on the back-end with the database servers. This communication is typically carried out today over TCP/IP connections, which put a heavy burden on the CPU of the application server. In some configurations, measurements have shown that processing the TCP/IP stack consumes 80% to 90% of the CPU cycles, not leaving much CPU capacity for the original purpose of application servers – to run applications.

Voltaire InfiniBand DB2 ICE Cluster Performance Copywrite © 2003 Voltaire, Inc.

- 2 -

Moreover, the bandwidth constraints and high latency of TCP/IP also come into play with the server-to-server communications in a clustered database. The servers running a distributed database tend to exchange a great deal of information. If the traffic between servers is slowed, the overall performance of the database is reduced. Equally important, the high latency inherent in TCP/IP limits the ability to scale a database cluster beyond just a few nodes. For some applications such as Decision Support Systems (DSS) and data warehousing, the high speed communication between the server-nodes is even more critical - in some cases TCP/IP-related latency has an exponentially negative impact on performance. Standards-Based, High Speed Interconnect is Required In the past, large database solutions required large and expensive SMP (Symmetric Multi-Processing) servers. SMP servers are based on architectures that use large memory and multiple CPUs, which are connected using proprietary high-speed interconnects. Database clusters require a high speed interconnect in order to perform at the levels of large SMP servers. InfiniBand architecture, the first industry standard high speed interconnect delivers the performance that is required for building such clusters (See Figure 1).

Figure 1: Emulating SMP Architecture with InfiniBand To address the demanding performance and scalability requirements of the data center, IBM DB2 is leveraging Voltaire’s high performance InfiniBand solutions as the standard interconnect for database clustering solutions. By replacing TCP/IP connections with Voltaire InfiniBand, IBM DB2 ICE solutions can achieve the performance and scalability of expensive SMP servers.

Voltaire InfiniBand DB2 ICE Cluster Performance Copywrite © 2003 Voltaire, Inc.

- 3 -

Introducing InfiniBand InfiniBand is a high speed, high density, serial interconnect, specifically designed from the ground up, to address exactly the performance and latency limitations of TCP/IP. A PCI-sized InfiniBand adapter (called an HCA) has two 10Gbps ports (vs. one 1Gbps port for a typical Ethernet card). Each of these ports can support an aggregate throughput of 20Gbps, a 20x improvement over a typical Ethernet card. In addition, by embedding the networking stack in silicon and using Remote Direct Memory Access (RDMA) techniques, there is a significant improvement in CPU overhead. This frees up CPU cycles for the actual applications, thereby improving the application performance tremendously. Figures 2, 3, 4 provide a quick comparison of InfiniBand to TCP/IP

Figure 4: CPU Overhead InfiniBand (SDP) vs. TCP/IP

Source: Voltaire and IBM DB2

Figure 2: Bandwidth InfiniBand (SDP) vs. TCP/IP

Source: Voltaire and IBM DB2 Infin

S

Robust InfiniBand hardware is now avVoltaire, Agilent and Mellanox Corporanow dependent on the quality and peLevel Protocols) driving the InfiniBanddetermines the performance of the In The InfiniBand protocol chosen for IBfor business applications (which are laprotocol that does not require any chatechniques, SDP imposes very little CP10 times the bandwidth of standard Eoverhead. By supporting upcoming in(AIO), SDP boosts the performance e

Voltaire InfiniBand DB2 ICE Cluster PerformaCopywrite © 2003 Voltaire, Inc.

Figure 3: Latency iBand (SDP) vs. TCP/IPource: Voltaire and IBM DB2

ailable from a variety of companies such as tion. The performance of InfiniBand systems is

rformance of the transport layer software (Upper hardware. Choosing the right software finiBand system.

M DB2 is Sockets Direct Protocol (SDP). Designed rgely sockets based), SDP is a general purpose nges to the existing applications. Using RDMA U overhead for I/O operations and delivers up to thernet interconnects, with much reduced CPU novations in Linux such as support for ASync I/O

ven more.

nce - 4 -

Test Results: IBM DB2 ICE with Voltaire InfiniBand Delivers 50% Performance Increase Recent performance tests of IBM DB2 ICE clusters leveraging Voltaire InfiniBand solutions revealed a 50% transaction performance increase when compared to TCP/IP as the cluster interconnect. The same database transaction was run over an IBM DB2 ICE cluster using TCP/IP and then over an IBM DB2 ICE cluster using Voltaire InfiniBand as the interconnect. The test revealed that the same database transaction takes twice the amount of time to complete using TCP/IP. The test confirms that Voltaire InfiniBand solutions deliver exceptional application-level performance of clustered IBM DB2 Universal Database. Implications The performance test evaluated independent transaction queries which resulted in time savings that measured in seconds. In a real world environment, where a DSS calculation may take hours to complete, a 50% transaction performance increase offers significant time savings – potentially as much as several hours. The reduction in time needed to complete complex database calculations means that IT managers can: • Run complex calculations more frequently to generate fresher, more accurate data to

make better-informed business decisions, and • Free up costly IT resources to conduct other mission-critical tasks in the data center. Test Configuration Details Two IBM DB2 ICE clustering test scenarios were created: the first used TCP/IP; the second test used Voltaire InfiniBand solutions for the interconnect. The tests were run 10 times and the results were averaged. The test results demonstrated that identical database transactions running on an IBM DB2 ICE cluster take an average of 125 seconds using TCP/IP as the interconnect and an average of 83 seconds using Voltaire InfiniBand. (See Figure 5). Even better performance is expected as the number of nodes in the cluster increases. The database transaction query emulated a data warehouse environment for a cost allocation system in a financial institution. The transaction query involved multiple tables of instances.

Voltaire InfiniBand DCopywrite © 2003 Vo

Source: Voltaire and IBM DB2

B2 ICE Cluster Performance ltaire, Inc.

- 5 -

The IBM DB2 ICE and Voltaire test bed configuration consisted of a dual-node cluster using low cost, standard 32 bit Intel architecture-based servers running IBM Universal Database 8.1 for Linux. To connect the cluster, Voltaire provided its Voltaire ISR 6000 switch router, 2 Voltaire HCA 400 dual-port, 4X InfiniBand host channel adapters and high performance software stack for Linux. (See Figure 6). Important to note, the test used a standard IBM DB2 ICE cluster, a standard operating system (Linux) and required no changes to the application, no changes to the operating system, and no changes to the server configuration or attached devices. Testers simply removed the Ethernet cards and replaced them with InfiniBand adapters (HCAs) and connected to an InfiniBand switch router.

Figure 6: Test Bed Configuration:

Voltaire InfiniBand DB2 Cluster Solution

Deploying IBM DB2 ICE over Voltaire InfiniBand Solutions When a new technology such as InfiniBand is deployed in the data center, the issue of integrating it with the existing infrastructure becomes critical. Voltaire InfiniBand solutions require no changes to IBM DB2 ICE clusters. PCI/PCI-X based HCA boards allow InfiniBand connections to be simply and quickly integrated into existing high performance server platforms. Furthermore, Voltaire InfiniBand solutions easily integrate with the existing networking and storage infrastructure in the data center through the use of Voltaire InfiniBand routers for connectivity with IP networks and Fibre Channel SANs. When adopting new technologies such as InfiniBand, it is always reassuring to know that the existing software infrastructure just works, without any changes.

Voltaire InfiniBand DB2 ICE Cluster Performance Copywrite © 2003 Voltaire, Inc.

- 6 -

About IBM DB2 ICE for Linux________________________ IBM DB2 ICE is a Linux database cluster that helps businesses of any size implement a low-cost, high-performance data center based on IBM Universal Database for Linux and IBM eServers. Designed to generate a fast return on investment and lower the total cost of computing, IBM DB2 ICE provides businesses with the essential building blocks for high-available and scalable Linux database clustering. The offering can scale from 2 to 1,000 nodes and can be deployed at a rate of four nodes per hour. IBM DB2 ICE is a completely integrated, high-performance and pre-tested solution that includes:

• DB2 Universal Database and DB2 Express;

• IBM eServer 325 and eServer xSeries x335, x345, x445, BladeCenter or Linux Cluster 1350 eServers, with choice of Intel 32 bit or AMD 64 bit processors;

• SuSE Linux Enterprise Server 8 or RedHat Advanced Server 2.1 operating system;

• Voltaire InfiniBand Database Kit for clustering, high-speed networking and storage connectivity;

• Tivoli System Automation for Linux, and;

• Certified support for SAP solutions such as SAP R/3, mySAP CRM and mySAP Business Intelligence

The IBM DB2 Integrated Cluster Environment with Voltaire’s InfiniBand database solution was awarded the “Best Cluster Solution” at LinuxWorld Conference & Expo in August, 2003. To learn more about IBM DB2 ICE, please visit: http://www-3.ibm.com/software/data/db2/linux/ice/

Voltaire InfiniBand DB2 ICE Cluster Performance Copywrite © 2003 Voltaire, Inc.

- 7 -

Voltaire InfiniBand DB2 ICE Cluster Performance Copywrite © 2003 Voltaire, Inc.

- 8 -

About Voltaire InfiniBand Database Solutions Voltaire’s InfiniBand database clustering solutions combined with entry-level servers provide IT managers with a powerful and scalable architecture for high volume, high transaction applications such as distributed databases, reducing Total Cost of Ownership of database clusters by as much as 80 percent. Voltaire was the first InfiniBand solutions provider to receive IBM DB2-Ready certification is the only InfiniBand solution recommended by IBM for IBM DB2 ICE clusters. More information is available at http://www.3.ibm.com/software/data/db2/linux/validate/udb81ihv.html. The Voltaire InfiniBand Database Kit is an end-to-end solution that enables IT managers to rapidly deploy InfiniBand-powered database clusters. Through its high speed, low latency capabilities, the kit allows IT managers to move their database applications from expensive high-end servers to low cost commodity servers clusters. The kit includes the following: • Voltaire ISR 6000 – InfiniBand switch router • VoltaireVision management software embedded in the ISR 6000 • Four dual port 4X InfiniBand (10 Gbps) HCA Adapters • A complete software stack for Linux (SuSE and Red Hat) including drivers and Upper

Layer Protocols • 4X InfiniBand cables To learn more about Voltaire InfiniBand solutions, please visit www.voltaire.com. Voltaire, Inc. 54 Middlesex Turnpike Bedford, MA 01730 [email protected] 1-800-865-8247