Top Banner
5 Easy Steps to a High-Performance Cluster 1 5 Easy Steps to a High Performance Cluster Slash your product development cycle with an easy-to-manage cluster of computing power. Produced by the editors of Desktop Engineering DESKTOP ENGINEERING WITH IBM ON: Sponsored by IBM Platform Computing: SOLUTION SERIES
15

DESKTOP ENGINEERING 5 Easy Steps to a High Performance …...5 Easy Steps to a High-Performance Cluster 6 Such was the case at Swift Engineering. Swift is an innovator in the design

Jul 28, 2020

Download

Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: DESKTOP ENGINEERING 5 Easy Steps to a High Performance …...5 Easy Steps to a High-Performance Cluster 6 Such was the case at Swift Engineering. Swift is an innovator in the design

5 Easy Steps to a High-Performance Cluster 1

5 Easy Steps to a High Performance ClusterSlash your product development cycle with an easy-to-manage cluster of computing power.

Produced by the editors of Desktop Engineering

DESKTOP ENGINEERING WITH IBM ON:

Sponsored by IBM Platform Computing:

SOLUTION SERIES

Page 2: DESKTOP ENGINEERING 5 Easy Steps to a High Performance …...5 Easy Steps to a High-Performance Cluster 6 Such was the case at Swift Engineering. Swift is an innovator in the design

5 Easy Steps to a High-Performance Cluster 2

CONTENTSHPC Drives Engineering Innovation ........................................................................... 3

What is a Cluster?.......................................................................................................... 4

Why Clusters ................................................................................................................. 4

Cluster Management ...................................................................................................... 5

An Integrated Cluster Management Solution ............................................................... 6

Managed, Integrated Clusters Lower Cost of Ownership ......................................... 10

Next Steps Toward HPC Clusters .............................................................................. 12

APPENDIXGlossary of Terms ...........................................................................................................14

IBM Platform Computing Family .................................................................................15

Additional Resources ......................................................................................................15

Page 3: DESKTOP ENGINEERING 5 Easy Steps to a High Performance …...5 Easy Steps to a High-Performance Cluster 6 Such was the case at Swift Engineering. Swift is an innovator in the design

5 Easy Steps to a High-Performance Cluster 3

HPC DRIVES ENGINEERING INNOVATION

Engineers are being asked to do more in less time to meet ever-tightening time-to-market schedules. To do so, they need to accelerate design by making use of advanced engineering software. However, such software requires computing processing power not available in a typical engineering workstation.

For example, computer-aided engineering (CAE) applications, such as computational fluid dynamics (CFD) and finite element analysis (FEA) simulation software, can radically shorten the design cycle by allowing engineers to virtually test new ideas quickly. Photorealistic ren-

dering software allows many companies to begin promoting a product long before the first one rolls off the production line. Unfortunately, both CAE and rendering applications can slow workstations to a crawl, defeating their potential to decrease the product design and development process.

In a recent survey of Desktop Engineering subscribers, 88% of respondents reported using CAE and visualization ap-plications. If your company is in the majority that could benefit from access to more computing power, a cluster comprised of commodity servers may be a viable solution to consider, especially now that they’re easier to purchase, deploy, configure and maintain than ever before.

A high performance computing (HPC) cluster refers to a group of servers built from off-the-shelf components that are connected via certain interconnect technologies. A cluster can deliver aggregated computing power from its many processors with many cores — sometimes hun-dreds, even thousands — to meet the processing demands of more complex engineering software, and therefore de-

liver results faster than individual workstations. Thanks to Moore’s Law, the speed of commodity processors continues to improve every 18 months. Clusters built on those processors and other widely available components can deliver a superior price/perfor-mance advantage over alternative solutions such as custom built systems or supercom-puters. For many small- and mid-sized engineering-based businesses, clusters are the sweet spot for accelerating their complex simulations and renderings.

“Using HPC for modeling is critical in product design. To build a physical test specimen with concrete, we would need 28 days just for the concrete to reach design strength. With HPC, we can simulate that environment in two days. Product developers use HPC to design and optimize products for their specific goals in a very short timeframe.”1

— Frank Ding, R&D Engineer, Simpson Strong-Tie

1. Source: “Strengthening product design with HPC,” Intel Corp., 2012

HIGH PERFORMANCE COMPUTING CONTINUUM

CLUSTERSMulti-coreWorkstations Supercomputers

Clusters occupy the broad middle of the HPC continuum, allowing engineers to perform more powerful computations than are feasible with workstations.

Page 4: DESKTOP ENGINEERING 5 Easy Steps to a High Performance …...5 Easy Steps to a High-Performance Cluster 6 Such was the case at Swift Engineering. Swift is an innovator in the design

5 Easy Steps to a High-Performance Cluster 4

What is a Cluster?In the simplest of terms, a technical computing cluster is a pool of computing resources formed by linking together multiple computers, called nodes, via interconnect technolo-gies, such as InfiniBand and Ethernet. Vendors offer preconfigured cluster solutions that package multiple nodes into a cabinet, or clusters can be created by linking together exist-ing computer hardware.

Whether turnkey or homegrown, the basic idea behind a cluster is the same: Take a com-plex computing problem and divide it into smaller pieces that the cluster’s many cores can solve in parallel, resulting in shortened runtimes.

You’re probably familiar with the term “parallel processing.” Clusters are built to enable this programming paradigm. Today, many engineering applications such as CAE and ren-dering software have been updated to cluster-ready versions (“parallelized”) to take advan-tage of the many cores available in a computer and the clustered computing architecture. These parallel applications can run much faster on a cluster comprised of nodes with multiple cores.

Why Clusters? The benefits that advanced computing technologies bring to design engineering teams are clear. Performing more simulations early in the design process, completing more design it-erations, and increasing productivity are paramount to many companies. Ultimately, being able to simulate more design iterations results in a better product getting to market faster. Clusters enable all those benefits, and can be used by multiple teams within the company running various types of applications, which improves IT resource utilization and leads to significant savings.

However, according to a survey of Desktop Engineering subscribers, only 17% say they are using HPC clusters. Why?

They cite three main concerns:

1Lack of familiarity and knowledge of HPC computing clusters. While HPC clusters have be-come easier to use, Desktop Engineering’s survey shows that engineers have not kept up

on advances that have made clusters extremely user friendly. Reading this paper and perus-ing the links to more information in the appendix will give you a good working knowledge of those cluster advancements.

2Tight IT budgets. Money is always a concern. The companies that thrive choose tech-nologies that can deliver a tremendous return on investment by fueling productivity

and innovation gains. As we will show via the real-world examples mentioned in this paper, clusters enable companies to maximize throughput from their application software, which quickly pays off by increasing productivity and shortening time to market.

The initial investment is only part of the return on investment (ROI) story. Legacy clus-ters built using open source software had a very low cost barrier to entry. However, as the No. 3 concern Desktop Engineering subscribers have about clusters illustrates, the real cost of open source software quickly becomes apparent.

Page 5: DESKTOP ENGINEERING 5 Easy Steps to a High Performance …...5 Easy Steps to a High-Performance Cluster 6 Such was the case at Swift Engineering. Swift is an innovator in the design

5 Easy Steps to a High-Performance Cluster 5

3Lack of skilled staf f to manage an expanded HPC computing environment. Migrating applications and managing an HPC environment using open source soft-

ware is a valid one. Lack of proper setup and management are the fastest ways to derail a cluster’s benefits. The great news here is that it’s easier than ever to deploy, configure and manage clusters — thanks to the advances in cluster management software that are outlined on the following pages.

CLUSTER MANAGEMENT

Cluster management software helps identify, monitor and manage various hardware and software components in a cluster, from the CPU, memory, and accelerators it may have, to networking devices, operating systems and application workloads. Cluster manage-ment software can divide computing resources based on different workload require-ments and policies, which are designed by the IT department, to make a cluster work most efficiently.

Historically, clusters have been deployed, configured and maintained using a variety of open-source software tools. Many of these free or inexpensive tools could be cobbled

together to get a cluster up and running. One piece of soft-ware may have been used to provision (identify and regis-ter) each node in a cluster; another piece would have been needed to configure the network. Other software handled job scheduling and load balancing, while still other appli-cations allowed IT managers to administer and monitor the cluster. They all had different user interfaces and dif-ferent learning curves, but were often supported solely via informal user forums.

Some companies successfully set up and maintained such cheap or free cluster management software tools. How-

ever, many didn’t have the IT expertise or the time to scale the learning curve to make the various pieces of open-source cluster management software work together. That led to problems.

When a cluster was employed using open-source software without the expertise needed to manage it, horror stories followed. Maybe one person in IT really knew all the open-source cluster management software in and out, but then he or she moved on and left a company struggling to understand what was installed. Or maybe everything was humming along until an update to one of the many pieces of software caused the entire cluster to fail. In many cases, the hidden costs of using open-source cluster management software revealed itself slowly via lost productivity and inefficient use of resources. Ac-cording to “The Hidden Costs of Open Source,” an IBM white paper, IT’s time was wasted tending to the many pieces of software or searching forums for answers — and engineers’ time was wasted waiting for the cluster to come back online or perform up to its promised potential.

Cluster management software can divide computing resources based on different workload requirements and policies, which are designed by the IT department, to make a cluster work most efficiently.

Page 6: DESKTOP ENGINEERING 5 Easy Steps to a High Performance …...5 Easy Steps to a High-Performance Cluster 6 Such was the case at Swift Engineering. Swift is an innovator in the design

5 Easy Steps to a High-Performance Cluster 6

Such was the case at Swift Engineering. Swift is an innovator in the design and manu-facture of composite structures that have produced championship winning racecars and next generation aviation and aerospace designs and components. Its open source cluster, installed to speed its CFD simulation, was taking more time and effort to maintain than it was worth.

“We needed a HPC solution where we spent more time solving complex problems than we did administering the system,” said Dr. John F. Winkler, Swift’s chief aerodynamicist. To add to the challenge, clusters are becoming increasingly complex. The size of clusters continues to grow, so more nodes need to be managed. The components used in a cluster are no longer in a simple, unified format. For example, many clusters today include both CPUs and graphics accelerators to which jobs can be distributed. Networking technologies used to connect servers within a cluster are getting more advanced. Applications are also getting more sophisticated, with different computing requirements from what they had before. Thankfully, for Swift Engineering and others who want to deploy a cluster but are con-cerned about the daunting management tasks associated with open-source management software, there is a better way.

An Integrated Cluster Management SolutionAs the market for clusters has expanded to big business, such as life sciences, the oil & gas industry and financial services, cluster management software has also become easier to de-ploy, manage and use. Established technology providers, such as IBM, now offer complete HPC cluster management solutions. In 2012, IBM completed its acquisition of Platform Computing, a provider of cluster and workload management software with a 20-year track record and a roster of more than 2,000 clients.

IBM Platform HPC is an integrated cluster solution featuring a centralized web interface that is designed to make it easy to use and manage a cluster. This single product includes a workload scheduler based on Platform LSF that can prioritize various workloads run-ning on the cluster using sophisticated policies set by the IT department to ensure overall performance of the cluster; a cluster management solution that can provision and moni-tor a cluster, and report its status in real time; a message passing interface to speed the

performance of parallel applications running on the cluster; and a set of application templates that simplifies job submission for users running the most commonly used ISV applications. Platform HPC is a fully integrated and certified solution designed to ensure ease of use, improved applica-tion performance and simplified troubleshooting for HPC cluster users. As an open, heterogeneous product, it supports all x86 hardware platforms.

The company says Platform HPC can be used by IT and end users alike to handle all aspects of cluster management.

Intel Cluster ReadyPlatform HPC is a member of the Intel Cluster Ready program, which simplifies the buying, deploying, and managing HPC clusters by setting rules for hardware and software interoperability. Platform HPC users can launch the Intel Cluster Checker validation tool by setting up a simple configuration in the menu.

Page 7: DESKTOP ENGINEERING 5 Easy Steps to a High Performance …...5 Easy Steps to a High-Performance Cluster 6 Such was the case at Swift Engineering. Swift is an innovator in the design

5 Easy Steps to a High-Performance Cluster 7

“All of the capabilities within Platform HPC are tightly integrated,” Nick Werstiuk, Worldwide Product Line Executive for Platform Computing. “It’s not just a software stack between the operating system and cluster. Platform HPC is a single product with a single installer and user interface. The product is also tightly integrated with the ap-plication layer. Scripting guidelines and job submission templates for commonly used commercial applications enable seemless application integration, and ensure that users are immediately productive by simplifying job submission, reducing job setup time and minimizing operation errors.”

Platform HPC customer experiences support those claims.

Platform HPC

Windows HPC Server

Altair Gridworks

Bright Cluster Manager

Oracle Grid Engine

MOAB Cluster Suite

Linux Cluster Management ✔ ✔

Windows Support ✔ ✔ ✶✶ ✶✶ ✶✶

Workload Driven Dynamic OS Switch ✔ ✔ ✔

Integrated Web Portal ✔ ✶✶ ✶✶ ✶✶ ✶✶

Application Integration ✔ ✶ ✶✶ ✶✶

One-step Install ✔ ✶✶ ✔

Commercial MPI with Optimization ✔ ✶✶

3rd Party Management SW Integrations ✔ ✶✶

Reliable & Scalable Resource Management ✔ ✶ ✔ ✶ ✶

Windows Support ✔ ✔ ✶✶ ✶✶ ✶✶

Basic Scheduling (fairshare, preemption, backfill etc.)

✔ ✔ ✔ ✔ ✔

Advanced Scheduling (app profile, arbitrary resource limit etc.)

✔ ✔ ✶✶ ✔

Application Integration ✔ ✶ ✶✶ ✶✶

MPI Integration ✔ ✶✶ ✔ ✶✶ ✶✶

Workload Driven OS Switch ✔ ✔ ✶✶ ✔

Comparison of Cluster Management Solutions

Page 8: DESKTOP ENGINEERING 5 Easy Steps to a High Performance …...5 Easy Steps to a High-Performance Cluster 6 Such was the case at Swift Engineering. Swift is an innovator in the design

5 Easy Steps to a High-Performance Cluster 8

The web interface in Platform HPC was a “huge benefit compared to not having a web interface before,” says Chris Collins, the head of Research and Specialist Computing Support at the University of East Anglia (UEA), a research university in the UK. “With the challenge of getting non-HPC users to use the system, we could allow them to use the web front-end instead of using the command-line interface.”

In addition to making the cluster easy to use, Platform HPC also makes some of the most popular parallel en-gineering software easy to deploy on the cluster via its application templates. Open-source clusters often require users to submit their application by writing a lengthy script and execute through a command line, which can be time consuming and error prone. The application tem-plates in Platform HPC take care of the script building process. Users just need to enter the necessary param-eters for running their applications via a user-friendly interface before hitting the run button. IBM Platform

Computing says it is a much simpler and effective process compared to open source and other alternative solutions.

Applications running on different operating systems can all be managed by Platform HPC, as the Virginia Tech Transportation Institute discovered.

The web interface in Platform HPC was a “huge benefit compared to not having a web interface before.”— Chris Collins, head of Research and Specialist Computing Support | UEA

Platform HPC’s Integrated Cluster Configuration and Management Tools

OS OS OS OS OS

Ansys Fluent

Home- Grown AppLS-DYNAMSC

NASTRAN Blast

Unified Web-Based Interface

Cluster Management

Workload Management

MPI Library

GPU Scheduling

OS Multi-Boot

App Integration Mon

itor

ing

&

Repo

rtin

g

Page 9: DESKTOP ENGINEERING 5 Easy Steps to a High Performance …...5 Easy Steps to a High-Performance Cluster 6 Such was the case at Swift Engineering. Swift is an innovator in the design

5 Easy Steps to a High-Performance Cluster 9

The Institute gathers information from thousands of vehicles to further its transporta-tion safety research. It needed to run MATLAB mathematics visualization software and its own proprietary software under Windows while running other applications in a Linux environment. It was able to have the best of both worlds with Platform HPC, which supports both Windows and Linux users.

“The stability and maturity of the Platform HPC product was an attraction to us. At the same time, Platform continues to innovate,” says Clark Gaylord, Chief Informa-tion Officer at Virginia Tech Transportation Institute. “Being able to use the computer cluster in this heterogeneous, cross-platform environment is very important to moving our research forward.”

Platform HPC is also backed by IBM support and service, so help is there if you need it. Gaylord says Platform HPC was instrumental in the training that supported the Institute’s transition from the Windows HPC environment. If any issues arise, there is one number to call for both hardware and software questions. And Platform HPC is certified by a variety of server, storage and interconnect vendors, so you can count on hardware support as well.

“One of Platform Computing’s strengths is its support team, which you don’t neces-sarily get with those from other companies,” says Nathan Sykes, CFD team leader for

Red Bull Technology, which supports the championship F1 (Formula One) Red Bull Racing team. “We can turn to Platform Computing for help when something doesn’t work and they are very responsive, no matter the time of the day.”

That support helps new customers get up and running fast, and increase the speed of their simulation and rendering jobs, as Swift Engineering found out.

“We transitioned from a two workstation solution to a full-scale HPC cluster within a week very quickly and very efficiently,” says Swift’s Dr. Winkler.

Platform HPC enables Swift to schedule large computa-tions to be run in parallel so design engineers are not wait-

ing around for their analysis to complete overnight.

“Platform HPC is the ideal solution for Swift,” says Chief Engineer Chris Norris. “The management software enables Swift to solve bigger problems with more enhanced graphics at real time speed. Vast amounts of data can be processed 50 times faster.”

“Platform HPC is the ideal solution for Swift … The management software enables Swift to solve bigger problems with more enhanced graphics at real time speed. Vast amounts of data can be processed 50 times faster.”— Chris Norris, Chief Engineer | Swift Engineering

Page 10: DESKTOP ENGINEERING 5 Easy Steps to a High Performance …...5 Easy Steps to a High-Performance Cluster 6 Such was the case at Swift Engineering. Swift is an innovator in the design

5 Easy Steps to a High-Performance Cluster 10

Managed, Integrated Clusters Lower Cost of Ownership An integrated solution like Platform HPC does initially cost more than open-source management software. SMBs may be tempted to take the lowest-cost route and build a cluster using only open-source software. But when you consider the total cost of owner-

ship, that option loses its luster.

According to IBM Platform Computing, the total cost of ownership (TCO) of cluster management software for a 32-node cluster is about 35% more for a self-assembled approach, compared to Platform HPC.

A turnkey solution like Platform HPC improves TCO in a number of ways.

When initially setting up a cluster, an integrated software solution reduces the time it takes to put that pool of pro-

cessing power to work. Rather than accepting lost productivity during a long ramp-up time while various open-source tools are being configured, Platform HPC installs ev-erything at once and speeds configuration. In fact, IBM Platform Computing hosted a webinar (available on demand) that showed how users can get a cluster up and running within 24 hours.

The savings also comes from ease of job submission. Platform HPC has built-in tem-plates that make it fast and easy to integrate applications, including CAE applications such as those from ANSYS, MSC, Dassault Systèmes, and even home-grown applica-tions. A drag-and-drop form builder on the web interface allows the application tem-plates to be quickly customized for end users.

Platform HPC has built-in templates that make it fast and easy to integrate applications, including CAE applications such as those from ANSYS, MSC, Dassault Systèmes, and even home-grown applications.

Three-year management software TCO for a 32-node cluster.

■ Software Cost

■ Operation Cost

■ Setup Cost$120,000

$100,000

$80,000

$60,000

$40,000

$20,000

$0

SELF-ASSEMBLED

PLATFORM HPC

Page 11: DESKTOP ENGINEERING 5 Easy Steps to a High Performance …...5 Easy Steps to a High-Performance Cluster 6 Such was the case at Swift Engineering. Swift is an innovator in the design

5 Easy Steps to a High-Performance Cluster 11

Once the cluster is up and running, Platform HPC will schedule the workloads based on predefined policies to ensure the cluster is processing jobs at peak efficiency. Some self-assembled management solutions based on open source software lack this ability entirely — so all jobs go into one queue, regardless of size and the re-sources available at that time. In fact, 61% of Desktop En-gineering’s survey respondents are not using a job sched-uler or workload manager with their HPC cluster. That means they’re not operating nearly as efficiently as they could be. Underutilized resources lead to wasted cycle time and energy consumption, which brings higher cost of ownership. In contrast, Platform HPC schedules jobs according to policies you set as well as specific workload requirements, so your cluster resources can be used more efficiently and effectively, resulting in reduced total cost of ownership for your overall IT infrastructure.

In addition, Platform HPC is scalable, so you can be confident in your ability to easily add more cores as your needs evolve. This is critical for many SMBs, as engineering workloads can change dramatically based on new projects. According to the company, with Platform HPC, you can ramp up your cluster resources quickly and know it will support popular new applications that you may need in the future.

In short, the management concerns associated with HPC clusters can be addressed by an integrated management solution. Platform HPC is designed to relieve the burden of cluster management and return productivity to both IT managers and end users.

Examples of Platform HPC Application Templates• ANSYS Mechanical

• ANSYS Fluent

• ClustalW

• HMMER

• LS-DYNA

• MSC Nastran

• NCBI Blast

• NWChem

• Schlumberger ECLIPSE

• SIMULIA Abaqus

• Generic template for in-house/open source apps

Additional templates available for download.

Calculate Cluster CostsThe IBM Platform HPC Total Cost of Ownership (TCO) tool offers a 3-year total cost of ownership view of your distributed computing environment and savings that you could potentially experience by using IBM Platform HPC in place of competing cluster management software. This model can estimate savings with the deployment of intelligent cluster management software. Click here to use the TCO tool.The use of this simple tool does not substitute for detailed analysis. To have IBM perform a business value assessment for your environment, and provide a more accurate estimate of potential savings, please contact your IBM representative.

Page 12: DESKTOP ENGINEERING 5 Easy Steps to a High Performance …...5 Easy Steps to a High-Performance Cluster 6 Such was the case at Swift Engineering. Swift is an innovator in the design

5 Easy Steps to a High-Performance Cluster 12

NEXT STEPS TOWARD HPC CLUSTERS

Is an HPC cluster right for you? Many engineering problems can be solved much faster and with higher fidelity when massive amounts of computing power are accessible. After years of compounding price reductions and performance gains, easily managed HPC power is within the grasp of engineering teams at many SMBs.

If you think your organization might benefit from an HPC cluster, it’s time to take the next steps. Here is a must-do list for implementing a successful cluster:

1 Clearly define the engineering problems you need to solve, and then make sure a massive in-crease in computing power will solve those problems. For example, if you need to signifi-

cantly increase the number of high-fidelity design iterations that you simulate, or need much more realistic renderings of your production, clusters can deliver a great return on investment. If computing power will clearly improve engineering productivity and creativ-ity, and push your product to market faster, a cluster should be considered. Also consider adding other groups to share the problem-solving prowess of a cluster. As Desktop En-gineering’s subscriber survey shows, most of the time (70%) clusters are being used by multiple teams within the company running various types of applications.

2 Engage the person or people who will be managing the cluster. Explain the benefits of a cluster you have discovered in step 1, and then begin to craft a joint plan with IT or the per-

son who will manage the cluster. In the plan, outline the economic rationale that you will present to your managers and/or executive team. Clearly show how clusters can increase innovation and competitiveness while increasing productivity. Make sure you address steps

3-5 below in your proposal.

3 Define your hardware and software requirements. How much computing power and network speed your or-

ganization needs to solve your engineering challenges? Do you need a cluster with 16 nodes and 64 cores? Or 128 nodes and 1,024 cores? Or something else? And how fast does your network need to distribute the processing and results? You can turn to some of the many reputable clus-ter hardware providers in the market today to identify the optimal configurations for your needs.

Hardware and software choices go hand in hand. Ensure your applications will operate efficiently with the cluster and cluster management software. The options here are

continually increasing, and currently include offerings from Altair, ANSYS, MSC, Math-Works, Siemens PLM Software, Dassault Systèmes, SIMULIA and many custom appli-cations that engineers have written themselves. Vendors such as IBM are now providing workload optimized reference architecture platforms tailored for specific industry applica-tions.

“Platform HPC is an invaluable tool for us at HPC Wales. We needed a product that would not only be robust and reliable, but also easy to use, ensuring compute jobs would run smoothly.”

— Martyn Guest, Technical Director | HPC Wales

Page 13: DESKTOP ENGINEERING 5 Easy Steps to a High Performance …...5 Easy Steps to a High-Performance Cluster 6 Such was the case at Swift Engineering. Swift is an innovator in the design

4Decide whether you will run an unmanaged or managed cluster. With an unmanaged clus-ter, jobs are queued up and processed and submitted without regard to the cluster’s

efficiency. A managed cluster incorporates workload management policies and resource management capabilities that allow for easy and efficient job scheduling. While some may argue that an unmanaged cluster requires the least amount of overhead support, this is rarely true — and typically results in poor cluster utilization, wasted resources and lower productivity. In fact, a cluster managed by sophisticated infrastructure soft-ware does not require significant admin support. With integrated cluster management software that offers a complete set of functionalities, and is easy to use and well sup-ported, a part-time system administrator is often all many SMBs need.

5 Identify the best cluster management software for your organization. This is the secret ingredient to making your cluster a high-utilization asset that delivers enormous

value. For most SMBs, an integrated package from a proven vendor such as IBM prom-ises to be the best solution. Compared to open-source offerings, an established, inte-grated suite of cluster management software typically offers SMBs the best combination of ramp-up speed, ease of use and administration, engineering application support and integration, workload optimization, vendor support and scalability at a lower TCO. Integrated, easy-to-use cluster management software enables faster time to results and an increased competitive advantage for your firm.

If necessary, engage a local business partner from IBM to help you prove how a modern, all-in-one cluster solution can dramatically reduce specialized IT skills that open-source clusters require. You can also get guidance from engineering software providers to learn what kind of cluster configurations you will need to run your applications.

Page 14: DESKTOP ENGINEERING 5 Easy Steps to a High Performance …...5 Easy Steps to a High-Performance Cluster 6 Such was the case at Swift Engineering. Swift is an innovator in the design

5 Easy Steps to a High-Performance Cluster 14

APPENDIX

Glossary of terms • Accelerators: a computer processor designed to accelerate certain compute intensive algorithms or library functions. It can have its own memory and be placed alongside CPUs.

• Dynamic Provisioning: Assigning cluster resources automatically based on changing workload and job resource requirements.

• Failover: Avoiding downtime by ensuring another node in a cluster can take over the workload of a failed node without disrupting the job processing.

• Grid Computing: A loosely coupled, often geographically dispersed system of computer resources that run various types of workloads.

• InfiniBand: A high-throughput communication link that is often used to facilitate communications between nodes in an HPC cluster.

• Job Scheduling: A software capability designed to manage jobs running on a cluster to ensure they are being completed as efficiently as possible.

• Load Balancing: Distributing a workload across multiple nodes in a cluster to optimize efficiency and avoid overloading or underloading any particular node.

• Message Passing Interface (MPI): MPI is a language-independent communications protocol used to program parallel computing environment such as a cluster. Both point-to-point communication between compute nodes, and collective communication among multiple nodes are supported.2

• Node: A computer used as a server, often as part of a cluster.

• Parallel Software: An application capable of using more than one processing core simultaneously, often to divide and speed up graphically or computationally intensive functions.

• Software Stack: A type of software definition that can be used to identify groups of software to install at the same time and in a specific sequence on target systems. A software stack can include installable files and software definitions, patches, and other software stacks.

• Symmetric Multi-processing (SMP): An architecture in which two or more processors are connected to a shared memory, have access to all connected devices, and are controlled by a single operating system.

• Virtualization: In computing, to create an approximation of physical computing hardware or software, such as a hard drive partition, server or operating system, so that devices, applications and users can interact with it as if it were a distinct piece of hardware or software.

• Workload Management: A software solution enabling users to manage workload distributions in order to achieve optimal application performance while making the best use of computing resources, such as bandwidth and software licenses.

2. Source: Wikipedia

Page 15: DESKTOP ENGINEERING 5 Easy Steps to a High Performance …...5 Easy Steps to a High-Performance Cluster 6 Such was the case at Swift Engineering. Swift is an innovator in the design

5 Easy Steps to a High-Performance Cluster 15

Platform Computing website

IBM Technical Computing website

Clusters for Dummies e-book

IBM Platform Computing brochure

IBM Platform Computing Videos

White Paper: The Hidden Costs of Open Source

On-demand webinar: Build a cluster in 24 hours

Case Study: Swift Engineering

Case Study Video: Swift Engineering

Case Study: Red Bull Technology

Case Study Video: Red Bull Technology

Case Study: University of East Anglia

Case Study Video: Virginia Tech Transportation Institute

IBM Platform Computing FamilyIBM Platform HPC: Unlike the other, more discrete cluster management tools, this product family offers full cluster computing management. It includes workload management and cluster monitoring and provisioning capabilities, as well as application submission template and performance enhancement solutions. This single tool includes all you need to use and run a cluster.

IBM Platform LSF: This dedicated workload scheduler provides a set of policy-driven scheduling features that enable medium to large customers to use all their entire compute infrastructure resources and ensure optimal application performance.

IBM Platform MPI: MPI speeds the performance of parallel applications running on distributed computing platforms. It includes an implementation of Message Passing Interface (MPI), and supports a wide array of operating systems. It also interconnects, allowing parallel applications to run reliably on an array of systems and architectures. The core MPI functions are included in the Platform HPC integrated product.

IBM Platform Cluster Manager: This tool includes the key management functions of IBM Platform HPC, and is designed for monitoring, managing and provisioning clusters. The Web interface can be used to manage small, department clusters all the way up to HPC cloud configurations.

Additional Resources