HPCDeploymentGuide

Design and Deployment Guide for Windows HPC Server 2008

Microsoft Corporation

Published: September 2008

AbstractThis guide provides detailed information and step-by-step procedures for designing and installing

a high performance computing cluster using Windows® HPC Server 2008. You can use this guide

to first plan the deployment of your HPC cluster, and when you are ready to deploy you can follow

the procedures to configure the head node, add compute nodes to the cluster, and verify that your

cluster deployment has been successful.

Copyright Information

Information in this document, including URL and other Internet Web site references, is subject to

change without notice. Unless otherwise noted, the companies, organizations, products, domain

names, e-mail addresses, logos, people, places, and events depicted in examples herein are

fictitious. No association with any real company, organization, product, domain name, e-mail

address, logo, person, place, or event is intended or should be inferred. Complying with all

applicable copyright laws is the responsibility of the user. Without limiting the rights under

copyright, no part of this document may be reproduced, stored in or introduced into a retrieval

system, or transmitted in any form or by any means (electronic, mechanical, photocopying,

recording, or otherwise), or for any purpose, without the express written permission of Microsoft

Corporation.

Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual

property rights covering subject matter in this document. Except as expressly provided in any

written license agreement from Microsoft, the furnishing of this document does not give you any

license to these patents, trademarks, copyrights, or other intellectual property.

© 2008 Microsoft Corporation. All rights reserved.

Microsoft Active Directory, Windows, Windows PowerShell, Windows Server, and Windows Vista

are trademarks of the Microsoft group of companies.

All other trademarks are property of their respective owners.

Contents

Design and Deployment Guide for Windows HPC Server 2008.....................................................5

Checklist: Deploy an HPC Cluster (Overview)................................................................................5

Step 1: Prepare for Your Deployment.............................................................................................6

Step 2: Deploy the Head Node.....................................................................................................12

Step 3: Configure the Head Node.................................................................................................14

Step 4: Add Compute Nodes to the Cluster..................................................................................21

Step 5: Run Diagnostic Tests on the Cluster.................................................................................27

Step 6: Run a Test Job on the Cluster..........................................................................................27

Additional Resources.................................................................................................................... 31

Appendices................................................................................................................................... 31

Appendix 1: HPC Cluster Networking........................................................................................32

Appendix 2: Creating a Node XML File......................................................................................43

Appendix 3: Node Template Tasks and Properties....................................................................51

Appendix 4: Job Template Properties........................................................................................59

Appendix 5: Scriptable Power Control Tools..............................................................................65

Appendix 6: Using HPC PowerShell..........................................................................................67

Design and Deployment Guide for Windows HPC Server 2008

This guide provides conceptual information for planning the deployment of a high performance

computing cluster using Windows® HPC Server 2008. It also provides step-by-step procedures

for deploying the head node in your cluster, adding compute nodes, and for verifying that your

deployment was successful.

You can configure your HPC cluster for high availability by installing the head node in the

context of a failover cluster. If the server that is acting as the head node fails, the other

server in the failover cluster automatically begins acting as the head node (through a

process known as failover). For more information about running an HPC cluster with

failover clustering, see http://go.microsoft.com/fwlink/?LinkId=123894.

Checklist: Deploy an HPC Cluster (Overview)

The following checklist describes the overall process of designing and deploying a Windows HPC

Server 2008 cluster. Each task in the checklist is linked to the section in this document that

describes the steps to perform the task.

Task Description

Step 1: Prepare for Your Deployment Before you start deploying your HPC cluster,

review the list of prerequisites and initial

considerations.

Step 2: Deploy the Head Node Deploy the head node by installing Windows

Server 2008 and HPC Pack 2008.

Step 3: Configure the Head Node Configure the head node by following the steps

in the configuration to-do list.

Step 4: Add Compute Nodes to the Cluster Add nodes to the cluster by deploying them

from bare metal, by importing an XML file, or by

manually configuring them.

Step 5: Run Diagnostic Tests on the Cluster Run diagnostic tests to verify that the

deployment of the cluster was successful.

Step 6: Run a Test Job on the Cluster Run some basic jobs on the cluster to verify

that the cluster is operational.

Note

5

http://go.microsoft.com/fwlink/?LinkId=123894

Step 1: Prepare for Your Deployment

The first step in the deployment of your HPC cluster is to make important decisions, such as

deciding how you will be adding nodes to your cluster, and choosing a network topology for your

cluster. The following checklist describes the steps involved in preparing for your deployment.

Checklist: Prepare for your deployment

Task Description

1.1. Review initial considerations and system

requirements

Review the list of initial considerations and

system requirements to ensure that you have

all the necessary hardware and software

components to deploy an HPC cluster.

1.2. Decide how to add compute nodes to your

cluster

Decide if you will be adding compute nodes to

your cluster from bare metal, as preconfigured

nodes, or using an XML file.

1.3. Choose the Active Directory domain for

your cluster

Choose the Active Directory® domain to which

you will join the head node and compute nodes

of your HPC cluster.

1.4. Choose a user account for installation and

diagnostics

Choose an existing domain account with

enough privileges to perform installation and

diagnostics tasks.

1.5. Choose a network topology for your cluster Choose how the nodes in your cluster will be

connected, and how the cluster will be

connected to your enterprise network.

1.6. Prepare for multicast (optional) If you will be deploying nodes from bare metal

and want to multicast the operating system

image that you will be using during deployment,

configure your network switches appropriately.

1.7. Prepare for the integration of scriptable

power control tools (optional)

If you want to use your own power controls

tools to start, shut down, and reboot compute

nodes remotely, obtain and test all the

necessary components of your power control

tools.

6

1.1. Review initial considerations and system requirementsThe following sections list some initial considerations that you need to review, as well as

hardware and software requirements for Windows HPC Server 2008.

Initial considerationsReview the following initial considerations before you deploy your HPC cluster.

Compatibility with previous versions

The following list describes compatibility between Windows HPC Server 2008 and Windows

Compute Cluster Server 2003:

Windows HPC Server 2008 provides application programming interface (API)-level

compatibility for applications that are integrated with Windows Compute Cluster Server 2003.

These applications might, however, require changes to run on Windows Server® 2008. If you

encounter problems running your application on Windows Server 2008, you should consult

your software vendor.

Windows HPC Server 2008 supports job submission from Windows Compute Cluster Server

2003 clients, including jobs that are submitted through the use of the command-line tools, the

Compute Cluster Job Manager, and the COM APIs.

The Windows HPC Server 2008 client tools, including the cluster administration console

(HPC Cluster Manager), the job scheduling console (HPC Job Manager), the command-line

tools, and the APIs cannot be used to manage or submit jobs to a Windows Compute Cluster

Server 2003 cluster.

Clusters that have both Windows Compute Cluster Server 2003 nodes and Windows HPC

Server 2008 nodes are not supported.

A side-by-side installation of Windows HPC Server 2008 and Windows Compute Cluster

Server 2003 on the same computer is not supported. This includes the Windows HPC Server

2008 client utilities.

The upgrade of a Windows Compute Cluster Server 2003 head node to a Windows HPC

Server 2008 head node is not supported.

Server roles added during installation

The installation of HPC Pack 2008 adds the following server roles to the head node:

Dynamic Host Configuration Protocol (DHCP) Server, to provide IP addresses and related

information for compute nodes.

Windows Deployment Services, to deploy compute nodes remotely.

File Services, to manage shared folders.

7

Network Policy and Access Services, which enables Routing and Remote Access so that

network address translation (NAT) services can be provided to the cluster nodes.

Hardware requirementsHardware requirements for Windows HPC Server 2008 are very similar to those for the 64-bit

editions of Windows Server 2008.

For more information about installing Windows Server 2008, including system

requirements, see Installing Windows Server 2008 (http://go.microsoft.com/fwlink/?

LinkID=119578).

Processor (x64-based):

Minimum: 1.4 GHz

Recommended: 2 GHz or faster

RAM:

Minimum: 512 MB

Recommended: 2 GB or more

Available disk space:

Minimum: 50 GB

Recommended: 80 GB or more

Drive:

DVD-ROM drive

Network adapters:

The number of network adapters on the head node and on the compute nodes depends on

the network topology that you choose for your cluster. For more information about the

different HPC cluster network topologies, see Appendix 1: HPC Cluster Networking.

Software requirementsThe following list outlines the software requirements for the head node and the compute nodes in

a Windows HPC Server 2008 cluster:

Windows Server 2008 HPC Edition, or another 64-bit edition of Windows Server 2008

Microsoft HPC Pack 2008

Microsoft HPC Pack 2008 cannot be installed on any edition of Windows Server 2008 R2.

It can only be installed on Windows Server 2008 HPC Edition, or another 64-bit edition of

Windows Server 2008.

To enable users to submit jobs to your HPC cluster, you can install the utilities included with

Microsoft HPC Pack 2008 on client computers. Those client computers must be running any of

the following operating systems:

Windows XP Professional with Service Pack 3 or later (x86- or x64-based)

Note Important

8

http://go.microsoft.com/fwlink/?LinkID=119578

Windows Vista® Enterprise, Windows Vista Business, Windows Vista Home, or Windows

Vista Ultimate

Windows Server 2003 Standard Edition or Windows Server 2003 Enterprise Edition with

Service Pack 2 or later (x86- or x64-based)

Windows Server 2003, Compute Cluster Edition

Windows Server 2003 R2 Standard Edition or Windows Server 2003 R2 Enterprise Edition

(x86- or x64-based)

1.2. Decide how to add compute nodes to your clusterThere are three ways to add compute nodes to your cluster:

From bare metal. The operating system and all the necessary HPC cluster components are

automatically installed on each compute node as it is added to the cluster. No manual

installation of the operating system or other software is required.

Add preconfigured compute nodes. The compute nodes are already running Windows

Server 2008 HPC Edition, or another 64-bit edition of Windows Server 2008, and Microsoft

HPC Pack 2008 is manually installed on each node.

Import a node XML file. An XML file that contains a list of all the nodes that will be deployed

is used. This XML file can be used to add nodes from bare metal or from preconfigured

nodes. For more information about node XML files, see Appendix 2: Creating a Node XML

File.

The following is a list of details to take into consideration when choosing how to add nodes to

your HPC cluster:

When deploying nodes from bare metal, Windows HPC Server 2008 automatically generates

computer names for your compute nodes. During the configuration process, you will be

required to specify the naming convention to use when automatically generating computer

names for the new nodes.

Compute nodes are assigned their computer name in the order that they are deployed.

If you want to add compute nodes from bare metal and assign computer names in a different

way, you can use a node XML file. For more information about node XML files, see Appendix

2: Creating a Node XML File.

If you want to add preconfigured nodes to your cluster, you will need to install Windows

Server 2008 HPC Edition, or another 64-bit edition of Windows Server 2008 on each node (if

not already installed), as well as Microsoft HPC Pack 2008.

9

1.3. Choose the Active Directory domain for your clusterThe head node and the compute nodes in your HPC cluster must be members of an Active

Directory domain. Before deploying your cluster, you must choose the Active Directory domain

that you will use for your HPC cluster.

If you do not have an Active Directory domain to which you can join your cluster, or if you prefer

not to join an existing domain, you can install the Active Directory Domain Services role on the

head node and then configure a domain controller on that node. For more information about

installing the Active Directory Domain Services role on a computer that is running Windows

Server 2008, see the AD DS Installation and Removal Step-by-Step Guide

(http://go.microsoft.com/fwlink/?LinkID=119580).

If you choose to install and configure an Active Directory domain controller on the head

node, consult with your network administrator about the correct way to isolate the new

Active Directory domain from the enterprise network, or how to join the new domain to an

existing Active Directory forest.

1.4. Choose a user account for installation and diagnosticsDuring the configuration process of your HPC cluster, you must provide credentials for a domain

user account that will be used for installation and diagnostics. You must choose an existing

account or create a new account, before starting your cluster deployment.

The following is a list of details to take into consideration when choosing the user account:

The user account that you choose must be a domain account with enough privileges to create

Active Directory computer accounts for the compute nodes. Alternatively, you can create the

computer accounts manually or ask your domain administrator to create them for you.

If part of your deployment requires access to resources on the enterprise network, the user

account must have the necessary permissions to access those resources—for example,

installation files that are available on a network server.

If you want to restart nodes remotely from the cluster administration console (HPC Cluster

Manager), the account must be a member of the local Administrators group on the head

node. This requirement is only necessary if you do not have scriptable power control tools

that you can use to remotely restart the compute nodes.

1.5. Choose a network topology for your clusterWindows HPC Server 2008 supports five cluster topologies. These topologies are distinguished

by how the compute nodes in the cluster are connected to each other and to the enterprise

network. The five supported cluster topologies are:

Topology 1: Compute Nodes Isolated on a Private Network

Caution

10


Topology 2: All Nodes on Enterprise and Private Networks

Topology 3: Compute Nodes Isolated on Private and Application Networks

Topology 4: All Nodes on Enterprise, Private, and Application Networks

Topology 5: All Nodes on an Enterprise Network

For more information about each network topology, see Appendix 1: HPC Cluster Networking.

When you are choosing a network topology, you must take into consideration your existing

network infrastructure:

Decide which network in the topology that you have chosen will serve as the enterprise

network, the private network, and the application network.

Do not have the network adapter that is connected to the enterprise network on the head

node in an automatic configuration (that is, the IP address for that adapter does not start with:

169.254). That adapter must have a valid IP address, dynamically or manually assigned

(static).

If you choose a topology that includes a private network, and you are planning to add nodes

to your cluster from bare metal:

Ensure that there are no Pre-Boot Execution Environment (PXE) servers on the private

network.

If you want to use an existing DHCP server for your private network, ensure that it is

configured to recognize the head node as the PXE server in the network.

If you want to enable DHCP server on your head node for the private or application networks

and there are other DHCP servers connected to those networks, you must disable those

DHCP servers.

If you have an existing Domain Name System (DNS) server connected to the same network

as the compute nodes, no action is necessary, but the compute nodes will be automatically

deregistered from that DNS server.

Contact your system administrator to determine if Internet Protocol security (IPsec) is

enforced on your domain through Group Policy. If IPsec is enforced on your domain through

Group Policy, you may experience issues during deployment. A workaround is to make your

head node an IPsec boundary server so that compute nodes can communicate with the head

node during PXE boot.

1.6. Prepare for multicast (optional)If you will be deploying nodes from bare metal and want to multicast the operating system image

that you will be using during deployment, we recommend that you prepare for multicast by:

Enabling Internet Group Management Protocol (IGMP) snooping on your network switches, if

this feature is available. This will help to reduce multicast traffic.

Disabling Spanning Tree Protocol (STP) on your network switches, if this feature is enabled.

For more information about these settings, contact your network administrator or your

networking hardware vendor.

Note

11

1.7. Prepare for the integration of scriptable power control tools (optional)The cluster administration console (HPC Cluster Manager) includes actions to start, shut down,

and reboot compute nodes remotely. These actions are linked to a script file (CcpPower.cmd) that

performs these power control operations using operating system commands. You can replace the

default operating system commands in that script file with your own power control scripts, such as

Intelligent Platform Management Interface (IPMI) scripts that are provided by your vendor of

cluster solutions.

In preparation for this integration, you must obtain all the necessary scripts, dynamically linked

library (DLL) files, and all other components of your power control tools. After you have obtained

all the necessary components, test them independently and ensure that they work as intended on

the computers that you will be deploying as compute nodes in your cluster.

For information about modifying CcpPower.cmd to integrate your own scriptable power control

tools, see Appendix 5: Scriptable Power Control Tools.

Step 2: Deploy the Head Node

The next step in the deployment of your HPC cluster is to deploy the head node. The following

checklist describes the steps involved in deploying the head node.

Checklist: Deploy the head node

Task Description

2.1. Install Windows Server 2008 on the head

node computer

Install Windows Server 2008 HPC Edition, or

another 64-bit edition of Windows Server 2008

on the computer that will act as the head node.

2.2. Join the head node computer to a domain Join the computer that will act as the head node

to a Microsoft Active Directory Domain.

2.3. Install Microsoft HPC Pack 2008 on the

head node computer

Install Microsoft HPC Pack 2008 on the

computer that will act as the head node, using

the installation media or from a network

location.

12

2.1. Install Windows Server 2008 on the head node computerTo deploy the head node of your HPC cluster, you must start by installing Windows Server 2008

HPC Edition, or another 64-bit edition of Windows Server 2008 on the computer that will act as

the head node. For more information about installing Windows Server 2008, including system

requirements, see Installing Windows Server 2008 (http://go.microsoft.com/fwlink/?

LinkID=119578).

We strongly recommend that you perform a clean installation of Windows Server 2008

before installing Microsoft HPC Pack 2008. If you want to install Microsoft HPC

Pack 2008 on an existing installation of Windows Server 2008, remove all server roles

first and then follow the procedures in this guide.

It is recommended that you obtain the latest device drivers for your head node computer

from the Web site of your hardware vendors.

2.2. Join the head node computer to a domainAs described in the Step 1: Prepare for Your Deployment section, the head node must be a

member of an Active Directory domain. After you have installed Windows Server 2008 on the

head node, manually join the head node to an existing Active Directory domain.

2.3. Install Microsoft HPC Pack 2008 on the head node computerAfter Windows Server 2008 is installed on the head node computer, and the head node is joined

to an Active Directory domain, you can install Microsoft® HPC Pack 2008 on the head node.

1. To start the Microsoft HPC Pack 2008 installation wizard on the computer that will act as

the head node, run setup.exe from the HPC Pack 2008 installation media or from a

network location.

2. On the Getting Started page, click Next.

3. On the Microsoft Software License Terms page, read or print the software license

terms in the license agreement, and accept or reject the terms of that agreement. If you

accept the terms, click Next.

4. On the Select Installation Type page, click Create a new HPC cluster by creating a

head node, and then click Next.

5. Continue to follow the steps in the installation wizard.

Important Note To install Microsoft HPC Pack 2008 on the head node computer

13


Step 3: Configure the Head Node

After you have deployed the head node of your HPC cluster, you must configure the head node

by following the configuration to-do list in HPC Cluster Manager.

Checklist: Configure the head nodeThe following checklist includes the items in the configuration to-do list in HPC Cluster Manager

that you need to complete in order to configure your head node.

Task Description

3.1. Configure the HPC cluster network Configure the cluster network by using the

Network Configuration Wizard.

3.2. Provide installation credentials Specify which credentials to use for system

configuration and when adding new nodes to

the cluster.

3.3. Configure the naming of new nodes Specify the naming convention to use when

generating names automatically for new

compute nodes.

3.4. Create a node template Create a template that defines the steps to

follow when configuring a compute node.

3.5. Add drivers for the operating system

images (optional)

If you will be deploying compute nodes from

bare metal and those nodes require special

device drivers, add drivers for the operating

system images that you created for your node

template on the previous task.

3.6. Add or remove users (optional) If you will be giving access to the cluster to

other members of your organization, add or

remove users or administrators for your cluster.

3.1. Configure the HPC cluster networkThe HPC cluster network configuration is the first step in the configuration process of your head

node. The HPC cluster network is configured by following the Network Configuration Wizard in

HPC Cluster Manager. When configuring the HPC cluster network, you must choose the network

topology that you have selected for your cluster, as described in “1.5. Choose a network topology

for your cluster”, in Step 1: Prepare for Your Deployment.

Before you start configuring the HPC cluster network in HPC Cluster Manager, ensure

that the head node and the computers that you will add as compute nodes to the cluster

Important

14

are physically connected according to the network topology that you have chosen for

your cluster. Also, ensure that you are able to identify to which network each one of the

network adapters in the head node is connected.

1. If HPC Cluster Manager is not already open on the head node, open it. Click Start, point

to All Programs, click Microsoft HPC Pack, and then click HPC Cluster Manager.

2. In the To-do List, click Configure your network. The Network Configuration Wizard

appears.

3. On the Network Topology Selection page, click the topology that you have chosen for

your cluster, and then click Next.

4. On the Enterprise Network Adapter Selection page, in the Network adapter list, click

the name of the network adapter that is physically connected to your enterprise network,

and then click Next.

Important

To ensure that you are selecting the correct network adapter, use the

information displayed on this wizard page after you select a network adapter

from the list. Use the IP address, domain information, and Media Access

Control (MAC) address of each adapter as a reference.

5. If you chose topology number 5 for your cluster, jump to step 9 in this procedure.

Otherwise, repeat step 4 for the private network adapter.

6. On the Private Network Configuration page, type a static IP address and a subnet

mask for the head node. Optionally, select network services for that network:

a. To give access to resources on the enterprise network to compute nodes that are

connected to this network, select the Enable network address translation (NAT) on

the head node check box.

b. To enable DHCP services for the nodes connected to this network, select the Enable

DHCP and define a scope check box, and then type the starting and ending IP

addresses for the DHCP scope. If the Gateway and DNS server IP addresses have

not been automatically detected, type each of these addresses.

Note

For more information about enabling NAT and DHCP on your cluster

network, see “HPC network services” in Appendix 1: HPC Cluster

Networking.

7. Click Next after you are done configuring the private network.

8. Repeat steps 4, 6, and 7 for the application network adapter. Click Next after you are

done configuring the application network.

9. On the Firewall Setup page, select the firewall setting for the cluster:

a. To apply firewall settings automatically to head nodes and compute nodes on each

network, click ON for that network.

To configure the HPC cluster network

15

b. To disable the firewall on a network, click OFF.

c. If you do not want to change any firewall settings, click Do not manage firewall

settings.

Note

For more information about firewall settings for your cluster, see “Windows

Firewall configuration” in Appendix 1: HPC Cluster Networking.

10. On the Review page, verify your settings and click Configure. If you want to change any

of the settings, navigate to the appropriate wizard page by clicking it on the navigation

pane or by clicking Previous.

11. After the network configuration process is completed, on the Configuration Summary

page, review the list of configuration items. If you want to save a report of the network

configuration, click Save the configuration report.

12. To close the wizard, click Finish.

3.2. Provide installation credentialsInstallation credentials must be provided in order to configure new compute nodes. These

credentials will be used when installing the operating system, applications, and when adding

nodes to the Active Directory domain. Also, these same credentials will be used when running

diagnostic tests on the cluster nodes. For more information, see “1.4. Choose a user account for

installation and diagnostics” in Step 1: Prepare for Your Deployment.

1. In the To-do List, click Provide installation credentials. The Installation Credentials

dialog box appears.

2. Type the user name, including the domain (DOMAIN\User), and then the password for

the domain user account you will use to deploy compute nodes and to run diagnostic

tests.

Important

The account must be a domain account with enough privileges to create

Active Directory computer accounts for the compute nodes. Alternatively, you

can create the computer accounts manually or ask your domain administrator

to create them for you.

Important

If part of your deployment requires access to resources on the enterprise

network, the account should have the necessary permissions to access

those resources.

Important

If you want to restart nodes remotely from the cluster administration console

(HPC Cluster Manager), the account must be added as an HPC cluster

To provide installation credentials

16

administrator on the head node. This requirement is only necessary if you do

not have scripted power control tools that you can use to remotely restart the

compute nodes.

3. To save the specified credentials, click OK.

3.3. Configure the naming of new nodesIf you deploy compute nodes from bare metal, and you are not using a node XML file to import

nodes to the cluster, Windows HPC Server 2008 will automatically generate computer names for

the new nodes that are being deployed. You need to specify how those names will be generated,

by defining a naming series.

The naming series is defined by selecting a root name and the starting number that will

accompany that name. The starting number is enclosed in percentage signs (%). For example:

ClusterNode%1000%.

When you deploy compute nodes from bare metal, nodes will be named in sequence, as they

become available. For example, if you deploy three nodes after specifying the following naming

series: ClusterNode-%100%, those nodes will be assigned these names:

ClusterNode-100

ClusterNode-101

ClusterNode-102

Compute node names are limited to 15 characters. When specifying the compute node

naming series, take into account the number of compute nodes in your deployment and

ensure that the series that you specify will not generate names that exceed 15

characters. For example, if your deployment will consist of 1,000 compute nodes, and

your starting number is 1, your root name cannot have more than 12 characters;

otherwise, your node number 1,000 will need a name that consists of 16 characters.

1. In the To-do List, click Configure the naming of new nodes. The Specify Compute

Node Naming Series dialog box appears.

2. Type the naming series that you want to use. The preview helps you to see an example

of how the naming series will be applied to the names of the compute nodes.

Note

You cannot specify a compute node naming series that consists only of

numbers.

3. To save the compute node naming series that you have specified, click OK.

3.4. Create a node templateNode templates are new in Windows HPC Server 2008. They define the necessary tasks for

configuring and adding compute nodes to your cluster. With a node template, you can deploy an

Important To specify the compute node naming series

17

operating system image, add specific drivers and software to compute nodes, or simply add a

preconfigured node to your cluster. Because you might have more than one type of compute

node, or you may be adding compute nodes to your cluster in different ways, you can create

different templates that apply to different nodes or situations.

You can create two types of node templates:

With an operating system image. This type of template includes a step to deploy an

operating system on the compute nodes. Use this type of template when adding compute

nodes from bare metal.

Without an operating system image. This type of template is used to add preconfigured

compute nodes to the cluster, or to update existing nodes.

The type of template that you create for the initial deployment of your HPC cluster depends on

how you decided to add compute nodes to your cluster. For more information, see “1.2. Decide

how to add compute nodes to your cluster” in Step 1: Prepare for Your Deployment.

If you will create a node template with an operating system image, you will need the

installation media for Windows Server 2008 HPC Edition or another 64-bit edition of

Windows Server 2008, or you must have the installation files available on a network

location that is accessible from the head node computer.

1. In the To-do List, click Create a node template. The Create Node Template Wizard

appears.

2. On the Specify Template Name page, type a descriptive name for the template, and

then click Next.

3. If you will be adding compute nodes to your cluster from bare metal:

a. On the Select Deployment Type page, click With operating system, and then click

Next.

b. If the operating system image that you want to use for your deployment is already

listed in the Image Name list, click that image and then jump to step 3.f. If you want

to use a different operating system image, on the Select Operating System Image

page, click Add Image.

c. On the Add Operating System Image window, click Create a new operating

system image, and then type or browse to the location of the Windows setup file for

one of the 64-bit editions of Windows Server 2008.

d. Type a descriptive name for the new operating system image, and then click OK.

e. After the image is added, in the Image Name list, click the image that you want to

use with the template.

f. Optionally, specify if you want to multicast the operating system image during

deployment. For more information, see “1.6. Prepare for multicast (optional)” in Step

1: Prepare for Your Deployment.

g. Optionally, specify if you want to include a product key to activate the operating

system on the compute nodes, and then type the product key that should be used.

Important To create a node template

18

h. Click Next to continue.

i. On the Specify Local Administrator Password for Compute Node page, click Use

a specific password, and then type and confirm the password that you want to use.

j. Click Next to continue, and then jump to step 5 in this procedure.

4. If you will be adding preconfigured compute nodes to your cluster, on the Select

Deployment Type page, click Without operating system, and then click Next.

5. On the Specify Windows Updates page, specify if you want to add a step in the

template to download and install updates using Microsoft Update or the enterprise

Windows Server Update Services (WSUS). Also, you can specify specific updates to be

added to the template. Click Next to continue.

6. On the Review page, click Create.

Note

The node templates that you create with the Create Node Template Wizard include

the most common deployment and configuration tasks. You can add more tasks to

the node templates that you create by using the Node Template Editor. For more

information, see Appendix 3: Node Template Tasks and Properties.

3.5. Add drivers for the operating system images (optional)If you will be deploying compute nodes from bare metal and those nodes require special device

drivers, you will need to add those drivers during the configuration process of your head node.

Drivers must be in the .inf format, and must be accessible from the head node.

The device drivers that you add will be available to all operating system images in the

image store.

It is recommended that you obtain the latest device drivers for your compute nodes from

the Web site of your hardware vendors.

1. In the To-do List, click Manage drivers. The Manage Drivers dialog box appears.

2. To add a driver, click Add.

3. Type or browse to the location of the setup information file for the driver that you want to

add (.inf format), and then click Open.

4. Repeat the two previous steps for all the drivers that you want to add.

5. After you are done adding drivers, click Close.

Note Note To add drivers for the operating system images

19

3.6. Add or remove users (optional)If you will be giving access to the cluster to other members of your organization, you need to add

them as HPC cluster users or HPC cluster administrators. Also, you can remove users or

administrators that were added by default during installation.

The Domain Users group is added as an HPC cluster user during installation. If you do

not want all users in the domain to have access to your cluster, you can remove the

Domain Users group from the list of HPC cluster users, and add a different domain

group specifically created for users of your HPC cluster, or you can add individual domain

users.

1. In the To-do List, click Add or remove users.

2. To add a user to the cluster:

a. In the Actions pane, click Add User. The Select Users or Groups dialog box

appears.

b. Type the user name of the user that you want to add, and then click Check Names.

For more information, on the Select Users or Groups window, click examples.

c. Repeat the previous step for all users that you want to add.

d. After you are done adding users, click OK.

3. To add an administrator to the cluster:

a. In the Actions pane, click Add Administrator. The Select Users or Groups dialog

box appears.

b. Type the user name of the administrator that you want to add, and then click Check

Names. For more information, on the Select Users or Groups window, click

examples.

c. Repeat the previous step for all administrators that you want to add.

d. After you are done adding administrators, click OK.

4. To remove a user or administrator, select it on the Users list, and then click Remove.

Note

You cannot remove the domain Administrator account from the list of cluster

administrators.

Step 4: Add Compute Nodes to the Cluster

Windows HPC Server 2008 simplifies the deployment process of compute nodes by providing

automatic node imaging, automatic naming of nodes, and other capabilities to streamline

Important To add or remove users for the cluster

20

deployment tasks. Also, it provides tools that you can use to monitor the progress of your

deployment.

Unlike previous versions of Windows HPC Server 2008, the default in Windows HPC

Server 2008 is to respond only to Pre-Boot Execution (PXE) requests that come from

existing compute nodes. This default setting is automatically changed when you use the

Add Node Wizard to add nodes from bare metal. Also, you can manually change this

setting in the Options menu, under Deployment Settings.

After creating a node template, you can use the Add Node Wizard to add compute nodes to your

HPC cluster. There are three ways by which you can add compute nodes to your cluster:

Deploy compute nodes from bare metal

Add compute nodes by importing a node XML file

Add preconfigured compute nodes

For more information about each of these three node deployment options, see “1.2. Decide how

to add compute nodes to your cluster” in Step 1: Prepare for Your Deployment.

In this section:

4.1. Deploy Compute Nodes from Bare Metal

4.2. Add Compute Nodes by Importing a Node XML File

4.3. Add Preconfigured Compute Nodes

4.4. Monitor deployment progress

4.5 Cancel the deployment of a node

4.1. Deploy compute nodes from bare metalThe following procedure describes how to add compute nodes to your HPC cluster from bare

metal, by using a node template that includes a step to deploy an operating system image.

To complete this procedure, you must have a template that includes a step to deploy an

operating system image. If you do not have a template that includes a step to deploy an

operating system image, create one by following the steps in “3.4. Create a node

template”, in Step 3: Configure the Head Node.

Before turning on a compute node for this procedure, verify in the configuration of the

BIOS of that computer that the compute node will boot from the network adapter that is

connected to the private network, instead of booting from the local hard drive or another

device, and that Pre-boot Execution Environment (PXE) boot is enabled for that network

adapter.



2. In Node Management, in the Actions pane, click Add Node. The Add Node Wizard

appears.

Important Important Important To deploy compute nodes from bare metal

21

3. On the Select Deployment Method page, click Deploy compute nodes from bare

metal using an operating system image, and then click Next.

4. On the Select New Nodes page, in the Node template list, click the name of a node

template that includes a step to deploy an operating system image.

5. Turn on the computers that you want to add as compute nodes to your cluster.

Computers will be listed on the Add Node Wizard as they contact the head node during

PXE boot. They will be named using the naming series that you specified when you

configured the head node. For more information, see “3.3. Configure the naming of new

nodes” in Step 3: Configure the Head Node.

6. When all computers that you have turned on are listed, click Select all, and then click

Deploy. If you see a node that you do not want to deploy at this time, you can unselect it.

7. On the Completing the Add Node Wizard page, if you will be deploying more nodes,

click Continue responding to all PXE requests. If you will not be deploying more

nodes, click Respond only to PXE requests that come from existing compute nodes.

8. To monitor deployment progress, select the Go to Node Management to track

progress check box, and then click Finish. For more information, see 4.4. Monitor

deployment progress.

4.2. Add compute nodes by importing a node XML fileThe following procedure describes how to add compute nodes by importing a node XML file.

To complete this procedure, you must have a valid node XML file. For more information,

see Appendix 2: Creating a Node XML File.




appears.

3. On the Select Deployment Method page, click Import compute nodes from a node

XML file, and then click Next.

4. On the Select Node XML File page, type or browse to the location of the node XML file,

and then click Import.

5. To monitor deployment progress, on the Completing the Add Node Wizard page, select

the Go to Node Management to track progress check box, and then click Finish. For

more information, see 4.4. Monitor deployment progress.

Important To add compute nodes by importing a node XML file

22

4.3. Add preconfigured compute nodesA preconfigured compute node is a computer that has HPC Pack 2008 already installed and that

is connected to the HPC cluster network according to the network topology that you have chosen

for your cluster. After HPC Pack 2008 is installed on all the compute nodes that you want to add

to your cluster, you can use the Add Node Wizard on the head node to add the preconfigured

nodes to your cluster.

The following procedures describe how to add preconfigured compute nodes to your HPC cluster.

The first procedure describes how to install HPC Pack 2008 on the computers that will act as

compute nodes, and the second procedure describes how to add the preconfigured compute

nodes to the cluster.

The computers that you will add to your cluster as preconfigured compute nodes must

already be running Windows Server® 2008 HPC Edition, or another 64-bit edition of the

Windows Server 2008 operating system.

We strongly recommend that you perform a clean installation of Windows Server 2008

before installing HPC Pack 2008. If you want to install HPC Pack 2008 on an existing

installation of Windows Server 2008, remove all server roles first and then follow the

procedures in this guide.

To complete this procedure, you must have a node template that does not include a step

to deploy an operating system image. If you do not have a node template that does not

include a step to deploy an operating system image, create one by following the steps in

“3.4. Create a node template”, in Step 3: Configure the Head Node.

1. To start the HPC Pack 2008 installation wizard on the computer that will act as a compute

node, run setup.exe from the HPC Pack 2008 installation media or from a network

location.

2. On the Getting Started page, click Next.

3. On the Microsoft Software License Terms page, read or print the software license

terms in the license agreement, and accept or reject the terms of that agreement. If you

accept the terms, click Next.

4. On the Select Installation Type page, click Join an existing HPC cluster by creating a

new compute node, and then click Next.

5. On the Join Cluster page, type the computer name of the head node on your cluster,

and then click Next.

6. On the Select Installation Location page, click Next.

7. On the Install Required Components page, click Install.

8. On the Installation Complete page, click Close.

After HPC Pack 2008 is installed on all the compute nodes that you want to add to your cluster,

follow the steps in the Add Node Wizard on the head node to add the preconfigured nodes to

your cluster.

Important Important Important To install HPC Pack 2008 on a compute node computer

23




appears.

3. On the Select Deployment Method page, click Add compute nodes that have already

been configured, and then click Next.

4. Turn on all the preconfigured nodes that you want to add to your cluster.

5. After all the preconfigured nodes are turned on, on the Before Deploying page, click

Next.

6. On the Select New Nodes page, in the Node template list, click the name of a node

template that does not include a step to deploy an operating system image.

7. Select the preconfigured compute nodes that you want to add to your cluster. To select all

the preconfigured compute nodes, click Select all.

8. To add the selected compute nodes to your cluster, click Add.

9. On the Completing the Add Node Wizard page, if you will be deploying more nodes,

click Continue responding to all PXE requests. If you will not be deploying more

nodes, click Respond only to PXE requests that come from existing compute nodes.

10. To monitor deployment progress, select the Go to Node Management to track

progress check box, and then click Finish. For more information, see 4.4. Monitor

deployment progress.

4.4. Monitor deployment progressDuring the deployment process of a compute node, its state is set to Provisioning. After the

deployment process is complete, the state changes to Offline. You must bring compute nodes

online before they can process jobs.

You can monitor the progress of the deployment process of compute nodes in Node

Management, and bring online nodes that have finished deploying. You can also see detailed

information for each deployment operation, and any errors that may have occurred.



2. To view information about the deployment operations:

a. In Node Management, in the Navigation Pane, click Operations.

b. To view more information about a specific operation, click that operation. The Detail

Pane will list the log entries for that operation.

3. To view the list of compute nodes that are currently being deployed:

a. In Node Management, in the Navigation Pane, under Nodes, under By State, click

Provisioning.

To add preconfigured compute nodes to your cluster To monitor deployment progress

24

b. To view the list of operations related to the deployment of a specific node, double-

click that node, and then click the Operations tab.

4. To bring online the nodes that have finished deploying:

a. In Node Management, in the Navigation Pane, under Nodes, under By State, click

Offline.

b. Select all the nodes that you want to bring online. To select all nodes that are

currently offline, on the list of offline nodes, click any node and then press CTRL+A.

c. In the Actions pane, click Bring Online.

5. If the deployment process of a compute node fails, the state of that node is set to

Unknown and the health is set to Provisioning Failed. To determine the reason of the

failure, review the provisioning log for that node and the list of operations that were

performed:

a. In Node Management, in the Navigation Pane, under Nodes, under By Health,

click Provisioning Failed.

b. To review the provisioning log for a node, in the views pane, click the node, and then

in the Detail Pane, click the Provisioning Log tab

c. To view the list of operations related to the deployment failure, on the Properties tab,

click View operations. The pivoted view in Node Management will list all the

operations related to that node.

d. To view more information about a specific operation, click that operation. The Detail

Pane will list the log entries for that operation.

4.5 Cancel the deployment of a nodeYou can stop the deployment process of a compute node from HPC Cluster Manager by

canceling the provisioning operations.

1. To view only compute nodes that are currently being deployed, in Node Management, in

the Navigation Pane, under Nodes, under By State, click Provisioning.

2. In the views pane, click the node that you want to stop deploying.

3. To cancel the provisioning operations, in the Detail Pane, on the Properties tab, click

Cancel Operations. The deployment process will stop, the node will be moved to the

Unknown state, and the health for that node will be changed to Provisioning Failed.

To cancel the deployment of a node

25

Step 5: Run Diagnostic Tests on the Cluster

After you have configured your head node and added all compute nodes to the cluster, you

should run diagnostic tests to validate cluster functionality and troubleshoot any configuration

issues.



2. In Configuration, in the Navigation Pane, click To-do List.

3. In the To-do List, click Validate your cluster (under Diagnostics).

4. In the Run Diagnostics dialog box, ensure that the Run all functional tests and All

nodes options are selected, and then click Run.

5. To view the progress of the diagnostic tests and the test results, in Diagnostics, in the

Navigation Pane, click Test Results.

6. To view detailed information about a test, double-click the test. To expand the information

in a section of the test results, click the down arrow for that section.

Step 6: Run a Test Job on the Cluster

After you have finished deploying your cluster, you can run a simple test job to verify that your

cluster is fully functional. The following checklist describes the steps involved in running a simple

test job on your cluster.

Checklist: Run a test job on the HPC cluster

Task Description

6.1. Create a Job Template Create a job template by running the Generate

Job Template Wizard in HPC Cluster

Manager.

6.2. Create and Submit a Job Create and submit a basic job in HPC Cluster

Manager.

6.3. Create and Submit a Job Using the

Command-Line Interface (Optional)

Create and submit a basic job by using the

HPC command-line tools.

6.4. Create and Submit a Job Using HPC

PowerShell (Optional)

Create and submit a basic job by using the

cmdlets in HPC PowerShell.

To run diagnostic tests on the cluster

26

6.1. Create a job templateJob templates simplify the job management of your HPC cluster by helping you to limit the kinds

of jobs that can be submitted to your cluster, what resources are assigned to jobs, and which

users can submit jobs. HPC Cluster Manager includes the Generate Node Template Wizard to

help you create basic job templates



2. In Configuration, in the Navigation Pane, click Job Templates.

3. In the Actions pane, click New. The Generate Node Template Wizard appears.

4. On the Enter Template Name page, type Test Template for the name of the new job

template, and optionally a description. Click Next to continue.

5. On the Limit Run Time page, select the Run jobs no longer than check box, and then

click Next without changing any settings. This will limit all jobs that are submitted using

this template to run for no longer than one minute.

6. On the Set Priorities page, click Next without changing any settings. This will run jobs

that are submitted using this template with Normal priority.

7. On the Set Project Names page, click Next without changing any settings. This will allow

jobs from any project to be submitted using this template.

8. On the Limit Node Groups page, click Next without changing any settings. This will

allow jobs that are submitted using this template to run on any node group.

9. On the Finish page, click Finish.

6.2. Create and submit a jobThis section describes how to submit a job in HPC Cluster Manager that:

Displays a directory list of the files in the C:\Program Files folder of a compute node in your

cluster.

Uses the job template that you created in the previous section, which limits to 1 minute the

maximum duration of time that a job can run.

Runs at low priority.

1. In Job Management, in the Actions pane, click New Job.

2. In Job Details, specify the following job parameters:

a. In the Job template list, click Test Template (the template that you created in

section “6.1. Create a job template”).

b. When you are prompted if you want to change the job template for the job, click Yes.

c. In the Job name box, type Folder Contents.

To create a simple job template To create and submit a job

27

d. In the Priority list, click Lowest.

3. To add a task, click Task List, and then specify the following task parameters:

a. To add a new basic task to the job, click Add.

b. In the Task name box, type a name for the new task.

c. In the Command line box, type dir.

d. In the Work directory box, type c:\Program Files.

e. To add this task, click Save.

4. To limit the job so that it only runs on a specific compute node in your HPC cluster, click

Resource Selection, and then specify the following resource parameters:

a. Select the Run this job only on nodes in the following list check box.

b. Select the check box for one of the nodes in your HPC cluster.

5. To submit the job, click Submit.

6. If you are prompted to enter your credentials, type your user name and password, and

then click OK.

7. To view the progress and the results of the job that you submitted:

a. In Job Management, in the Navigation Pane, click All Jobs.

b. In the views pane, click the job that you submitted.

c. When the state of the job is Finished, in the Details Pane, double-click the task that

you created in step 3.

d. In the Task Properties window, in the Results tab, the Output box will display the

directory list of c:\Program Files for the compute node that you selected in step 4.

e. If you want to copy the results to the clipboard, click Copy output to clipboard.

6.3. Create and submit a job using the command-line interface (optional)You can create and submit a job similar to the job that you created and submitted in the previous

section, using the command-line interface tools that are included with Windows HPC

Server 2008.

1. Open a Command Prompt window. Click Start, point to All Programs, click

Accessories, and then click Command Prompt.

2. To create a new job, type the following command:

job new /jobname:"Folder Contents" /priority:"Lowest" /RunTime:0:0:1

/requestednodes:”<ComputeNodeName>”

Where <ComputeNodeName> is the name of a compute node in your HPC cluster.

3. To add a task to the job, type the following command:

To create and submit a job using the command-line interface

28

job add <JobID> /workdir:"C:\Program Files" dir

Where <JobID> is the identification number for the job, as displayed on the command-

line interface after typing the command in step 2.

4. To submit the job, type the following command:

job submit /id:<JobID>

Where <JobID> is the identification number for the job, as displayed on the command-

line interface after typing the command in step 2.

5. If you are prompted to enter your credentials, type your password, and then type ENTER.

6.4. Create and submit a job using HPC PowerShell (optional)You can also create and submit the same job that you created and submitted in the previous

section, using HPC PowerShell.

For more information about HPC PowerShell, see Appendix 6: Using HPC PowerShell.

1. On the head node, click Start, point to All Programs, and then click Microsoft HPC

Pack.

2. Right-click HPC PowerShell, and then click Run as administrator.

3. If you are prompted by Windows PowerShell if you want to run the ccppsh.format.ps1xml

script, type A, and then press ENTER.

4. To create a new job, type the following cmdlet:

$j = New-HpcJob -Name "Folder Contents" -Priority Lowest -RunTime "0:0:1" -

RequestedNodes “<ComputeNodeName>”

Where <ComputeNodeName> is the name of a compute node in your HPC cluster.

5. To add a task to the job, type the following cmdlet:

$j | Add-HpcTask -WorkDir "C:\Program Files" -CommandLine "dir"

6. To submit the job, type the following cmdlet:

$j | Submit-HpcJob

7. If you are prompted to enter your credentials, type your password, and then type ENTER.

Notes

You can also type all three cmdlets in one line:

New-HpcJob -Name "Folder Contents" -Priority Lowest -RunTime "0:0:1" -

RequestedNodes “<ComputeNodeName>” | Add-HpcTask -WorkDir "C:\Program

Files" -CommandLine "dir" | Submit-HpcJob

Where <ComputeNodeName> is the name of a compute node in your HPC

cluster.

Note To create and submit a job using HPC PowerShell

29

Related documents For more information about creating and submitting jobs, see the Submitting Jobs in Windows

HPC Server 2008 Step-by-Step Guide (http://go.microsoft.com/fwlink/?LinkId=121887).

For more information about configuring job submission and scheduling policies, see the

Configuring Job Submission and Scheduling Policies in Windows HPC Server 2008 Step-by-

Step Guide (http://go.microsoft.com/fwlink/?LinkId=121888).

Additional Resources

Additional online resources, including step-by-step guides, technical reference documentation

and troubleshooting guides are available on the Windows HPC Server 2008 Technical Library

(http://go.microsoft.com/fwlink/?LinkId=119594).

HPC Cluster Manager includes comprehensive Help documentation. This documentation is

available from the user interface by clicking any of the in-context help links or by pressing F1.

Appendices

In this section:

Appendix 1: HPC Cluster Networking

Appendix 2: Creating a Node XML File

Appendix 3: Node Template Tasks and Properties

Appendix 4: Job Template Properties

Appendix 5: Scriptable Power Control Tools

Appendix 6: Using HPC PowerShell

Appendix 1: HPC Cluster NetworkingWindows HPC Server 2008 supports five cluster topologies designed to meet a wide range of

user needs and performance, scalability, manageability, and access requirements. These

topologies are distinguished by how the compute nodes in the cluster are connected to each

other and to the enterprise network. Depending on the network topology that you choose for your

cluster, certain network services, such as Dynamic Host Configuration Protocol (DHCP) and

network address translation (NAT), can be provided by the head node to the compute nodes.

You must choose the network topology that you will use for your cluster well in advance of setting

up an HPC cluster.

This section includes the following topics:

HPC cluster networks

Supported HPC cluster network topologies

30






HPC network services

Windows Firewall configuration

HPC cluster networksThe following table lists and describes the networks to which an HPC cluster can be connected.

Network Name Description

Enterprise network An organizational network to which the head

node is connected and optionally the compute

nodes. The enterprise network is often the

network that most users in an organization log

on to when performing their job. All intra-cluster

management and deployment traffic is carried

on the enterprise network unless a private

network (and optionally, an application network)

also connects the cluster nodes.

Private network A dedicated network that carries intra-cluster

communication between nodes. This network

carries management, deployment, and

application traffic if no application network

exists.

Application network A dedicated network, preferably with high

bandwidth and low latency. These

characteristics are important so that this

network can perform latency-sensitive tasks,

such as carrying parallel Message Passing

Interface (MPI) application communication

between compute nodes.

Supported HPC cluster network topologiesThere are five cluster topologies supported by Windows HPC Server 2008:

Topology 1: Compute Nodes Isolated on a Private Network

Topology 2: All Nodes on Enterprise and Private Networks

Topology 3: Compute Nodes Isolated on Private and Application Networks

Topology 4: All Nodes on Enterprise, Private, and Application Networks

Topology 5: All Nodes on an Enterprise Network

31

Topology 1: Compute nodes isolated on a private network

The following image illustrates how the head node and the compute nodes are connected to the

cluster networks in this topology:

The following table lists and describes details about the different components in this topology:

Component Description

Network adapters The head node has two network adapters.

Each compute node has one network

adapter.

The head node is connected to both an

enterprise network and to a private

network.

The compute nodes are connected only to

the private network.

Traffic The private network carries all

communication between the head node

and the compute nodes, including

deployment, management and application

traffic (for example, MPI communication).

Network services The default configuration for this topology is

NAT enabled on the private network in

order to provide the compute nodes with

address translation and access to services

and resources on the enterprise network.

DHCP is enabled by default on the private

network to assign IP addresses to compute

nodes.

If a DHCP server is already installed on the

private network, then both NAT and DHCP

will be disabled by default.

32


Security The default configuration on the cluster has

the firewall turned ON for the enterprise

network and turned OFF for the private

network.

Considerations when selecting this topology Cluster performance is more consistent

because intra-cluster communication is

routed onto the private network.

Network traffic between compute nodes

and resources on the enterprise network

(such as databases and file servers) pass

through the head node. For this reason,

and depending on the amount of traffic, this

might impact cluster performance.

Compute nodes are not directly accessible

by users on the enterprise network. This

has implications when developing and

debugging parallel applications for use on

the cluster.

Topology 2: All nodes on enterprise and private networks





Network adapters The head node has two network adapters.

Each compute node has two network

33


adapters.

All nodes in cluster are connected to both

the enterprise network and to a dedicated

private cluster network.

Traffic Communication between nodes, including

deployment, management, and application

traffic, is carried on the private network in

this topology.

Traffic from the enterprise network can be

routed directly to a compute node.

Network services The default configuration for this topology

has DHCP enabled on the private network,

to provide IP addresses to the compute

nodes.

NAT is not required in this topology

because the compute nodes are connected

to the enterprise network, so this option is

disabled by default.



network and turned OFF for the private

network.

Considerations when selecting this topology This topology offers more consistent cluster

performance because intra-cluster

communication is routed onto a private

network.

This topology is well suited for developing

and debugging applications because all

compute nodes are connected to the

enterprise network.

This topology provides easy access to

compute nodes by users on the enterprise

network.

This topology provides faster access to

enterprise network resources by the

compute nodes.

34

Topology 3: Compute nodes isolated on private and application networks





Network adapters The head node has three network

adapters: one for the enterprise network,

one for the private network, and a high-

speed adapter that is connected to the

application network.

Each compute node has two network

adapters, one for the private network and

another for the application network.

Traffic The private network carries deployment

and management communication between

the head node and the compute nodes.

Jobs running on the cluster use the high-

performance application network for cross-

node communication.


has both DHCP and NAT enabled for the

private network, to provide IP addressing

and address translation for compute nodes.

DHCP is enabled by default on the

application network, but not NAT.

If a DHCP is already installed on the private

network, then both NAT and DHCP will be

35


disabled by default.



network and turned OFF on the private and

application networks.



communication is routed onto the private

and application networks.

Compute nodes are not directly accessible

by users on the enterprise network in this

topology.

Topology 4: All nodes on enterprise, private, and application networks





Network adapters The head node has three network

adapters.

All compute nodes have three network

adapters.

The network adapters are for the enterprise

network, the private network, and a high

36


speed adapter for the high performance


Traffic The private cluster network carries only

deployment and management traffic.

The application network carries latency-

sensitive traffic, such as MPI

communication between nodes.

Network traffic from the enterprise network

reaches the compute nodes directly.


has DHCP enabled for the private and

application networks to provide IP

addresses to the compute nodes on both

networks.

NAT is disabled for the private and

application networks because the compute

nodes are connected to the enterprise

network.



network and turned OFF on the private and

application networks.



communication is routed onto a private and


This topology is well suited for developing

and debugging applications because all

cluster nodes are connected to the

enterprise network.



network.



compute nodes.

37

Topology 5: All nodes on an enterprise network





Network adapters The head node has one network adapter.

All compute nodes have one network

adapter.

All nodes are on the enterprise network.

Traffic All traffic, including intra-cluster,

application, and enterprise traffic, is carried

over the enterprise network. This

maximizes access to the compute nodes by

users and developers on the enterprise

network.

Network services This topology does not require NAT or

DHCP because the compute nodes are

connected to the enterprise network.



network.

Considerations when selecting this topology This topology offers easy access to


network.

Access of resources on the enterprise

network by individual compute nodes is

faster.

This topology, like topologies 2 and 4, is

well suited for developing and debugging

applications because all cluster nodes are

38


connected to the enterprise network.



network.



compute nodes.

Because all nodes are connected only to

the enterprise network, you cannot use

Windows Deployment Services to deploy

compute node images using the new

deployment tools in Windows HPC

Server 2008.

HPC network servicesDepending on the network topology that you have chosen for your HPC cluster, the following

network services can be provided by the head node to the compute nodes connected to the

different cluster networks:

Network Address Translation (NAT)

Dynamic Host Configuration Protocol (DHCP) server

This section describes these HPC network services.

Network address translation (NAT)

Network address translation (NAT) provides a method for translating Internet Protocol version 4

(IPv4) addresses of computers on one network into IPv4 addresses of computers on a different

network.

Enabling NAT on the head node enables compute nodes on the private or application networks to

access resources on the enterprise network. You do not need to enable NAT if you have another

server providing NAT or routing services on the private or application networks. Also, you do not

need NAT if all nodes are connected to the enterprise network.

DHCP server

A DHCP server assigns IP addresses to network clients. Depending on the detected configuration

of your HPC cluster and the network topology that you choose for your cluster, the compute

nodes will receive IP addresses from either the head node running DHCP, or from a dedicated

DHCP server on the private network, or via DHCP services coming from a server on the

enterprise network.

39

Windows Firewall configurationWindows HPC Server 2008 opens firewall ports on the head node and compute nodes to enable

internal services to run. By default, Windows Firewall is enabled only on the enterprise network,

and disabled on the private and application networks to provide the best performance and

manageability experience.

If you have applications that require access to the head node or to the cluster nodes on

specific ports, you will have to manually open those ports in Windows Firewall.

Firewall ports required by Windows HPC Server 2008

The following table lists all the ports that are opened by Windows HPC Server 2008 for

communication between cluster services on the head node and the compute nodes.

Port Number (TCP) Required By

5969 Required by the client tools on the enterprise

network to connect to the HPC Job Scheduler

Service on the head node.

9892, 9893 Used by the HPC Management Service on the

compute nodes to communicate with the HPC

System Definition Model (SDM) Service on the

head node.

5970 Used for communication between the HPC

Management Service on the compute nodes

and the HPC Job Scheduler Service on the

head node.

9794 Used for communication between

ExecutionClient.exe on the compute nodes and

the HPC Management Service on the head

node. ExecutionClient.exe is used during the

deployment process of a compute node. It

performs tasks such as imaging the computer,

installing all the necessary HPC components,

and joining the computer to the domain.

9087, 9088, 9089 Used for communication between the client

application on the enterprise network and the

services provided by the Windows

Communication Foundation (WCF) broker

node.

1856 Used by the HPC Job Scheduler Service on the

head node to communicate with the HPC Node

Important

40

Port Number (TCP) Required By

Manager Service on the compute nodes.

8677 Used for communication between the HPC MPI

Service on the head node and the HPC MPI

Service on the compute nodes.

6729 Used for management services traffic coming

from the compute nodes to the head node or

WCF broker node.

5800 Used for communication between the HPC

command-line tools on the enterprise network

and the HPC Job Scheduler Service on the

head node.

5801 Used by the remote node service on the

enterprise network to enumerate nodes in a

node group, or to bring a node online or take it

offline.

5999 Used by HPC Cluster Manager on the

enterprise network to communicate with the

HPC Job Scheduler Service on the head node.

443 Used by the clients on the enterprise network to

connect to the HPC Basic Profile Web Service

on the head node.

Appendix 2: Creating a Node XML FileA node XML file contains a list of compute nodes that you want to add to your cluster. This list

includes:

When adding compute nodes from bare metal, a hardware identification parameter for each

compute node, such as the System Management BIOS (SMBIOS) GUID or the Media Access

Control address (MAC) address.

When adding preconfigured nodes that are already running one of the 64-bit editions of the

Windows Server 2008 operating system, and HPC Pack 2008 has been installed, a computer

name for identification purposes.

Other properties, such as the physical location of each compute node and the Windows

product key that should be used to activate the operating system.

Benefits of using a node XML file for deployment

The following list outlines some of the benefits of using a node XML file when adding compute

nodes to your cluster:

41

You can pre-stage a PXE deployment of compute nodes for your HPC cluster by importing a

node XML file with a list of all the computers that you will be adding to the cluster. The

compute nodes can be deployed both from bare metal or as preconfigured nodes.

Preconfigured nodes that are added to your HPC cluster using a node XML file do not need

to be manually approved into the cluster. This makes the deployment process more efficient

and streamlined.

Importing a node XML file is a simple and efficient way for you to associate properties with

compute nodes. Examples of properties that can be associated with compute nodes are:

location (including data center, rack, and chassis), a Windows product key, node templates,

or tags that are used to automatically create node groups.

You can give specific computer names (NetBIOS names) to compute nodes that are

deployed from bare metal, without having to worry about powering them on in a specific

order. Using a node XML file, computer names will already be associated with a specific

SMBIOS GUID or MAC address (or both).

The node XML file schemaThe node XML file is based on an XML Schema Definition (XSD) language file:

NodeConfigurationFile.xsd. This XSD file is available on the head node, in the Bin folder of the

installation path for HPC Pack 2008. For example, if you are using the default installation path,

the XSD file is available here:

C:\Program Files\Microsoft HPC Pack\Bin\NodeConfigurationFile.xsd

The following table lists and describes the attributes and elements that are defined in the node

XML file schema:

Attribute, Element, or

Element:Attribute

Required Description

Location No Optional element.

Contains attributes with

information about the location

of the compute node.

Location:DataCenter No Optional attribute of the

Location element.

Specifies the name of the

data center where the

compute node is located.

Location:Rack No Optional attribute of the

Location element.

Specifies the name or

number of the server rack

42


Element:Attribute


where the compute node is

located.

Location:Chassis No Optional attribute of the

Location element.

Specifies the name or

number of the chassis that is

used for the compute node.

Template No Optional element.

This element is required

when deploying compute

nodes from bare metal.

Contains attributes with

information about the node

template that will be used to

deploy the compute node.

Template:Name Yes Required attribute of the

Template element.

This attribute is required only

when a Template element is

included.


node template that will be

used to deploy the compute

node.

If the specified node template

name does not exist on the

head node, the deployment

will fail.

If you are deploying compute

nodes from bare metal, this

attribute must specify the

name of a node template that

includes a step to deploy an

operating system image, or

your deployment will fail.

Template:Provisioned No Optional attribute of the

Template element.

43


Element:Attribute


Specifies if the node is a

preconfigured node

(Provisioned=“true”, or

Provisioned=“1”), or not

(Provisioned=“false”, or

Provisioned=“0”).

If this attribute is not

specified, the node is not

considered a preconfigured

node, and the node template

will be applied to the node

when the node is added to

the cluster.

If Provisioned=“true”, or

Provisioned=“1” is

specified, the node template

will not be applied to the

node when the node is added

to the cluster, but the node

will be imported with that

node template associated

with it.

If Provisioned=“false”, or

Provisioned=“0” is

specified, the node template

will be applied to the node

when the node is added to

the cluster.



attribute must be

Provisioned=“false”,

Provisioned=“0”, or must

not be specified. Also, the

node template must include a

step to deploy an operating

system image.

MacAddress No Optional element.

Specifies the MAC address of

44


Element:Attribute


the network adapter that will

be used by the compute

node.


nodes from bare metal, you

must specify this element or

the MachineGuid parameter,

or the deployment will fail.

You must also specify this

element if the cluster nodes

in your system have SMBIOS

GUIDs that are not unique

(that is, two or more nodes in

the node XML file have the

same value for the

MachineGuid parameter).

There can be multiple

instances of this element, if

the compute node uses more

than one adapter.

Ensure that you specify only

those MAC addresses that

exist in the compute node.

Specifying a MAC address

that does not exist in a

compute node, might cause

the import of that node to fail.

Important

When you specify a MAC

address in the node XML

file, do not include any

blank spaces, colons

(“:”), hyphens (“-”), or

dots (“.”). Include only the

twelve hexadecimal digits

for the MAC address. For

example, the following

MAC address is correctly

specified: 00301B445F02.

45


Element:Attribute


Tag No Optional element.


node group to which the

compute node should be

added during deployment.

There can be multiple

instances of this element, if

the compute node should be

added to more than one node

group.

Name Yes Required attribute.

Specifies the computer name

(NetBIOS name) of the

compute node.



attribute specifies the

computer name that will be

assigned to the node during

deployment.

If you are deploying

preconfigured nodes, this

attribute specifies the current

computer name of the

compute node.

If the specified name is that

of a preconfigured node that

has already been added to

the cluster (that is, it is not in

the Unknown state), the

node XML file will fail to

import.

Domain No Optional attribute.

Specifies the Active

Directory® domain to which

the compute node should be

added.

46


Element:Attribute


If this attribute is not

specified, the Active Directory

domain of the head node is

used.

ManagementIpAddress No Optional attribute.

Specifies information that is

required for the integration of

scriptable power control tools

like Intelligent Platform

Management Interface (IPMI)

scripts.

You only need to specify this

attribute if you are using

scriptable power control tools

to manage power on your

cluster.

MachineGuid No Optional attribute.

Specifies the SMBIOS GUID

of the computer where the

compute node is deployed.


nodes from bare metal, you

must specify this parameter

or the MacAddress element,

or the node XML file will fail

to import.

ProductKey No Optional attribute.

Specifies the Windows

product key that will be used

to activate the operating

system on the compute node.

The product key is used

during the activation task of a

node template that includes a

step to deploy an operating

system image.

The product key that you

47


Element:Attribute


specify must match the

edition of the operating

system in the image that is

used by the node template.

Important

You must specify a

Windows product key if

you are using an

operating system image

of a retail version of

Windows Server 2008, or

the evaluation version of

Windows Server 2008

HPC Edition.

Creating a node XML file for deployment from bare metalThe node XML file can be created in any XML editor or text editor, but it must follow the node

XML file schema. Also, a node XML file can be created from an HPC cluster that is already

configured, by exporting it from HPC Cluster Manager.

For detailed information about creating a node XML file, see the Creating a Node XML

File in Windows HPC Server 2008 Step-by-Step Guide (http://go.microsoft.com/fwlink/?

LinkId=139371).

When creating a node XML file for a deployment from bare metal, you will need a hardware

identification parameter for each compute node. This parameter can be the SMBIOS GUID or the

MAC address of the computer.

When creating a node XML file for deployment from bare metal:

Specify the MAC address of a compute node in the MacAddress attribute for that compute

node.

Specify the SMBIOS GUID of a compute node in the MachineGuid attribute for that compute

node.

If both the SMBIOS GUID and MAC address of a compute node are specified, the SMBIOS

GUID is used.

If for some reason you do not have access to the SMBIOS GUID of a node, you can use only

the MAC address.

Note

48



Ensure that you specify only those MAC addresses that exist in each compute node.

Specifying a MAC address that does not exist in a compute node, might cause the import of

that node to fail.

You must specify a node template for each compute node listed, and that node template must

include a step to deploy an operating system image. If yosu do not specify a node template or

if you specify a node template that does not include a step to deploy an operating system

image, the deployment will fail.

Ensure that the node template names that are specified in the node XML file match the

names of the node templates listed on the head node.

Specify any location information that you want to be attached to the node.

If you want nodes to be automatically added to specific node groups during deployment,

specify the Tag attribute with the name of the node group for each compute node.

You must include a Windows product key if you are using an operating system image of a

retail version of Windows Server 2008, or the evaluation version of Windows Server 2008

HPC Edition

If your integration of scriptable power control tools requires a BMC IP address for each

compute node, it can be added to the node XML file.

Sample node XML file<?xml version="1.0" encoding="utf-8" standalone="yes" ?>

<Nodes xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xmlns:xsd="http://www.w3.org/2001/XMLSchema"

xmlns="http://schemas.microsoft.com/HpcNodeConfigurationFile/2007/12">

<Node

Name="ComputeNodeName1"

Domain="CONTOSO"

MachineGuid="{4c4c4544-0038-5710-804b-c6c04f464331}">

<Location

DataCenter="Data Center 1"

Rack="2"

Chassis="1" />

<Template

Name="Default ComputeNode Template" Provisioned="true" />

<MacAddress>00301B445F02</MacAddress>

<MacAddress>001B2104EDF5</MacAddress>

<Tag>ComputeNodes</Tag>

<Tag>Rack2</Tag>

49

</Node>

</Nodes>

Appendix 3: Node Template Tasks and PropertiesYou can use the Node Template Editor in HPC Cluster Manager to add deployment,

configuration, and maintenance tasks to a node template. Some of these tasks are added to new

templates when you create them using the Create Node Template Wizard.



2. In Configuration, click Node Templates.

3. In the Views pane, click the node template to which you want to add tasks.

4. In the Actions pane, click Edit. The Node Template Editor appears.

5. To add a task, click Add Task, and then click the task that you want to add from the list of

available tasks. For more information about each node task and its parameters, see

Available node template tasks.

6. Repeat the previous step for all tasks that you want to add to the node template. After

you are done adding tasks, click Save.

Available node template tasksThere are four types of node template tasks:

Provisioning

Configuration

Deployment

Maintenance

Provisioning

Provisioning tasks are performed on the head node before the deployment process of the

compute nodes takes place. The following table lists the provisioning task that you can add or

modify on a node template, and the properties that are associated with it.

Task Name Task Description Properties

Create Computer Account Creates a computer account

in Active Directory for the

compute node.

Domain (optional): specifies

the name of the domain on

which the computer account

will be created. If this

To add tasks to a node template using the Node Template Editor

50


property is not specified, the

domain of the head node is

used.

ComputerPath (optional):

specifies the path in Active

Directory where the computer

account will be created. The

default path is

cn=Computers. If this


computer path of the head

node is used.

Important

When you specify one of

these two properties, you

must also specify the

other one. If you only

specify one property, it

will be ignored and the

domain and computer

path of the head node will

be used.

Configuration

Configuration tasks are performed on a compute node after the compute node is booted into

Windows Preinstallation Environment (Windows PE) at the beginning of the deployment process.

The following table lists the configuration tasks that you can add or modify on a node template

and the properties that are associated with each.


Run Windows PE Command Runs a command in

Windows PE.

ContinueOnFailure (optional):

if True is selected, the

configuration task will not fail if

the command fails to run

successfully. If False is

selected, the configuration task

will fail if the command fails to

run successfully.

51


ErrorWhiteList (optional):

specifies the return error codes

that should be ignored for the

command. The default return

code that is expected from the

command when it runs

successfully is zero (0). If the

command returns a success

code other than zero, then you

must add that return code to

the list of error codes that

should be ignored, or it will be

interpreted as an error code

and the task will fail.

Command (required): specifies

the Windows PE command that

you want to run.

Multicast Copy Copies a file from the head

node using the multicast

protocol.

UnicastFallback (optional): if

True is selected and multicast

fails, the file will be copied

using the Server Message

Block (SMB) protocol. If False

is selected and multicast fails,

the task will fail.

DestFile (required): specifies

the absolute path to the drive

of the compute node to which

the file will be copied. If you

have added the Partition Disk

task to the node template,

ensure that you specify a path

that is valid for the partitions

that will be created with that

task.

SourceFile (required):

specifies the name and path of

the file to copy, relative to the

Microsoft HPC Pack\Data\

InstallShare folder.

Unicast Copy Copies a file from the head Directory (optional): specifies

52


node using the Server

Message Block (SMB)

protocol.

if a file (False) or a folder

(True) is being copied.

Destination (required):

specifies the absolute path to

the drive of the compute node

to which the file will be copied.

If you have added the Partition

Disk task to the node template,

ensure that you specify a path

that is valid for the partitions

that will be created with that

task.

Source (required): specifies

the name and path of the file to

copy, relative to the Microsoft

HPC Pack\Data\InstallShare

folder.

Partition Disk Partitions the disk on the

compute node using a script

for Diskpart.

DiskPartScript (Required):

specifies the name and path of

the script to use with Diskpart.

Mount Share Shares a folder during the

Windows PE phase of the

operating system installation.

DriveLetter (optional):

specifies the drive letter where

the folder will be shared.

User (optional): specifies the

user name to use when sharing

the folder.

UserPassword (optional):

specifies the password to use

when sharing the folder.

Path (required): specifies the

name and path of the folder

that will be shared.

Install Windows Installs the Windows Server

operating system on the

compute node.

Custom Unattend File

(optional): specifies the

absolute path to the custom

unattend file to use for

installation.

Installation Drive (optional):


53


the Windows Server operating

system will be installed. If you

have added the Partition Disk

task to the node template,

ensure that you specify a drive

letter that is valid for the

partitions that will be created

with that task.

Local Administrator

password (optional): specifies

the password for the local

Administrator account on the

compute node.

Product Key (optional):

specifies the product key to

use with this node template for

the activation of the operating

system.

Autogenerate Local Admin

Password (required): if True is

selected, the password for the

local Administrator account is

automatically generated. After

a password is automatically

generated, it is secret and

cannot be recovered. If False

is selected, then you should

specify a password using the

Local Administrator

password attribute.

Image (required): specifies the

image to use for the installation

of the operating system.

Apply WIM Image Extracts the files in a WIM

file to a local disk on the

compute node.

DestinationPath (required):

specifies the path on the

compute node where the files

in the Windows Imaging

Format (WIM) file will be

extracted. If you have added

the Partition Disk task to the

54


node template, ensure that you

specify a path that is valid for

the partitions that will be

created with that task.

WimPath (required): specifies

the path on the compute node

where the WIM file that will be

extracted is stored.

Restart Restarts the compute node. None

Deployment

Deployment tasks are performed on a compute node after the operating system has been

installed. The following table lists the deployment tasks that you can add or modify on a node

template and the properties that are associated with each.


Unicast Copy Copies a file from the head

node using the Server

Message Block (SMB)

protocol.

Directory (optional): specifies

if a file (False) or a folder

(True) is being copied.

Destination (required):

specifies the absolute path on

the drive of the compute node

where the file will be copied.

Source (required): specifies

the name and path of the file to

copy, relative to the Microsoft

HPC Pack\Data\InstallShare

folder.

Run OS command Runs a command as the local

Administrator.


if True is selected, deployment

will not fail if the command fails

to run successfully. If False is

selected, deployment will fail if


successfully.

ErrorWhiteList (optional):

specifies the return error codes

55


that should be ignored for the

command. The default return

code that is expected from the

command when it runs

successfully is zero (0). If the

command returns a success

code other than zero, then you

must add that return code to

the list of error codes that

should be ignored, or it will be

interpreted as an error code

and the task will fail.

Command (required):

specifies the command that

you want to run as

Administrator.

Install HPC Pack Installs HPC Pack on the

compute node.

Setup Source Directory

(required): specifies the

location of the installation files

for the HPC Pack.

Mount Share Shares a folder on the

compute node.

DriveLetter (optional):


the folder will be shared.

User (optional): specifies the

user name to use when

sharing the folder.

UserPassword (optional):

specifies the password to use

when sharing the folder.

Path (required): specifies the

name and path of the folder

that will be shared.

Join Domain Joins the compute node to an

Active Directory domain.

Domain (optional): specifies

the name of the domain to

which the compute node will

be joined. If this property is not

specified, the domain of the

head node is used.

Restart Restarts the compute node. None

56


Log Off Logs off the compute node. None

Maintenance

Maintenance tasks are performed on a compute node when you select a node in Node

Management and then click Maintain. The following table lists the maintenance tasks that you

can add or modify on a node template and the properties that are associated with each.


Post Install Command Runs a command on the

compute node after HPC

Pack has been installed.


if True is selected, the

maintenance task will not fail if


successfully. If False is

selected, the maintenance task

will fail if the command fails to

run successfully.

Timeout (optional): specifies

the number of seconds before

the command times out. If this


default timeout value is 60

seconds.

WorkingDirectory (optional):

specifies the folder where the

command runs.

Command (required):

specifies the command to run.

This command runs using the

installation credentials that

were provided during the

configuration process of the

head node.

Activate Operating System Activates the operating

system on the compute

node.

None

Apply Updates Applies updates to the Patches (optional): specifies

57


compute node from Microsoft

Update or Windows Server

Update Services (WSUS).

the list of updates that will be

applied to the compute node.

Categories (required):

specifies the type of updates

that will be applied to the

compute node.

Appendix 4: Job Template PropertiesYou can use the Job Template Editor in HPC Cluster Manager to add properties to a job

template. Some of these properties are added to new templates when you create them using the

Generate Job Template Wizard.



2. In Configuration, click Job Templates.

3. In the Views pane, click the job template to which you want to add properties.

4. In the Actions pane, click Edit. The Job Template Editor appears.

5. To add a property, click Add, and then click the property that you want to add from the list

of available properties. For more information about each job template property, see

Available job template properties.

6. Repeat the previous step for all properties that you want to add to the job template. After

you are done adding properties, click Save.

Available job template propertiesThe following table lists all the job properties that you can set in a job template. The properties

that you set in a job template constrain the properties that a cluster user can choose when

submitting a job to the cluster using that template.

Job Property Description

Auto Calculate Maximum If True is selected as the only valid value, the cluster

user cannot specify the maximum number of resources

(cores, sockets, or nodes) assigned to the job, and

resources will be automatically calculated based on the

tasks in the job. If False is selected as the only valid

value, the cluster user must specify the maximum

number of resources assigned to the job. If True and

To add properties to a job template using the Job Template Editor

58


False are selected as valid values, the cluster user can

choose to specify the maximum number of resources, or

to have them automatically calculated.

Auto Calculate Minimum If True is selected as the only valid value, the cluster

user cannot specify the minimum number of resources

(cores, sockets, or nodes) assigned to the job, and

resources will be automatically calculated based on the

tasks in the job. If False is selected as the only valid

value, the cluster user must specify the minimum

number of resources assigned to the job. If True and


choose to specify the minimum number of resources, or

to have them automatically calculated.

Exclusive If True is selected as the only valid value, no other jobs

can run on a compute node at the same time as the job

being submitted by the cluster user. If False is selected

as the only valid value, the cluster user cannot select

this property when submitting the job. If True and False

are selected as valid values, the cluster user can select

or unselect this property when submitting the job.

Fail on Task Failure If True is selected as the only valid value, the failure of

any task in the job that is submitted by the cluster user

will cause the entire job to fail immediately. If False is

selected as the only valid value, the cluster user cannot

select this property when submitting the job. If True and


select or unselect this property when submitting the job.

Job Name Specifies a list of names that the cluster user can select

for the job. If you set this property, the cluster user

cannot specify a job name that is not on the list.

Licenses Specifies a list of licenses that the cluster user can

select for the job. If you set this property, the cluster

user cannot specify a license that is not on the list.

Licenses in this list can be validated by a job activation

filter that is defined by the cluster administrator.

Maximum Cores Specifies a range of values for the maximum number of

cores that the cluster user can assign to the job. This

property has no effect if any of these conditions is true:

59


The Auto Calculate Maximum property has been

added to the template with True selected as the only

valid value

The Unit Type property has been added to the

template and the list of the type of resources that

can be assigned to the job does not include Core

The user selects a different type of resource to

assign to the job (that is, sockets or nodes)

Maximum Nodes Specifies a range of values for the maximum number of

cluster nodes that the cluster user can assign to the job.

This property has no effect if any of these conditions is

true:



valid value



can be assigned to the job does not include Node


assign to the job (that is, sockets or cores)

Maximum Sockets Specifies a range of values for the maximum number of

sockets that the cluster user can assign to the job. This

property has no effect if:



valid value



can be assigned to the job does not include Socket


assign to the job (that is, nodes or cores)

Minimum Cores Specifies a range of values for the minimum number of

cores that the cluster user can assign to the job. This


The Auto Calculate Minimum property has been


valid value


60



can be assigned to the job does not include Core


assign to the job (that is, sockets or nodes)

Minimum Nodes Specifies a range of values for the minimum number of

cluster nodes that the cluster user can assign to the job.

This property has no effect if:



valid value



can be assigned to the job does not include Node


assign to the job (that is, sockets or cores)

Minimum Sockets Specifies a range of values for the minimum number of

sockets that the cluster user can assign to the job. This




valid value



can be assigned to the job does not include

Sockets


assign to the job (that is, nodes or cores)

Node Groups Specifies a list of node groups that the cluster user is

required to select for the job. If you set this property, the

cluster user can still specify a node group that is not on

the list, but cannot remove any of the node groups that

are listed as required.

Node Ordering Specifies the ordering to use when assigning nodes to

run the job. This property gives preference to nodes with

specific attributes. The node ordering options are:

Memory Size (Ascending). The job will be

assigned first to nodes that have the smallest

amount of memory.

61


Memory Size (Descending). The job will be

assigned first to nodes that have the largest amount

of memory.

Processor Number (Ascending). The job will be

assigned first to nodes that have the smallest

number of cores.

Processor Number (Descending). The job will be

assigned first to nodes that have the largest number

of cores.

Preemptable If True is selected as the default value (Default Value

parameter for this property), the job being submitted by

the cluster user can be preempted by another job that

has a higher priority, if there are not enough resources

to run the higher priority job. If False is selected as the

default value, the job being submitted by the cluster user

cannot be preempted by another job.

Note

This property has no effect if the preemption

policy for the HPC Job Scheduler Service is set

to No pre-emption. To review or configure the

preemption policy for the HPC Job Scheduler

Service, in HPC Cluster Manager, from the

Options menu, click Job Scheduler

Configuration.

Note

If True and False are selected as valid values

for this property (Valid Value parameter), a

cluster user that is using the job template can

submit a job that cannot be preempted by using

the HPC API, independently of the value that

was selected as the default value. It is not

possible to specify that a job cannot be

preempted by submitting it using HPC Cluster

Manager, HPC Job Manager, the HPC

PowerShell, or the HPC command-line tools. It

is only possible to do this by using the HPC API.

For more information about the HPC API, see

the Microsoft HPC Pack

(http://go.microsoft.com/fwlink/?LinkID=123849).

62



Note

If you want to allow only certain cluster users to

submit jobs that cannot be preempted, create a

job template that includes this property with

False selected as the default and the only valid

value. Then, select which users can submit jobs

with the job template by setting permissions for

the template. Permissions for a template can be

set in HPC Cluster Manager, in Configuration,

under Job Templates (Set Permissions

action).

Priority Specifies a list of priority values that the user can select

for the job. If you set this property, the cluster user

cannot specify a priority value that is not on the list.

Project Specifies a list of project names that the cluster user can

select for the job. If you set this property, the cluster

user cannot specify a project name that is not on the list.

Requested Nodes Specifies a list of nodes that the cluster user can select

to run the job. If you set this property, the cluster user

cannot select a node that is not on the list.

Run Time Specifies a range of values for the amount of time that

the cluster user can specify the job is allowed to run. If a

task in the job is still running after the specified run time

is reached, the task is stopped and the job is

automatically canceled by the HPC Job Scheduler

Service.

Run Until Canceled If True is selected as the only valid value, the job runs

until it is canceled or until its run time expires. If False is

selected as the only valid value, the cluster user cannot

select this property when submitting the job. If True and


select or unselect this property when submitting the job.

Service Name Specifies a list of services names that the cluster user

can select for a Service-Oriented Application (SOA) job.

If you set this property, the cluster user cannot specify a

service name that is not on the list.

Unit Type Specifies a list of the type of resources (cores, sockets,

or nodes) that can be assigned to the job. If you set this

63


property, the cluster user cannot specify a resource type

that is not on the list.

Appendix 5: Scriptable Power Control ToolsThe cluster administration console (HPC Cluster Manager) includes actions to start, shut down,

and restart compute nodes remotely: Start, Reboot, and Shut Down in the Actions pane in

Node Management. These actions are linked to the CcpPower.cmd script, which performs these

power control operations, with the exception of the start action that is not enabled, using

operating system commands.

You can replace the default operating system commands in CcpPower.cmd with custom power

control scripts, like Intelligent Platform Management Interface (IPMI) scripts.

CcpPower.cmd is available in the Bin folder of the installation path for HPC Pack 2008. For

example, if you are using the default installation path, the file is available here:

C:\Program Files\Microsoft HPC Pack\Bin\CcpPower.cmd

The default CcpPower.cmd file has the following code:

@setlocal

@echo off

if L%1 == Lon goto on

if L%1 == Loff goto off

if L%1 == Lcycle goto cycle

echo "usage:CcpPower.cmd [on|off|cycle] nodename [ipaddress]"

goto done

:on

exit /b 1

goto done

:off

shutdown /s /t 0 /f /m \\%2

goto done

:cycle

shutdown /r /t 0 /f /m \\%2

64

goto done

:done

exit /b %ERRORLEVEL%

endlocal

To enable scriptable power control tools for the Shut Down and Reboot actions in HPC Cluster

Manager, replace the entries of the shutdown command in CcpPower.cmd with the name and path

of your tool or tools for shutting down and restarting the node. To enable tools for the Start action,

replace the exit entry in the :on section with the name and path of your tool for this action.

Also, you must associate a management IP address with each compute node in the cluster (for

example, the IP address for the Base Management Controller (BMC) of the compute node). The

management IP address is the third string (%3) that is passed to the CcpPower.cmd script by HPC

Cluster Manager, and should be provided to your power control tools when you add them in

CcpPower.cmd. A management IP address can be associated with each compute node in the

cluster in the following ways:

When compute nodes are deployed using a node XML file, by specifying the

ManagementIpAddress attribute for each node. For more information, see Appendix 2:

Creating a Node XML File.

By using the Set-HpcNode cmdlet in the HPC PowerShell, with the ManagementIpAddress

parameter. For more information about this cmdlet, see the Windows HPC Server 2008

PowerShell Reference (http://go.microsoft.com/fwlink/?LinkID=120725). Alternatively, in the

HPC PowerShell, type Get-Help Set-HpcNode.

Appendix 6: Using HPC PowerShell HPC PowerShell is built on Microsoft Windows PowerShell™ technology, and provides a

powerful command-line interface and a scripting platform to enable the automation of

administrative tasks. HPC PowerShell is installed by default on the head node, and can also be

installed on a client computer as part of the utilities available with Windows HPC Server 2008.

1. Click Start, point to All Programs, and then click Microsoft HPC Pack.

2. Right-click HPC PowerShell, and then click Run as administrator.

3. If you are prompted by Windows PowerShell to choose if you want to run the

ccppsh.format.ps1xml script, type A, and then press ENTER.

1. Click Start, point to All Programs, click Microsoft HPC Pack, and then click HPC

PowerShell.

2. If you are prompted by Windows PowerShell to choose if you want to run the

ccppsh.format.ps1xml script, type A, and then press ENTER.

To start HPC PowerShell on the head node To start HPC PowerShell on a client computer

65



You can also add the HPC PowerShell snap-in from Windows PowerShell.

1. Click Start, point to All Programs, click Windows PowerShell 1.0, and then click

Windows PowerShell.

2. In Windows PowerShell, type the following cmdlet:

Add-PsSnapin Microsoft.HPC

Add the HPC PowerShell snap-in to your Windows PowerShell profileIf you have a Windows PowerShell profile, you can add the HPC PowerShell snap-in to it so that

it is available in every PowerShell session under your user name. For more information about

Windows PowerShell profiles, see Windows PowerShell Profiles (http://go.microsoft.com/fwlink/?

LinkID=119587).

1. Open Windows PowerShell. Click Start, point to All Programs, click Windows

PowerShell 1.0, and then click Windows PowerShell.

2. To edit your profile in Notepad, type:

notepad $profile

3. Type the following cmdlet as a new line in the profile:

Add-PsSnapin Microsoft.HPC

4. To save the profile, in the File menu, click Save.

5. To close Notepad, in the File menu, click Exit.

View Help in HPC PowerShell In-context help is available for HPC PowerShell cmdlets:

To view a list of the cmdlets that are available in HPC PowerShell, type the following cmdlet:

Get-Command –PSSnapin Microsoft.HPC

To view basic help information for a specific cmdlet, type:

Get-Help <cmdlet>

Where <cmdlet> is an HPC PowerShell cmdlet.

To view detailed information for a specific cmdlet, type:

Get-Help <cmdlet> -Detailed

Where <cmdlet> is an HPC PowerShell cmdlet.

To view the help information on the screen one page at a time, type | More at the end. For

example, if you type:

Get-Help New-HpcJob –Detailed | More

To add HPC PowerShell from Windows PowerShellTo add the HPC PowerShell snap-in to your Windows PowerShell profile

66


Only the first page of the detailed information for the New-HpcJob cmdlet will be initially

displayed. As you press SPACE or ENTER, more information will be displayed. To stop

viewing the help information, press Q.

67

HPCDeploymentGuide

Documents

cluster deployment

windows server

windows hpc server

hpc cluster overview

deployment guide

windows powershell

windows vista

microsoft group of companies