All content in this presentation is protected – © 2009 APC by Schneider Electric Core | High Density | Rev 0 Build a Better Data Center with APC and Cisco.

All content in this presentation is protected – © 2009 APC by Schneider Electric Core | High Density | Rev 0

Build a Better Data Centerwith APC and Cisco

Choosing and specifying density

High density in an existing environment

High density in a new environment

© 2009 APC by Schneider Electric Core | High Density | Rev 0

APC|Cisco Partnership Overview

● Cisco Technology Developer Partner since 1999 creating tested and integrated solutions for:

● CallManager

● Unity Express

● Cisco Unified Communications Manager

● Cisco “Data Center of the Future” Partner● Online presence on www.thedcofthefuture.com

● Joint webcasts

● Presence in Cisco’s booth at Cisco Live in June

● Speaking opportunities with Cisco at Cisco Live


Energy and service cost control pressure

Increasing availability expectations

Regulatory requirements

Server Server consolidation & consolidation & virtualizationvirtualization

Rapid changes in IT Rapid changes in IT technologytechnology

High density High density blade server blade server power/heatpower/heat

Dynamic power variation

UncertainUncertainlong-term plans for long-term plans for capacity or densitycapacity or density

APC can help simplify the complexities of APC can help simplify the complexities of data center high density challengesdata center high density challenges

High density is a key factor in the challenges facing data centers


Would require 7 vented floor tiles per rack(7x more than normally allocated)

Raised floor, perimeter-cooled data centers face practical Raised floor, perimeter-cooled data centers face practical limitation of approximately 5 kW limitation of approximately 5 kW (average)(average) per rack per rack

Example: Example: COOLING requirement for an 20 kW rack = 2,100 cfm per rackCOOLING requirement for an 20 kW rack = 2,100 cfm per rack

Floor tile300 cfm

Floor tile300 cfm

Floor tile300 cfm

Floor tile300 cfm

Floor tile300 cfm

Floor tile300 cfm

Floor tile300 cfm

Requires substantial increases in aisle width and spacing between racks

Clogged raised floors Clogged raised floors compound the problemcompound the problem

46

White paper

Challenges to COOLING


Breaker chaos● Insufficient breaker positions

● “Cascading” panels

● Mislabeled or unlabeled breakers

● Becomes worse with redundancy

Adding new voltage levels & receptacle types to the rack

● Increases need for hot work● Trend towards minimizing hot work

Getting more power to the rack● Avoid tripping breakers

COMPLEXITY increases likelihood of downtimeCOMPLEXITY increases likelihood of downtime

2829

White papers

Challenges to POWER


Floor loading of fully-loaded high-density rack

● Sub-floor weight bearing limitations

● Supplemental raised floor pedestals may be required

Airflow limitations of front and rear doors

Cable chaos – Risk of blocked airflow and human error from proliferation of cables in the rack

Without effective high-density implementation, Without effective high-density implementation, rack and cable challenges proliferaterack and cable challenges proliferate

Challenges to SPACE and CABLING


Racking and Cabling Solutions for Nexus

NEW! These configuration guides are nowNEW! These configuration guides are nowAvailable on www.thedcofthefuture.comAvailable on www.thedcofthefuture.com


Same 500 kW data center – but different “average Same 500 kW data center – but different “average density” depending on how calculated:density” depending on how calculated:

746 watts/ft2

179 watts/ft2

119 watts/ft2

189 watts/ft2

5 kW/rack = ● Eliminates the ambiguities of watts/ft2

● Allows different densities for different areas of the data center

Ambiguous

• Include access area around racks?

• Include back-room area?

• Consider total mains power consumption ?

Total IT power# of racks

SpecifyingDensity

A better way

Traditional way

120

White paper

First we need a standardized way to specify density


Density specification principles

Specify at the ROW level● Rack: Too detailed – Not enough information up front

● Room: Not detailed enough – No flexibility for variation

● ROW: Just right!

Environment Average watts/rack

Typical peak-to-average

ratios

Lab Medium (4-10 kW/rack)

High (up to 4)

Server intensive(Typical data center)

Medium to high(4-15 kW/rack)

Medium (approx 2)

Storage intensiveLow

(< 4 kW/rack)Low (< 2)

Co-location High (10-15 kW/rack)

Medium (2)

Supercomputing High (10-15 kW/rack)

Low (close to 1)

Use watts/rack● Unambiguous

Establish peak-to-average ratio of approximately 2

● Too low impacts IT flexibility and causes inefficiency

● Too high leads to oversizing

SpecifyingDensity

These 3 basic principles simplify density implementationThese 3 basic principles simplify density implementation


Four cooling strategies for high density

A spectrum of strategies, depending upon level of A spectrum of strategies, depending upon level of investment and planninginvestment and planning

Low investmentLow investment

High investmentHigh investment

Spread the load

Suplemental cooling

High-density pod

Whole-roomhigh density

Least planning

Most planning

11

22

33

44

46

White paper


Disadvantages

● IT equipment placement can be surprisingly complex

● One addition of IT equipment can have surprising effects on other existing loads

● Difficult to figure out where the limits to growth are

● Uses the most floor space

● Data cabling issues

Option 1: Spread the load

Advantages

● Extends life of near-end-of-life data center

● Essentially “free” band-aid approach

Assessment servicesAssessment services can help plan can help plan implementation and determine the limits of this implementation and determine the limits of this

strategystrategy

Spread out high density equipment in the room

1 Spread the load 2 Supplemental cooling

3 High-density pod 4 Whole room

Cooling strategies:


Option 2: Supplemental cooling

Advantages

● Can target high density (brings the solution TO the hot spot)

● Defers capital cost

Disadvantages

● Room constraints could limit deployment

● May be constrained to available bulk cooling

Supplemental cooling devices are available and Supplemental cooling devices are available and can help with targeted high-density equipmentcan help with targeted high-density equipment

Air removal unit

Air distribution unit Single AC unit

1 Spread the load

2 Supplemental cooling 3 High-density pod

4 Whole room

Cooling strategies:


Option 3: High-density pods

Advantages

● Maximum density capability (30 kW/rack)

● High-efficiency design

● Optimal floor space utilization

● Allows for targeted availability

● Effective long-term strategy

● No need for raised floor

Disadvantages

● Needs to be planned for in advance

● Requires grouping of high-density equipment

Easy-to-implement, cost effective, Easy-to-implement, cost effective, high-efficiency solutionhigh-efficiency solution

1 Spread the load

2 Supplemental cooling


Cooling strategies:

134

White paper

High-density pod

Hot/cool air circulation is

localized within the zone

HEAT OUTTo building’s heat rejection system

Low-density room


Disadvantages

● Highly specialized and custom built

● Extreme capex compared to other options

● Can result in extreme underutilization of cooling capacity if not correctly engineered

Option 4: Whole-room high density

Advantages

● Handles all high density scenarios

● Smallest footprint

Not a widely adopted approach – requires Not a widely adopted approach – requires significant engineering for unique scenariossignificant engineering for unique scenarios

Cooling strategies:1 Spread the load


3 High-density pod

4 Whole room

Watch video about this purpose-built high-density data center

Go


Deployment strategies for cooling EXISTING data centers

Spread the load

● High density is very small fraction of load

● Location placement of IT equipment is flexible

● Open U-space in existing racks

Supplemental cooling

● High density is small fraction of load

● No flexibility over the placement of IT equipment

● Limited capital budget

Dedicated high-density pods

● High density is moderate to high fraction of load

● Sub-section of data center is available for pod(s)

Whole-room high-density cooling

11 22

33 44

46

White paper


Deployment strategies for coolingNEW data centers

Spread the load Supplemental cooling

● Perimeter cooled room specified at < 6 kW/rack

● High density is small fraction of load

● Limited capital budget

Dedicated high-density pods

● High density is moderate to high fraction of load

● Cooling redundancy requirements vary by equipment

● Future rack density requirements uncertain

Whole-room high-density cooling

● Large farms of high density servers (i.e. HPC)

● Location of all racks and density are known in advance

● Expensive to implement

11 22

33 44

46

White paper


High-density pod explained

● A “mini data center” with its own cooling

● Contributes no heat to rest of data center

● Works alongside existing room-based cooling

● Hot/cool air circulation localized within the pod by short air paths and/or containment

● Achieves optimum efficiency

1 Spread the load



Cooling strategies:


Cooling IN the row, close to the load

Cooling units Operates on hard floor or raised floor

Hot-aisle air enters from rear, preventing mixing of hot and cool air

Cold air is supplied to the cold aisle

Heat is captured and transferred to heat rejection system

1 Spread the load




Standardized modular multi-rack high-density pod

Integral row-based air conditioners

Air conditioners return ambient

room-temperature air

Hot air is exhausted to the hot aisle andreturns to the back of the air conditioners

1 Spread the load




Major efficiency benefits of row-based approach vs traditional room cooling

● Less air mixing Fans move only the air required by the IT equipment, instead of mixing the room air: 60% reduction in fan power

● Variable fan speed Fan speed dynamically tracks the actual IT load, instead of running at full speed: 50% typical further reduction in fan power

● Higher return temperature Air is captured by the CRAC at higher temperature, easing the transfer of heat to the heat rejection systems: 10% reduction in chiller power

● No rehumidification Air is processed by the CRAC at higher temperature, eliminating the energy associated with dehumidification / rehumidification: 10% reduction in CRAC power

● More economizer use Return water temp to chiller is higher, increasing the operating hours of economizer modes: 10% typical reduction in chiller power


Is there an efficiency vs density tradeoff?

The highest efficiency data centersThe highest efficiency data centers

will be will be high-densityhigh-density data centers data centers

● High density and high efficiency are related

● Many people think high density makes efficiency worse

Only true when trying to push existing data centers to cool high density

● High density enables high efficiency in a properly designed new data center or a pod in an existing data center● Shorter pipe lengths Less pump power

● Shorter air flow lengths

● Less air mixing

● Higher return air temperatures Less humidification/dehumidification power and chiller power

Less fan power


POWER: High-density power distribution

415 volt distribution (Elimination of PDU transformers)

● Increased energy efficiency Lower electric bill

● Decreased copper Less weight

● Smaller footprint More space for IT racks

Modular power distribution● Hot swappable → No need to predict

future rack power requirements

● Allows higher power densities in distribution products

● Auto-sensing of breaker size and location


MANAGEMENT: Adding intelligence

Knowing what’s going on, in real time

● Smart management can:

● Analyze the effect of proposed changes

● Suggest the best place for adding new servers

● Recognize overloads or trends in time for corrective action

● Identify stranded capacity Locations where there is available POWER, COOLING or SPACE but not enough of the other two

150

White paper


Choosing and specifying density

Row-level specification eliminates stranded

capacity, prevents hot spots, and provides

flexibility for future deployments. This leads to a

high efficiency data center.

High density in an existing environment

Assessment of existing cooling and power environment helps determine best strategy to implement high density. Cooling solutions range

from spreading out the load to high density pods.

High density in a new environment

An optimal data center physical infrastructure design strategy includes dedicated high-density pods with close-coupled cooling, 415 V power distribution, modular power distribution, and capacity & change management software.

High density Summary


Where to go for more info? Next Steps?

● Where to get more info● www.apc.com

● Tools.apcc.com

● www.thedcofthefuture.com

● Cisco Partner Central under Technology Developer Partners and APC

● APC White papers● APC WP#129: A Scalable, Reconfigurable, and Efficient Data Center

Power Distribution Architecture

● APC WP#118: Virtualization: Optimized Power and Cooling to Maximize Benefits

● APC WP#114: Implementing Energy Efficient Data Centers

● APC WP#126: An Improved Architecture for High-Efficiency, High-Density Data Centers

● APC WP#150: Power and Cooling Capacity Management for Data Centers

All content in this presentation is protected – © 2009 APC by Schneider Electric Core | High Density | Rev 0 Build a Better Data Center with APC and Cisco.

Documents

density high density

density apc

rack floor tile

cfm floor tile

different average density

floor loading

rack increases

cisco live slide