In the Data Center: Five Keys to Achieving Ultra-Low PUEs

Enterprise Data Center Solutions

Five keys to achieving ultra-low PUEsChris YetmanSenior Vice President, Operations

Ultra-low

PUEs

Getting From Here To There

2

You’re using the latest high-efficiency servers. You keep your server room at a higher temperature. You’ve virtualized to cut down on the number of servers. It seems like you’re doing everything right. But you’re still stuck at a less than enviable PUE.

33MW of data center, 18 acres

Commissioned PUEs as low as 1.12

15% reduction in TCO due to highly

efficient facility designs

Largest LEED Platinum data center in

North America

I feel your PUE pain

Today I’d like to share five things we’ve learned about getting to ultra-low PUEs. Check it out: we’ve been able to hit 1.15 PUEs for our customers.

We get it. We know this problem. We live and breathe it every day.

PREVIOUS JOB

Chief InformationOfficer (CIO)

As a former CIO, a current Chief of Operations, and, most importantly, someone who is passionate about building awesome data centers, I’d like to call on all CIOs.

Too often we are needlessly cautious when it comes to hardware. Too often we make IT decisions based on fear.

Why are we afraid to allow equipment to fail? Why are we working so hard to avoid breaking a server?

These decisions cost us more than we suspect.

1. Be Brave

1. Be Brave

Operating Envelope

ASHRAE

Allowed

Temp: 59°-90°

Humidity: 20-80%

Dew Point:Up to 63°

Temp: 65-80

Humidity: 42-60%

Dew Point:Up to 59°

The truth is, studies have shown IT equipment can handle higher temperatures than what the industry currently operates under.

The industry needs to diversify equipment-level risk. Why should the loss of any single piece of equipment compromise your business?

For Vantage, being located in Santa Clara, we utilize the free-cooling hours throughout the fall, winter, spring, and temperate summer days. We help each customer build the data center they want to build. If one feels comfortable and wants to move out of the recommended ASHRAE zone into allowable conditions, they create greater cost efficiency by utilizing natural air temperature and flow up to 80% of the year.

Even if you’re timid about rising to 90-degrees, being more flexible on humidity can have a positive effect.

ASHRAE

Recommended

1. Embrace Failure

Hello, my name is failure.

But you can call me

recovery.

Stop trying to prevent hardware failure. It’s not worth the cost. If you build with the knowledge that hardware will fail, you can instead focus on recovery.

This will save you money.

Operating at a higher operating envelope may mean more hardware failure, but you will save so much in energy efficiency and costs.

Those savings will cover the money needed to replace hardware, ten-fold.

Don’t let hardware failure be the worst-case-scenario. Hardware failure is nothing to fear.

1.3 1.2PUE

Operating

Envelope

$500K

7

Check out the cost savings!

A large data center running five-plus megawatts can spend $500,000 more in energy per month.

But let’s say you run the temperature up to 80-degrees in the cold aisle, taking your PUE down from 1.3 to 1.2.

In this scenario, you will save over $500,000 per year in energy, and break a whole lot less than that in hardware.

Imagine the temperature at 85, 90-degrees.

Focus on extending the envelope.

1. Bravery and Recovery = Cost Savings

8

Take the example of Netflix. Here’s a company that’s challenged its system to plan for recovery and resilience.

They created Chaos Monkey, a software tool that forces their engineers to deal with small failures, ones that, once eliminated, will keep from turning into major outages.

Netflix says the name “Chaos Monkey” comes from the idea of “unleashing a wild monkey with a weapon in data center or cloud region to randomly shoot down instances and chew through cable—all while we continue serving our customers without interruption.”

When Netflix loses a server—no big deal. The software reassigns the data elsewhere.

The challenge to developers everywhere is to write applications to be resilient.

1. Bravery and Recovery = Cost Savings

2. Get the Foundation Right

High-efficiency mechanical /

electrical equipment

High-voltagepower

distribution

Measurement

&

Control

Proper containment of

hot and cold aisles

9

Achieving low PUEs begins with right, high-efficiency equipment. For Vantage that means these four pillars listed above.

A solid foundation also includes minute attention to mechanical infrastructure like high-efficiency fan motors, direct drive units, and variable frequency drives on all motors. Did you know we don’t use fans with belts? That way we don’t have to worry about slippage and losing efficiency to those incremental, inevitable belt slips. All the little things add up.

Monitor Adjust Recirculate

Precise temperature control

3. Maintain Infinite Control

It’s critical to choose an environmental-electrical-monitoring system that allows for precise high-speed industrial process control.

The system tracks air conditions both inside and outside the data center—temperature, humidity, dew point, flow volume.

This allows the system to make adjustments with precision.

As the outside air temperature drops, the system is able to recirculate more heated air from

the data module into the supply air stream, allowing for precise temperature control.

This allows the system to modulate anywhere from 100% outside air economization down to

0% full recirculation.

The system can provide the minimal amount cooling to meet the envelope!

LossesCurrent

Conductors

480V tothe rack

4. Increase the Voltage

11

HigherVoltage

High voltage distribution to the IT racks reduces transmission loss, decreases the amount of copper running into the building, and minimizes transformation losses.

If you combined this with an open compute platform approach, where you transform down from 480V at the rack and share power supplies—think of all the energy you’d retain!

Here’s the straight electric talk:

• The losses due to typical transformation from 480V to

208/120V of a typical Power Distribution Unit

(PDU) account for anywhere from 1.5%-3.5%

• Providing 400V to the Busway Distribution Units

(BDU’s, equivalent to PDU) line to neutral voltage of 230V

can be supplied to the IT equipment

• The result is reduced losses, lower current supplied to the

IT equipment, and smaller conductors required to power

the servers

• Extending the higher voltages out to the rack saves on

transformation losses and conductor costs

5. Reduce Pressure Drop

12

Choose mechanical cooling equipment that enables the

lowest available pressure drop.

Start with the outside air damper array and filter bank.

Choose components to minimize air velocity. These

range from oversized dampers and filters to higher cross-

section cooling coils.

Look at your fans. Belt drives inevitably lead to slippage

and energy loss. Direct drive supply fans eliminate such

loss, and large diameter fan rotors increase flow volume

while maintaining reasonable equipment footprint.

Use natural cycles. Data modules utilize vertical hot-

aisle, allowing for convective air flow to return the hot air

from the IT equipment to the cooling plenums.

Avoid leakage by creating a strong barrier between hot

and cold aisles.

1. This is not an all-or-nothing environment.

Going from open/shut cases to varying

degrees of open-to-shut provides optimization.

2. Moving a lot of air is turbulent. Rigid filtering is

better. Don’t let your filters disintegrate.

3. Air will mix before it hits the equipment. Add

lots of measurement points along the air flow

path. The air the racks ingest matters the most.

Lessons Learned

• Runs at 1.15 PUEs

• Fully redundant 2N electrical configuration

• 9 megawatt, expandable to 18 megawatts

This is already a reality today.

1. Be brave. Embrace failure.

2. Maintain infinite control.

3. Pick the right equipment.

4. Increase the voltage.

5. Reduce pressure drop.

Getting to Ultra-Low PUEs

Let’s connect:

THANKYOU