SprayCool Energy Smart Data Center Programcfroundtable.org/meetings/032505/spraycool.pdf · SprayCool Energy Smart Data Center Program Jointly Funded by ... SprayCool is anSprayCool

1

HPC Products Group

ISR Proprietary3/24/2005

SprayCool Energy Smart Data Center Program

Jointly Funded by

OSD and Defense Microelectronics Activity

DoE and Pacific Northwest National Laboratory

2

HPC Products Group


Overview

• Brief Company Overview• Brief Technology Overview• FAQ• Energy Smart Data Center Program Overview

• Energy Savings• Facility Savings

• Conclusion

3

HPC Products Group


About ISR – Mission

“To enable our customers to compete and excel through our development and delivery of innovative

system solutions. Our customers choose us because we deliver the electronics industry’s

highest performing, most reliable and cost effectiveproducts and services.”

SprayCool is an Enabling TechnologySprayCool is an Enabling Technology

4

HPC Products Group


About ISR – History & Products

1980s 1990s 2000s1980s 1990s 2000s

2002 – Won DOE contract for SprayCool demonstration with hybrid electric vehicles2003 – Won 1st contract for SprayCool for display systems2003 – Migrated successful EFV products to TOC environment2003 – Completed 1st surveillance system to locate and identify hostile snipers2004 – First flight demonstration in tactical aviation – Navy EA-6B2004 – Won DOE contract for datacenter demonstration

1998 – Signed contract to provide USMC a parallel path for their critical EFV C4I suite1998 – Delivered 1st SprayCool base station for telecom industry1999 – Began work on new solutions developed to support high altitude applications2001 – Navy contract for SprayCool power supplies2001 – Awarded SDD contract for EFV program2001 – Achieved ISO 9000 certification

Isothermal Systems Research founded with SBIR funding aimed at solving thermal issues with space-based weapons systems for the Star Wars Program

1988 – ISR begins operations1989 – Company moves to Seattle, WA1992 – ISR relocates to eastern Washington1994 – Delivered 1st server to NSA1997 – NSA supercomputer

Broad Industry Experience, Now Focusing on Computing Applications

Broad Industry Experience, Now Focusing on Computing Applications

5

HPC Products Group


Market Segments

Revolutionary electronics optimized for spray cooling

Computing/Telecom

Ability to use electronics in extreme temperature

environments, while saving weight

Isolation of custom and COTS electronics from harsh

environments

Avionics / SensorsGround Mobile

6

HPC Products Group


Basic SprayCool Architectures

Spray COTS boards in sealed enclosure

Blade applications (VME, cPCI etc.)Heat exchanger can be remotely locatedSealed from all contaminatesLow-cost technology refresh

Global SprayCool Architecture

Modular SprayCool ArchitectureSpray only the high heat producing components locally

Motherboard applications (ATX etc.)Heat exchanger can be remotely locatedAllows sealed COTS enclosuresImproves performance of conduction cooled boards

Flexible, Scalable Solutions for Retrofit, Current and Future Capabilities Flexible, Scalable Solutions for Retrofit, Current and Future Capabilities

7

HPC Products Group


FAQs

• Fluids: • Safe, Inert, Non-Toxic, Non-Flammable Dielectric Liquid, No EPA

Regulations on Disposal or Transportation• ISR has Long-Term Fluid Compatibility Knowledge Database, No Issues

with Industry Standard Materials • Use Best Practices for Cable I/O Routing for Fluid I/O

• Hardware Applications: • Everything from Single Die to Total System Solutions• Processors, Power Electronics, RF Electronics, etc…

• Performance:• 500Watts/in2, Up to 2000Watts/cm2 (Most Computing Applications will

Never Exceed Several Hundred Watts/cm2)• Reliability:

• SprayCool Systems Have High Availability and Redundancy as Well as IMPI Capability

• SprayCool Technology Increases Electronics Reliability 3-4X• Cost:

• SprayCool System Save Money

8

HPC Products Group


Problem Plaguing Data Centers

• Air Cooling Appears to be Low Cost but Has Many Secondary Cost Impacts• Poor Thermal Performance and Thermal Cycling

• Results in Low Reliability • Results in Reduced Performance (Processor Throttling)

• High Air Flow Requirements, Air Handling, Floor Tiles, Top Down,A/C – At or Near Limits• Results in Increased Vertical and Horizontal White Space• Results in Excessive Noise• Results in High Energy Costs• Results in High Construction Cost

9

HPC Products Group


Program Overview

• Objective: • To Demonstrate TCO Savings when Applying SprayCool

Technology to High Performance Computing (Energy and Facility)

• Description: • Phase I: Convert a Rack of HP Air-Cooled rx2600 Servers and

Build MPB Prototype for Demonstration• Phase II: Install Rack of Globally SprayCool MPB and do

Feasibility Studies for Cooling Additional Items, e.g. Mass Data Storage, Power Distribution

• Phase III: Install Several Racks of SprayCool Processing Nodes, Employ Them, Demonstrate Mass Data Storage Cooling and Power Distribution

• Results:• Phase I: Initial Results Cooling Only Processors Yields a

COP Increase from 1.00 to 1.97• Phase II: Analysis shows a potential for a COP of 6.00• Phase III: Analysis shows potential for a COP as high as

13.00• Funding:

• Jointly Funded by DoE and DoD (PNNL and DMEA)

ISR Proprietary and Confidential

10

HPC Products Group


Phase I Methodology

• Rack of Air-Cooled rx2600 Servers Baselined• Benchmark Baseline Testing Performed at PNNL and ISR

Integration Lab• Testing Performed on Single Server in Air then again after

Conversion to SprayCool – Ambient Temperatures 21C• Testing Performed on Rack of 18 Servers in Air then again after

Conversion to SprayCool – Ambient Temperatures 21C• Servers Run As-Received from HP• Processors fully loaded with 6 instances of BurnI2• Servers benchmarked using Linpack (1 and 1.5 hour, single

instance)• Rack Level System Due for Installation at PNNL in Q2

• System will be in Production for One Year for Testing and Evaluation


11

HPC Products Group


Phase I Air-Cooled rx2600 Converted to SprayCool

SprayCool HP rx2600SprayCool HP rx2600

Air-Cooled HP rx2600Air-Cooled HP rx2600


12

HPC Products Group


Phase I SprayCool Rack of HP rx2600s

Rack ManifoldRack Manifold

Fluid I/OTMFluid I/OTM

Service Loop Utilizes Cable Arm

Service Loop Utilizes Cable Arm


13

HPC Products Group


Technology for the Future

The Future of High-PerformanceComputing

• 380 Watt Dual AMD Opteron6Ux160mm Board, 32Gb

• Advanced cooling combined with highly integrated 3D electronics

• Maximized memory density through stacking and advanced packaging

• Very Low Profile (VLP) DIMM leverages new DRAM module

• Increased performance• Provides higher reliability at lower

power • Reduced Size and Weight

The Future of High-PerformanceComputing

• 380 Watt Dual AMD Opteron6Ux160mm Board, 32Gb

• Advanced cooling combined with highly integrated 3D electronics

• Maximized memory density through stacking and advanced packaging

• Very Low Profile (VLP) DIMM leverages new DRAM module

• Increased performance• Provides higher reliability at lower

power • Reduced Size and Weight

14

HPC Products Group



Rugged Transit Case

Rugged Transit Case

1U Appro Dual Opteron Server

1U Appro Dual Opteron Server

2U SprayCool Thermal Management Unitn+1 Redundant Pumps Liquid to Air or Liquid to Liquid

2U SprayCool Thermal Management Unitn+1 Redundant Pumps Liquid to Air or Liquid to Liquid1U Redundant 48 VDC

Power Supply1U Redundant 48 VDC

Power Supply

KVM Switch KVM Switch ISR SprayCool Chassis20 Slot cPCI 6U x 160mm 700 Watts per Slot48 VDC Power DistributionEnvironmental IsolationTessera Dual Opteron

MPB

ISR SprayCool Chassis20 Slot cPCI 6U x 160mm 700 Watts per Slot48 VDC Power DistributionEnvironmental IsolationTessera Dual Opteron

MPB

2U SCSI hard drive unit

2U SCSI hard drive unit

Quick Release Levers for Simple Service

Quick Release Levers for Simple Service

15

HPC Products Group


Saves Energy

05

1015202530354045505560657075808590

50 55 60 65 70 75 80 85 90 95

PROCESSOR POWER [W]

PRO

CES

SOR

TEM

PER

ATU

RE

[°C

]

AIR COOL 240CFM, 22C AIR

AIR COOL 125CFM, 22C AIR

GEN I BARE, 30C FLUID

GEN I LIDDED, 30C FLUID

GEN II LIDDED, 30C FLUID

• SprayCool Processors Running at 100% Performance • At Maximum Performance Processors only dray 62Watts

as opposed to 89Watts for Air Cooled running with equivalent performance

• SprayCool Processors Running at 100% Performance • At Maximum Performance Processors only dray 62Watts

as opposed to 89Watts for Air Cooled running with equivalent performance

16

HPC Products Group


Saves Energy

(2)19,000 Watts

Fans

Server

TMU & Blower Chiller Cooling

Tower

CPUCPUs: ~ (1)156,000 W + 294,000 W

Outside

CPU

Dielectric Glycol Water

(6)36,000 Watts

Pumps

20,000

36,000Watts

Pumps

(7)72,000Watts

Compressor

(5)4,500 Watts

Pumps

Total for Cooling:

228,500

450,000 /

228,500 =

COP:

1.97

Total Energy: 678,500 Watts

(156 kW removed

from facility air)

(4)41,000 Watts

Fans

Lieberts

AirBypass

17

HPC Products Group


Saves Facilities Cost

• Facilities Construction Costs for a Tier 3 Critical Data Center are ~$1500sqft

• Average Square Footage per 19” Rack is 50sqft (Include Rack Space, Cold and Hot Isles, Walk Space, Battery Back-up, etc…

• So Average Construction Cost per Rack is $75,000• So Average Cost per Rack Unit for a 44RU Rack is

Around $1700• SprayCool Technology Ensures NO Horizontal or Vertical

White Space is Required for Cooling Electronics, Reclaiming Facilities Construction Costs

• SprayCool Solutions Provide Immediate ROI Through Cost Avoidance for New Facilities Construction

18

HPC Products Group

ISR Proprietary27-Jan-05

SprayCool Solutions From ISR

Saves Energy

Saves Facilities Cost


www.SprayCool.com

SprayCool Energy Smart Data Center Programcfroundtable.org/meetings/032505/spraycool.pdf · SprayCool Energy Smart Data Center Program Jointly Funded by ... SprayCool is anSprayCool

Documents