Federal Data Center Programs
7 X 24 Exchange October, 2007William [email protected]
Acknowledgement of LBNL Acknowledgement of LBNL sponsorssponsors
California Energy Commission – PIER California Energy Commission – PIER programprogram
Pacific Gas and Electric CompanyPacific Gas and Electric Company New York State Energy and New York State Energy and
Development Agency (NYSERDA)Development Agency (NYSERDA) US - Environmental Protection AgencyUS - Environmental Protection Agency US – Department of EnergyUS – Department of Energy
LBNL data center activitiesLBNL data center activities
Research RoadmapResearch Roadmap Benchmarking and 23 data center case studies Benchmarking and 23 data center case studies Self-benchmarking protocol Self-benchmarking protocol Power supply &UPS efficiency studiesPower supply &UPS efficiency studies Standby generation lossesStandby generation losses Performance metrics – Computation/wattPerformance metrics – Computation/watt Demonstration projectsDemonstration projects EPA report to CongressEPA report to Congress DOE Save Energy Now programDOE Save Energy Now program
Data center energy roadmapData center energy roadmap
July, 2003 – Research July, 2003 – Research roadmap developed roadmap developed for the California for the California Energy CommissionEnergy Commission
The problem receded The problem receded for several years as for several years as excess data center excess data center space was absorbedspace was absorbed
Now the concerns are Now the concerns are back back
Current LBNL Current LBNL CACA focus focus
Developing LEEDDeveloping LEEDTMTM type criteria for data centers type criteria for data centers Evaluating modular and scalable cooling Evaluating modular and scalable cooling
solutionssolutions Promoting use of air economizersPromoting use of air economizers
– study filtration optionsstudy filtration options– research failure modesresearch failure modes– collaborate with ASHRAEcollaborate with ASHRAE
Demonstrating spray cool technologyDemonstrating spray cool technology Continuing DC power initiativeContinuing DC power initiative
Current LBNL Current LBNL internalinternal focus focus
Designing new supercomputing facilityDesigning new supercomputing facility Goal - a model of energy efficiencyGoal - a model of energy efficiency
– Convened workshop of all supercomputer Convened workshop of all supercomputer vendorsvendors
– Better projections of future power needsBetter projections of future power needs– Pushing the envelop – environmental Pushing the envelop – environmental
conditions, power distribution conditions, power distribution Leading scientific computing facilityLeading scientific computing facility First hand exposure to barriersFirst hand exposure to barriers
Current LBNL Current LBNL FederalFederal focus focus
EPA activitiesEPA activities– Report to Congress follow-onReport to Congress follow-on– Energy Star server specEnergy Star server spec– Energy Star buildings – data center metricEnergy Star buildings – data center metric
DOE activitiesDOE activities– EERE - Save Energy Now EERE - Save Energy Now
Pilot assessmentsPilot assessments Profiler toolProfiler tool Assessment toolsAssessment tools
– FEMP – Technical assistanceFEMP – Technical assistance
Why Data Centers?Why Data Centers?
Highly energy-intensive and rapidly growing Highly energy-intensive and rapidly growing Consume 10 to 100 times more energy per Consume 10 to 100 times more energy per
square foot than a typical office buildingsquare foot than a typical office building Large potential impact on electricity supplyLarge potential impact on electricity supply
and distributionand distribution Used about 45 billionUsed about 45 billion
kWh in 2005kWh in 2005 At current rate, powerAt current rate, power
requirements couldrequirements coulddouble in 5 years.double in 5 years.
Where do data centers fit in?Where do data centers fit in?
Data centers are energy intensive facilitiesData centers are energy intensive facilities– Server racks now designed to carry 25 kW or moreServer racks now designed to carry 25 kW or more– Typical facility ~ 1MW, but can be > 20 MWTypical facility ~ 1MW, but can be > 20 MW
Information factoriesInformation factories
– Nationally Nationally 1.5% of US Electricity consumption 1.5% of US Electricity consumption in in 20062006
Could double in next 5 yearsCould double in next 5 years Critical national and global infrastructureCritical national and global infrastructure
– Few technology barriers to increased efficiency Few technology barriers to increased efficiency – Good candidates for efficiency investments by Good candidates for efficiency investments by
utilities to reduce peak loadsutilities to reduce peak loads
IT equipment load density – IT equipment load density – LBNL benchmark resultsLBNL benchmark results
IT Equipment Load Intensity
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
Data Center Number
Wat
ts/s
q. ft
.
2003 BenchmarksAve. ~ 25
2005 BenchmarksAve. ~ 52
Rising cost of ownershipRising cost of ownership From 2000 From 2000 –– 2006, computing performance 2006, computing performance
increased 25x but energy efficiency only 8xincreased 25x but energy efficiency only 8x– Amount of power consumed per $1,000 of Amount of power consumed per $1,000 of
server spending has increased 4xserver spending has increased 4x
Cost of electricity and supporting infrastructure Cost of electricity and supporting infrastructure now surpasses capital cost of IT equipmentnow surpasses capital cost of IT equipment
Perverse incentives --- IT and facilities costs Perverse incentives --- IT and facilities costs separateseparate
Source: The Uptime Institute, 2007
IT Industry is taking actionIT Industry is taking action
www.climatesaverscomputing.org.
www.thegreengrid.com
What is US Government’s What is US Government’s role?role?
EPA & DOE want to be the catalystEPA & DOE want to be the catalyst – Stimulate competition on energy efficiencyStimulate competition on energy efficiency – Foster discussions between key stakeholdersFoster discussions between key stakeholders– Provide key recommendations (EPA Report to Provide key recommendations (EPA Report to
Congress)Congress)– Develop standardized protocols and metrics to Develop standardized protocols and metrics to
measure energy consumption (e.g., ENERGY STAR)measure energy consumption (e.g., ENERGY STAR)
Take the lead on assessing and implementing Take the lead on assessing and implementing best practices in federal datacentersbest practices in federal datacenters
Promote initiatives globally Promote initiatives globally
Findings from EPA report to Findings from EPA report to CongressCongressTrends in data center energy use Trends in data center energy use Sector consumed about Sector consumed about 61 billion KWh61 billion KWh in 2006 in 2006
– Equates to Equates to ~1.5%~1.5% total U.S. electricity total U.S. electricity consumption and consumption and ~$4.5 billion~$4.5 billion
– Federal sector: ~6 billion kWh and ~$450 Federal sector: ~6 billion kWh and ~$450 millionmillion
Projected to increase to Projected to increase to 100 billion kWh100 billion kWh in in 20112011– Equates toEquates to ~2.5% ~2.5% of total U.S. electricity of total U.S. electricity
consumption and consumption and ~$7.4 billion~$7.4 billion
Projected electricity useProjected electricity useVarious scenarios 2007 - 2011Various scenarios 2007 - 2011
Percentage of electricity Percentage of electricity delivered delivered to IT equipmentto IT equipment
IT Power to Total Data Center Power
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Data Center Number
Rat
io
Average .57
Higher is better
Source: LBNL Benchmarking
Data Center total electrical Data Center total electrical demand/ IT equipment demanddemand/ IT equipment demand
Total Data Center Power/IT Power
0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Center Number
Rat
io
Average 1.83
Lower is better
Source: LBNL Benchmarking
LBNL design targetLBNL design target
Total Data Center Power/IT Power
1.00
1.50
2.00
2.50
3.00
3.50
LBNL’s new supercomputer facility is being designed to achieve a ratio of 1.2
EPA report EPA report recommendationsrecommendations
Standardized performance measurements for IT Standardized performance measurements for IT equipment and data centersequipment and data centers– Development of benchmark/metric for data centersDevelopment of benchmark/metric for data centers– ENERGY STAR label for serversENERGY STAR label for servers
Leadership by Federal GovernmentLeadership by Federal Government– Publicly report energy performance of datacenters Publicly report energy performance of datacenters – Conduct energy efficiency assessments in all Conduct energy efficiency assessments in all
datacenters in 2-3 yearsdatacenters in 2-3 years– Architect of the Capital, implement server-related Architect of the Capital, implement server-related
recommendations in Greening of the Capital reportrecommendations in Greening of the Capital report
Recommendations cont.Recommendations cont. Private sector challengePrivate sector challenge
– CEOs conduct CEOs conduct DOE Save Energy NowDOE Save Energy Now energy energy efficiency assessments, implement measures, efficiency assessments, implement measures, and report performanceand report performance
Information on best practicesInformation on best practices– Raise awareness and reduce perceived risk of Raise awareness and reduce perceived risk of
energy efficiency improvements in datacenterenergy efficiency improvements in datacenter– Government partner with private industry: Government partner with private industry:
case studies, best practicescase studies, best practices Research and developmentResearch and development
– Develop technologies and practices for Develop technologies and practices for datacenter energy efficiency (e.g., hardware, datacenter energy efficiency (e.g., hardware, software, power conversion)software, power conversion)
DOE data center program DOE data center program objectivesobjectives
Build tools, expertise, Build tools, expertise, and strategy in and strategy in collaboration with collaboration with industryindustry
Raise awareness of the Raise awareness of the opportunityopportunity
Recognize industry Recognize industry leadersleaders
DOE program strategyDOE program strategy
Build on Build on Save Energy NowSave Energy Now model model– DOE deployed software tools,DOE deployed software tools,
training curriculum, and qualifiedtraining curriculum, and qualifiedexperts to train and work withexperts to train and work withstaff at large U.S. industrial plants.staff at large U.S. industrial plants.
– 65% of recommended actions now65% of recommended actions nowcompleted, in progress, or planned.completed, in progress, or planned.
With industry input, develop appropriate tools, With industry input, develop appropriate tools, training, and qualified experts to improve data training, and qualified experts to improve data centers.centers.
Conduct pilots, peer reviewed by industry, promote Conduct pilots, peer reviewed by industry, promote and facilitate industry implementationand facilitate industry implementation..
Data center programData center program
Build on Build on Save Energy NowSave Energy Now modelmodel
With industry input, develop With industry input, develop appropriate tools, training, and appropriate tools, training, and qualified experts to improve qualified experts to improve data centers.data centers.
Conduct pilots, peer reviewed Conduct pilots, peer reviewed by industry, promote and by industry, promote and facilitate industry facilitate industry implementationimplementation..
DOE near term planDOE near term plan
Build strong liaisons and partnerships with industry Develop robust new energy assessment program Begin to develop tools and info on best practices
– Assessment framework and energy profiling tool– System level assessment protocol and analysis tools
Partner with industry – joint development/peer review Conduct pilot assessments Provide awareness training Screen for demonstration opportunities Update Federal procurement specifications
Server Load/ComputingOperations
Cooling Equipment
Power Conversion & Distribution
AlternativePower
Generation
• High voltage distribution• Use of DC power• Highly efficient UPS systems• Efficient redundancy
strategies
• Load management• Server innovation
Energy efficiency opportunities Energy efficiency opportunities are everywhereare everywhere
• Better air management• Move to liquid cooling• Optimized chilled-water plants• Use of free cooling
• On-site generation• Waste heat for cooling• Use of renewable energy/fuel cells
Potential benefits of improved Potential benefits of improved data center energy efficiencydata center energy efficiency
Save 20 billion kWh per year by 2015Save 20 billion kWh per year by 2015– Worth $2 billion, Worth $2 billion, ≈ ≈ annual annual
electricity use in 1.8 million electricity use in 1.8 million American homesAmerican homes
Potentially defer need to build 2,300 Potentially defer need to build 2,300 MW of new generating capacityMW of new generating capacity– Avoid 3.4 million metric tons of Avoid 3.4 million metric tons of
carbon emissions (like taking carbon emissions (like taking 675,000 cars off the road)675,000 cars off the road)
Extend life and capacity of existing Extend life and capacity of existing data center infrastructuresdata center infrastructures
Typical data center energy end Typical data center energy end useuse
Server Load/ComputingOperations
Cooling Equipment
Power Conversions& Distribution
100 Units
33 UnitsDelivered
35 Units
Performance varies
Data Center Server Load
51%
Data Center CRAC Units
25%
Cooling Tower Plant4%
Electrical Room Cooling
4%
Office Space Conditioning
1%
Lighting2%
Other13%
Computer Loads67%
HVAC - Air Movement
7%
Lighting2%
HVAC - Chiller and
Pumps24%
The relative percentages of the energy actually doing computing varied considerably.
Source: LBNL case studies
U.S. data centerU.S. data centerenergy efficiency goal by 2011energy efficiency goal by 2011 Industry, DOE and partners perform 1,000 Save Energy Industry, DOE and partners perform 1,000 Save Energy
Now assessments using a common protocol and tool suite Now assessments using a common protocol and tool suite to yield savings of 5 billion kWh per yearto yield savings of 5 billion kWh per year
Industry-DOE energy efficiency campaign saves an Industry-DOE energy efficiency campaign saves an additional 20 billion kWh per yearadditional 20 billion kWh per year
Total energy savings of 25 billion kWh per year by 2011; Total energy savings of 25 billion kWh per year by 2011; 25% below current trends, which yields:25% below current trends, which yields:- $2 billion of electricity cost savings$2 billion of electricity cost savings- carbon emission reduction of 5 MTCEE; equal carbon emission reduction of 5 MTCEE; equal
to the emissions from 3.5 million carsto the emissions from 3.5 million cars- All the electricity consumed in the state of All the electricity consumed in the state of
Utah in one yearUtah in one year
Web-based resourcesWeb-based resourceshttp://hightech.lbl.gov/datacenters.htmlGood starting point for those seeking efficiency measures
Best Practices
Case Studies
Design Guidance
Self-benchmarking GuideBenchmark data
Other Reports (demonstrations)
Additional resources Additional resources
Guides available through PG&E’s Energy Design Resources Website http://hightech.lbl.gov/dctraining/
TOP.html
How can industry participate? How can industry participate?
Participate in EPA and DOE workshopsParticipate in EPA and DOE workshops Participate in peer review of products, protocols and Participate in peer review of products, protocols and
best practicesbest practices Conduct self-benchmarking and report resultsConduct self-benchmarking and report results
– Use tools from LBNL site and download protocol at: Use tools from LBNL site and download protocol at: http://hightech.lbl.gov/datacenters.htmlhttp://hightech.lbl.gov/datacenters.html
Participate in DOE data center energy efficiency Participate in DOE data center energy efficiency program program – Register on web site to get regular updates Register on web site to get regular updates
www.eere.energy.gov/datacenters/
websites: websites:
http://hightech.lbl.gov/datacenters/http://hightech.lbl.gov/datacenters/www.energystar.gov/datacenters
www.eere.energy.gov/datacenters/www.eere.energy.gov/datacenters/