Introduction Hardware Power Saving Modes in HPC Power Mode Control System Evaluation Conclusion and Future Work Managing Hardware Power Saving Modes for High Performance Computing Second International Green Computing Conference 2011, Orlando Timo Minartz, Michael Knobloch, Thomas Ludwig, Bernd Mohr [email protected] Scientific Computing Department of Informatics University of Hamburg 26-07-2011 1 / 21
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Introduction Hardware Power Saving Modes in HPC Power Mode Control System Evaluation Conclusion and Future Work
Managing Hardware Power Saving Modes for
High Performance ComputingSecond International Green Computing Conference 2011,
Orlando
Timo Minartz, Michael Knobloch, Thomas Ludwig, Bernd Mohr
Scientific ComputingDepartment of Informatics
University of Hamburg
26-07-2011
1 / 21
Introduction Hardware Power Saving Modes in HPC Power Mode Control System Evaluation Conclusion and Future Work
Motivation
High Performance Computing
Increasing performance and efficiency of calculation units
But: Increasing need for calculation power increases size ofinstallations
High operational costs for large-scale high performanceinstallations
High carbon footprint of installations should be reduced forenvironmental and political reasons
2 / 21
Introduction Hardware Power Saving Modes in HPC Power Mode Control System Evaluation Conclusion and Future Work
1 Introduction
2 Hardware Power Saving Modes in HPC
3 Power Mode Control System
4 Evaluation
5 Conclusion and Future Work
3 / 21
Introduction Hardware Power Saving Modes in HPC Power Mode Control System Evaluation Conclusion and Future Work
Introduction (1)
High Performance Computing (HPC)
Important tool in natural sciences to analyze scientificquestions in silico
Modeling and simulation instead of performing time consumingand error prone experiments
Models from weather systems to protein folding tonanotechnology
Leads to new observations and unterstanding of phenomena
4 / 21
Introduction Hardware Power Saving Modes in HPC Power Mode Control System Evaluation Conclusion and Future Work
Introduction (2)
Top500 list – http://www.top500.org
Since 1993 the Top500 list gathers information about achievedperformance of supercomputers
Exponential growth in computing performance can be observed
Current (June 2011) rank #1: K computer by Fujitsu
Peak performance: 8773.63 TFlop/sPower consumption: 9898.56 KW
Green500 list – http://www.green500.org
Ranking based on energy-efficiency
Energy-efficiency is defined as Flop/s per Watt
K computer rank (June 2011): #6
5 / 21
Introduction Hardware Power Saving Modes in HPC Power Mode Control System Evaluation Conclusion and Future Work
Exascale computing
Roadmap
"Practical power limit" of 20 MW (U.S. Department of Energy)
Energy-efficiency must be increased from different viewpoints
The data-center itself including cooling facilities etc.The hardware running the scientific applicationsThe scientific applications themselves
6 / 21
Introduction Hardware Power Saving Modes in HPC Power Mode Control System Evaluation Conclusion and Future Work
Hardware power saving modes
Adaption of mechanism from mobile devices
Idle / power saving modes of hardware
Dynamic Voltage and Frequency Scaling of processors (DVFS)
Adapt frequency and disable ports of network devices
Spin down and flush caches of hard disks
HPC related problems
Synchronization problems
OS Jitter increase
Possible performance loss
7 / 21
Introduction Hardware Power Saving Modes in HPC Power Mode Control System Evaluation Conclusion and Future Work
Idle power saving potentialSheet5
Page 1
CPU Max Freq CPU Min Freq CPU Min Freq + NIC 100 Mbit
CPU Min Freq + NIC 100 Mbit + Disk sleep
0
20
40
60
80
100
120
140
160
180AMD OpteronIntel Xeon
Wa
tt
Opteron: up to 11 % power savingsXeon: up to 18 % power savings
8 / 21
Introduction Hardware Power Saving Modes in HPC Power Mode Control System Evaluation Conclusion and Future Work
CPU power saving potential for Xeon nodeSheet5
Page 1
1600 1733 1867 2000 2133 2267 2400 2533 2667 2800 28010
50
100
150
200
250
300
350
400LoadIdle
MHz
Wa
tt
30 % power savingsInteresting in phases of busy-waiting or memory-boundness
9 / 21
Introduction Hardware Power Saving Modes in HPC Power Mode Control System Evaluation Conclusion and Future Work
When to switch which component...
Performance/usage prediction
Utilization based approach
Instructions Per Second (IPS)
Other performance counters (e.g. memory bandwidth)
Knowledge about future hardware use
Application developer
Compiler
Libraries
10 / 21
Introduction Hardware Power Saving Modes in HPC Power Mode Control System Evaluation Conclusion and Future Work
Daemon architecture
11 / 21
Introduction Hardware Power Saving Modes in HPC Power Mode Control System Evaluation Conclusion and Future Work
Daemon design
Server daemon
Make decision about hardware power states
Processor
Reduce frequency (P-States) using cpufreq
Network card
Reduce speed / switch duplex mode using ethtool
Harddisk
Reduce OS access and enforce standby modes using hdparm
Client library
Linked to (MPI) application
Forwards desired device power state via sockets to server
12 / 21
Introduction Hardware Power Saving Modes in HPC Power Mode Control System Evaluation Conclusion and Future Work
Test MPI applications
partdiff-par
PDE solver
Computation intensive phases, communication intensivephases and IO phases
PEPC
Pretty Efficient Parallel Coulomb-solver
Computation intensive phases and communication intensivephases
13 / 21
Introduction Hardware Power Saving Modes in HPC Power Mode Control System Evaluation Conclusion and Future Work
Hardware
Details
3 × LMG 450 power meter
4 channels eachup to 20 samples persecond
5 × AMD Opteron 6168
Dual socket24 cores per node
5 × Intel Xeon X5560
Dual socket8 cores per nodeSMT disabled
14 / 21
Introduction Hardware Power Saving Modes in HPC Power Mode Control System Evaluation Conclusion and Future Work
Experimental setup
Application test setup
Instrumented: State switching dependent on phase
CPU Max Freq: Processor frequency set to maximum
CPU Min Freq: Processor frequency set to minimum
CPU Ondemand: Processor governor set to ondemand
Results
Time-to-Solution (TTS): Total time for test setup
Energy-to-Solution (ETS): Total energy for test setup
15 / 21
Introduction Hardware Power Saving Modes in HPC Power Mode Control System Evaluation Conclusion and Future Work
TTS and ETS for partdiff-par on Xeon nodesSheet5
Page 1
InstrumentedCPU Max Freq
CPU Min FreqCPU Ondemand Freq
0
50
100
150
200
250
300
350
400
450
0
20
40
60
80
100
120
140
160
180Time-to-SolutionEnergy-to-Solution
seco
nd
s
kJ5 % savings in Energy-to-SolutionTime-to-Solution increase of about 4 %
16 / 21
Introduction Hardware Power Saving Modes in HPC Power Mode Control System Evaluation Conclusion and Future Work
TTS and ETS for partdiff-par on Opteron nodesSheet5
Page 1
InstrumentedCPU Max Freq
CPU Min FreqCPU Ondemand Freq
0
200
400
600
800
1000
1200
0
100
200
300
400
500
600Time-to-SolutionEnergy-to-Solution
seco
nd
s
kJ8 % energy savingsRuntime increase of 9 %
17 / 21
Introduction Hardware Power Saving Modes in HPC Power Mode Control System Evaluation Conclusion and Future Work
TTS and ETS for PEPC on Opteron nodesSheet5
Page 1
InstrumentedCPU Instrumented
CPU Max FreqCPU Min Freq
0
500
1000
1500
2000
2500
0
200
400
600
800
1000
1200
1400
1600
1800
2000Time-to-SolutionEnergy-to-Solution
seco
nd
s
kJ7 % energy savings, 4 % runtime increase compared to4 % energy savings, 2 % runtime increase
18 / 21
Introduction Hardware Power Saving Modes in HPC Power Mode Control System Evaluation Conclusion and Future Work
Conclusions
Power consumption of idle nodes can be reduced by 11 % and18 % respectively
Power consumption can be decreased by more than 30 % inphases with unnecessary high utilization (e.g. busy-waiting)
Control device power states from userspace by introducing ahardware management daemon
Reduce the Energy-to-Solution by up to 8 % with anTime-to-Solution increase of about 9 % for presentedapplications
19 / 21
Introduction Hardware Power Saving Modes in HPC Power Mode Control System Evaluation Conclusion and Future Work
Future work
Identify energy saving possibilities (Scalasca enhancement)
Benchmark to measure power consumption in various states
Enhanced measurements with larger count of applications
20 / 21
Introduction Hardware Power Saving Modes in HPC Power Mode Control System Evaluation Conclusion and Future Work
CPU power saving potential (C-States enabled)Sheet5
Page 1
1600 1733 1867 2000 2133 2267 2400 2533 2667 2800 28010
50
100
150
200
250
300
350
400Load, cstates = offLoad, cstates = onIdle, cstates = offIdle, cstates = on
MHz
Wa
tt
21 / 21
Related Documents
