Energy models in data parallel CPU/GPU computations Master Degree in Computer Science and Networking Università di Pisa and Scuola Superiore Sant’Anna Academic Year 2014/2015 Supervisor: Candidate: Prof. Marco Danelutto Alessandro Lenzi Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 1 / 23
117
Embed
Energy Models in Data Parallel CPU/GPU Computations
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Energy models in data parallel CPU/GPU computations
Master Degree in Computer Science and Networking
Università di Pisa and Scuola Superiore Sant’Anna
Academic Year 2014/2015
Supervisor: Candidate:Prof. Marco Danelutto Alessandro Lenzi
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 1 / 23
Outline
1 IntroductionI Energy consumption issueI Heterogeneous, parallel architecturesI ProblemI Energy consumption in CPU/GPU computations
2 GPU Energy Model
3 CPU Energy Model
4 Aggregated model and usage
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 2 / 23
Section 1
Introduction
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 3 / 23
The problem of energy consumption
I Technical problemI Cities worth of power, power provvisioning is an issue
I Higher temperature, more failuresI Mobile devices with limited energy budget.
I Economic issue
I Cost of energy > cost of hardware
I Environmental concern
I 2% of greenhouse emissions cause by US IT sector
Energy consumption is nowadays a growing concern in IT industry.
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 4 / 23
The problem of energy consumption
I Technical problemI Cities worth of power, power provvisioning is an issueI Higher temperature, more failures
I Mobile devices with limited energy budget.I Economic issue
I Cost of energy > cost of hardware
I Environmental concern
I 2% of greenhouse emissions cause by US IT sector
Energy consumption is nowadays a growing concern in IT industry.
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 4 / 23
The problem of energy consumption
I Technical problemI Cities worth of power, power provvisioning is an issueI Higher temperature, more failuresI Mobile devices with limited energy budget.
I Economic issue
I Cost of energy > cost of hardware
I Environmental concern
I 2% of greenhouse emissions cause by US IT sector
Energy consumption is nowadays a growing concern in IT industry.
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 4 / 23
The problem of energy consumption
I Technical problemI Cities worth of power, power provvisioning is an issueI Higher temperature, more failuresI Mobile devices with limited energy budget.
I Economic issue
I Cost of energy > cost of hardwareI Environmental concern
I 2% of greenhouse emissions cause by US IT sector
Energy consumption is nowadays a growing concern in IT industry.
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 4 / 23
The problem of energy consumption
I Technical problemI Cities worth of power, power provvisioning is an issueI Higher temperature, more failuresI Mobile devices with limited energy budget.
I Economic issueI Cost of energy > cost of hardware
I Environmental concern
I 2% of greenhouse emissions cause by US IT sector
Energy consumption is nowadays a growing concern in IT industry.
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 4 / 23
The problem of energy consumption
I Technical problemI Cities worth of power, power provvisioning is an issueI Higher temperature, more failuresI Mobile devices with limited energy budget.
I Economic issueI Cost of energy > cost of hardware
I Environmental concern
I 2% of greenhouse emissions cause by US IT sector
Energy consumption is nowadays a growing concern in IT industry.
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 4 / 23
The problem of energy consumption
I Technical problemI Cities worth of power, power provvisioning is an issueI Higher temperature, more failuresI Mobile devices with limited energy budget.
I Economic issueI Cost of energy > cost of hardware
I Environmental concernI 2% of greenhouse emissions cause by US IT sector
Energy consumption is nowadays a growing concern in IT industry.
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 4 / 23
The problem of energy consumption
I Technical problemI Cities worth of power, power provvisioning is an issueI Higher temperature, more failuresI Mobile devices with limited energy budget.
I Economic issueI Cost of energy > cost of hardware
I Environmental concernI 2% of greenhouse emissions cause by US IT sector
Energy consumption is nowadays a growing concern in IT industry.
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 4 / 23
The problem of energy consumption
I Technical problemI Cities worth of power, power provvisioning is an issueI Higher temperature, more failuresI Mobile devices with limited energy budget.
I Economic issueI Cost of energy > cost of hardware
I Environmental concernI 2% of greenhouse emissions cause by US IT sector → 3% by 2020
Energy consumption is nowadays a growing concern in IT industry.
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 4 / 23
The problem of energy consumption
I Technical problemI Cities worth of power, power provvisioning is an issueI Higher temperature, more failuresI Mobile devices with limited energy budget.
I Economic issueI Cost of energy > cost of hardware
I Environmental concernI 2% of greenhouse emissions cause by US IT sector → 3% by 2020
Energy consumption is nowadays a growing concern in IT industry.
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 4 / 23
Scenario
Parallel, heterogeneous architectures everywhere!I Even low-end computers have multiple CPU cores and a GPU
I Different architectures, different energy footprint depending on thecomputation
I Saving energy requires efficient resource usage
I Structured parallel application development methodology
I Divide work according to energy consumption, not just time!
We selected a known, widely used data-parallel pattern
Map parallel pattern
I Map: higher order function, taking function f and collection A
I f applied in parallel on different partitionsI suitable for both CPU and GPU architectures
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 5 / 23
Scenario
Parallel, heterogeneous architectures everywhere!I Even low-end computers have multiple CPU cores and a GPUI Different architectures, different energy footprint depending on the
computation
I Saving energy requires efficient resource usage
I Structured parallel application development methodology
I Divide work according to energy consumption, not just time!
We selected a known, widely used data-parallel pattern
Map parallel pattern
I Map: higher order function, taking function f and collection A
I f applied in parallel on different partitionsI suitable for both CPU and GPU architectures
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 5 / 23
Scenario
Parallel, heterogeneous architectures everywhere!I Even low-end computers have multiple CPU cores and a GPUI Different architectures, different energy footprint depending on the
computation
I Saving energy requires efficient resource usage
I Structured parallel application development methodology
I Divide work according to energy consumption, not just time!
We selected a known, widely used data-parallel pattern
Map parallel pattern
I Map: higher order function, taking function f and collection A
I f applied in parallel on different partitionsI suitable for both CPU and GPU architectures
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 5 / 23
Scenario
Parallel, heterogeneous architectures everywhere!I Even low-end computers have multiple CPU cores and a GPUI Different architectures, different energy footprint depending on the
computation
I Saving energy requires efficient resource usageI Structured parallel application development methodology
I Divide work according to energy consumption, not just time!
We selected a known, widely used data-parallel pattern
Map parallel pattern
I Map: higher order function, taking function f and collection A
I f applied in parallel on different partitionsI suitable for both CPU and GPU architectures
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 5 / 23
Scenario
Parallel, heterogeneous architectures everywhere!I Even low-end computers have multiple CPU cores and a GPUI Different architectures, different energy footprint depending on the
computation
I Saving energy requires efficient resource usageI Structured parallel application development methodology
I Divide work according to energy consumption, not just time!
We selected a known, widely used data-parallel pattern
Map parallel pattern
I Map: higher order function, taking function f and collection A
I f applied in parallel on different partitionsI suitable for both CPU and GPU architectures
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 5 / 23
Scenario
Parallel, heterogeneous architectures everywhere!I Even low-end computers have multiple CPU cores and a GPUI Different architectures, different energy footprint depending on the
computation
I Saving energy requires efficient resource usageI Structured parallel application development methodology
I Divide work according to energy consumption, not just time!
We selected a known, widely used data-parallel pattern
Map parallel pattern
I Map: higher order function, taking function f and collection A
I f applied in parallel on different partitionsI suitable for both CPU and GPU architectures
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 5 / 23
Scenario
Parallel, heterogeneous architectures everywhere!I Even low-end computers have multiple CPU cores and a GPUI Different architectures, different energy footprint depending on the
computation
I Saving energy requires efficient resource usageI Structured parallel application development methodology
I Divide work according to energy consumption, not just time!
We selected a known, widely used data-parallel pattern
Map parallel pattern
I Map: higher order function, taking function f and collection A
I f applied in parallel on different partitionsI suitable for both CPU and GPU architectures
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 5 / 23
Scenario
Parallel, heterogeneous architectures everywhere!I Even low-end computers have multiple CPU cores and a GPUI Different architectures, different energy footprint depending on the
computation
I Saving energy requires efficient resource usageI Structured parallel application development methodology
I Divide work according to energy consumption, not just time!
We selected a known, widely used data-parallel pattern
Map parallel patternI Map: higher order function, taking function f and collection A
I f applied in parallel on different partitionsI suitable for both CPU and GPU architectures
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 5 / 23
Scenario
Parallel, heterogeneous architectures everywhere!I Even low-end computers have multiple CPU cores and a GPUI Different architectures, different energy footprint depending on the
computation
I Saving energy requires efficient resource usageI Structured parallel application development methodology
I Divide work according to energy consumption, not just time!
We selected a known, widely used data-parallel pattern
Map parallel patternI Map: higher order function, taking function f and collection A
I f applied in parallel on different partitions
I suitable for both CPU and GPU architectures
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 5 / 23
Scenario
Parallel, heterogeneous architectures everywhere!I Even low-end computers have multiple CPU cores and a GPUI Different architectures, different energy footprint depending on the
computation
I Saving energy requires efficient resource usageI Structured parallel application development methodology
I Divide work according to energy consumption, not just time!
We selected a known, widely used data-parallel pattern
Map parallel patternI Map: higher order function, taking function f and collection A
I f applied in parallel on different partitionsI suitable for both CPU and GPU architectures
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 5 / 23
Scenario
Parallel, heterogeneous architectures everywhere!I Even low-end computers have multiple CPU cores and a GPUI Different architectures, different energy footprint depending on the
computation
I Saving energy requires efficient resource usageI Structured parallel application development methodology
I Divide work according to energy consumption, not just time!
We selected a known, widely used data-parallel pattern
Map parallel patternI Map: higher order function, taking function f and collection A
I f applied in parallel on different partitionsI suitable for both CPU and GPU architectures
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 5 / 23
Problem
ProblemFind a model predicting the energy consumption of different map parallelapplications executed on both CPU and GPU.
Why modelling energy?I Divide tasks properly between CPU and GPU
I Select parallelism degree on both devicesI Autonomic manager: allows to respond to non-functional concernsI Model for the programmerI Aim: Save energy!
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 6 / 23
Problem
ProblemFind a model predicting the energy consumption of different map parallelapplications executed on both CPU and GPU.
Why modelling energy?I Divide tasks properly between CPU and GPUI Select parallelism degree on both devices
I Autonomic manager: allows to respond to non-functional concernsI Model for the programmerI Aim: Save energy!
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 6 / 23
Problem
ProblemFind a model predicting the energy consumption of different map parallelapplications executed on both CPU and GPU.
Why modelling energy?I Divide tasks properly between CPU and GPUI Select parallelism degree on both devicesI Autonomic manager: allows to respond to non-functional concerns
I Model for the programmerI Aim: Save energy!
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 6 / 23
Problem
ProblemFind a model predicting the energy consumption of different map parallelapplications executed on both CPU and GPU.
Why modelling energy?I Divide tasks properly between CPU and GPUI Select parallelism degree on both devicesI Autonomic manager: allows to respond to non-functional concernsI Model for the programmer
I Aim: Save energy!
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 6 / 23
Problem
ProblemFind a model predicting the energy consumption of different map parallelapplications executed on both CPU and GPU.
Why modelling energy?I Divide tasks properly between CPU and GPUI Select parallelism degree on both devicesI Autonomic manager: allows to respond to non-functional concernsI Model for the programmerI Aim: Save energy!
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 6 / 23
Problem
ProblemFind a model predicting the energy consumption of different map parallelapplications executed on both CPU and GPU.
Why modelling energy?I Divide tasks properly between CPU and GPUI Select parallelism degree on both devicesI Autonomic manager: allows to respond to non-functional concernsI Model for the programmerI Aim: Save energy!
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 6 / 23
Dividing map tasks between CPU/GPU cores
Partitioning between CPU and GPU; GPU part transferred through PCIebus
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 7 / 23
Dividing map tasks between CPU/GPU cores
Within the device, data is partitioned among Streaming Multiprocessors(SM)
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 7 / 23
Dividing map tasks between CPU/GPU cores
In a single SM, f is applied on the collection using threads, scheduled inbatch of 32 on CUDA cores
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 7 / 23
Dividing map tasks between CPU/GPU cores
On the host side, the collection is partitioned between the cores, that applyf in parallel
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 7 / 23
Dividing map tasks between CPU/GPU cores
Result is transferred back from GPU to host memory, obtaining the finalresult
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 7 / 23
Energy consumption of a map computation
E =∫ tend
0P(t)dt
I P(t) depends on computation and used resources (parallelism degree)
I Divide energy depending on phase, where P(t) constant
I Phases:
computing (host and device), communication and waiting
For a single phase: E = P×Tphase
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 8 / 23
Energy consumption of a map computation
E =∫ tend
0P(t)dt
I P(t) depends on computation and used resources (parallelism degree)
I Divide energy depending on phase, where P(t) constant
I Phases:
computing (host and device), communication and waiting
For a single phase: E = P×Tphase
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 8 / 23
Energy consumption of a map computation
E =∫ tend
0P(t)dt
I P(t) depends on computation and used resources (parallelism degree)
I Divide energy depending on phase, where P(t) constant
I Phases:
computing (host and device), communication and waiting
For a single phase: E = P×Tphase
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 8 / 23
Energy consumption of a map computation
E =∫ tend
0P(t)dt
I P(t) depends on computation and used resources (parallelism degree)
I Divide energy depending on phase, where P(t) constant
I Phases: computing (host and device),
communication and waiting
For a single phase: E = P×Tphase
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 8 / 23
Energy consumption of a map computation
E =∫ tend
0P(t)dt
I P(t) depends on computation and used resources (parallelism degree)
I Divide energy depending on phase, where P(t) constant
I Phases: computing (host and device), communication
and waiting
For a single phase: E = P×Tphase
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 8 / 23
Energy consumption of a map computation
E =∫ tend
0P(t)dt
I P(t) depends on computation and used resources (parallelism degree)
I Divide energy depending on phase, where P(t) constant
I Phases: computing (host and device), communication and waiting
For a single phase: E = P×Tphase
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 8 / 23
Energy consumption of a map computation
E =∫ tend
0P(t)dt
I P(t) depends on computation and used resources (parallelism degree)
I Divide energy depending on phase, where P(t) constant
I Phases: computing (host and device), communication and waiting
For a single phase: E = P×Tphase
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 8 / 23
Energy consumption of a map computation
E =∫ tend
0P(t)dt
I P(t) depends on computation and used resources (parallelism degree)
I Divide energy depending on phase, where P(t) constant
I Phases: computing (host and device), communication and waiting
For a single phase: E = P×Tphase
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 8 / 23
Section 2
GPU Energy Model
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 9 / 23
Predicting power with regression
Explanatory Variables: b, number of blocks (SM) used; w , number ofwarps (32 threads) for each bPower varies according to
√w for fixed b
P(b,w) = β0(b)+β1(b)√w
Cost of modellingFeasible only for repeated computations, otherwise too high.Time: at least 6 seconds Energy: about 1300 J
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 10 / 23
Predicting power with regression
Explanatory Variables: b, number of blocks (SM) used; w , number ofwarps (32 threads) for each bPower varies according to
√w for fixed b
P(b,w) = β0(b)+β1(b)√w
Cost of modellingFeasible only for repeated computations, otherwise too high.Time: at least 6 seconds Energy: about 1300 J
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 10 / 23
Predicting power with regression
Explanatory Variables: b, number of blocks (SM) used; w , number ofwarps (32 threads) for each bPower varies according to
√w for fixed b
P(b,w) = β0(b)+β1(b)√w
Cost of modellingFeasible only for repeated computations, otherwise too high.Time: at least 6 seconds Energy: about 1300 J
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 10 / 23
Predicting power with regression
Explanatory Variables: b, number of blocks (SM) used; w , number ofwarps (32 threads) for each bPower varies according to
√w for fixed b
P(b,w) = β0(b)+β1(b)√w
I Precise, but costly to calculateI Estimate β0(b) and β1(b) as
functions of b
I trade samples (energyconsumption) with precision
I 6 samples → error below 5.21%
Cost of modellingFeasible only for repeated computations, otherwise too high.Time: at least 6 seconds Energy: about 1300 J
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 10 / 23
Predicting power with regression
Explanatory Variables: b, number of blocks (SM) used; w , number ofwarps (32 threads) for each bPower varies according to
√w for fixed b
P(b,w) = β0(b)+β1(b)√w
I Precise, but costly to calculateI Estimate β0(b) and β1(b) as
functions of bI trade samples (energy
consumption) with precision
I 6 samples → error below 5.21%
Cost of modellingFeasible only for repeated computations, otherwise too high.Time: at least 6 seconds Energy: about 1300 J
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 10 / 23
Predicting power with regression
Explanatory Variables: b, number of blocks (SM) used; w , number ofwarps (32 threads) for each bPower varies according to
√w for fixed b
P(b,w) = β0(b)+β1(b)√w
I Precise, but costly to calculateI Estimate β0(b) and β1(b) as
functions of bI trade samples (energy
consumption) with precision
I 6 samples → error below 5.21%
Cost of modellingFeasible only for repeated computations, otherwise too high.Time: at least 6 seconds Energy: about 1300 J
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 10 / 23
Predicting power with regression
Explanatory Variables: b, number of blocks (SM) used; w , number ofwarps (32 threads) for each bPower varies according to
√w for fixed b
P(b,w) = β0(b)+β1(b)√w
I Precise, but costly to calculateI Estimate β0(b) and β1(b) as
functions of bI trade samples (energy
consumption) with precision
I 6 samples → error below 5.21%
Cost of modellingFeasible only for repeated computations, otherwise too high.Time: at least 6 seconds Energy: about 1300 J
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 10 / 23
Predicting power with regression
Explanatory Variables: b, number of blocks (SM) used; w , number ofwarps (32 threads) for each bPower varies according to
√w for fixed b
P(b,w) = β0(b)+β1(b)√w
I Precise, but costly to calculateI Estimate β0(b) and β1(b) as
functions of bI trade samples (energy
consumption) with precision
I 6 samples → error below 5.21%
Cost of modellingFeasible only for repeated computations, otherwise too high.Time: at least 6 seconds Energy: about 1300 J
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 10 / 23
Predicting power with regression
Explanatory Variables: b, number of blocks (SM) used; w , number ofwarps (32 threads) for each bPower varies according to
√w for fixed b
P(b,w) = β0(b)+β1(b)√w
I Precise, but costly to calculateI Estimate β0(b) and β1(b) as
functions of bI trade samples (energy
consumption) with precision
I 6 samples → error below 5.21%
Cost of modellingFeasible only for repeated computations, otherwise too high.Time: at least 6 seconds Energy: about 1300 J
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 10 / 23
Heuristic model for power estimation
IdeaI Similar behaviours for map on GPU
I Use a computation as metre to measureothers
I Study the metre M, use to predict theothers
We define αC , increment in power of C divided by increment of metre
αC (b,w) =PC (b,w)− (Pbase +Pblockb)
PM(b,w)− (Pbase +Pblockb)
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 11 / 23
Average Power for different computations using eight blocks and varying the amount of warps on Nvidia K20c
Vector AddMatrix Add
Matrix Multiplication
We define αC , increment in power of C divided by increment of metre
αC (b,w) =PC (b,w)− (Pbase +Pblockb)
PM(b,w)− (Pbase +Pblockb)
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 11 / 23
αC analysis and power estimator
I αC concentrated near αC
I over more than 2000 samples, 87% ofvalues of αC within ±1 from the average
Power predictor using αC
PC (b,w) = Pbase +αC (Pwarp+P ′warpb)√w
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 12 / 23
αC analysis and power estimator
I αC concentrated near αC
I over more than 2000 samples, 87% ofvalues of αC within ±1 from the average
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4253997603
0.9211608897
1.4169220192
1.9126831486
2.408444278
2.9042054075
3.3999665369
3.8957276664
4.3914887958
4.8872499252
5.3830110547
Freq
uenc
y
values
Empirical distribution of alpha for vector addition
Empirical frequency
Power predictor using αC
PC (b,w) = Pbase +αC (Pwarp+P ′warpb)√w
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 12 / 23
αC analysis and power estimator
I αC concentrated near αC
I over more than 2000 samples, 87% ofvalues of αC within ±1 from the average
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4253997603
0.9211608897
1.4169220192
1.9126831486
2.408444278
2.9042054075
3.3999665369
3.8957276664
4.3914887958
4.8872499252
5.3830110547
Freq
uenc
y
values
Empirical distribution of alpha for vector addition
Empirical frequency
Power predictor using αC
PC (b,w) = Pbase +αC (Pwarp+P ′warpb)√w
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 12 / 23
αC analysis and power estimator
I αC concentrated near αC
I over more than 2000 samples, 87% ofvalues of αC within ±1 from the average
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4253997603
0.9211608897
1.4169220192
1.9126831486
2.408444278
2.9042054075
3.3999665369
3.8957276664
4.3914887958
4.8872499252
5.3830110547
Freq
uenc
y
values
Empirical distribution of alpha for vector addition
Empirical frequency
Power predictor using αC
PC (b,w) = Pbase +αC (Pwarp+P ′warpb)√w
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 12 / 23
Validation of the heuristic
Heuristic is convenient if a single sample returns accurate estimation. Wecan model power with low energy footprint!
Validation
I Sample power for 500 random (b, w) pairs, calculate αC (b,w)
I try to predict power for random (b’, w’) using the achieved αC (b,w)
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 13 / 23
Validation of the heuristic
Heuristic is convenient if a single sample returns accurate estimation. Wecan model power with low energy footprint!
ValidationI Sample power for 500 random (b, w) pairs, calculate αC (b,w)
I try to predict power for random (b’, w’) using the achieved αC (b,w)
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 13 / 23
Validation of the heuristic
Heuristic is convenient if a single sample returns accurate estimation. Wecan model power with low energy footprint!
ValidationI Sample power for 500 random (b, w) pairs, calculate αC (b,w)
I try to predict power for random (b’, w’) using the achieved αC (b,w)
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 13 / 23
Validation of the heuristic
Heuristic is convenient if a single sample returns accurate estimation. Wecan model power with low energy footprint!
ValidationI Sample power for 500 random (b, w) pairs, calculate αC (b,w)
I try to predict power for random (b’, w’) using the achieved αC (b,w)
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
0.11
0.12
0.13
0.14
0.15
-15.81%
-14.48%
-13.15%
-11.82%
-10.49%
-9.16%-7.84%
-6.51%-5.18%
-3.85%-2.52%
-1.19%0.13%
1.46%2.79%
4.12%5.45%
6.78%8.10%
9.43%10.76%
12.09%
13.42%
14.74%
16.07%
17.40%
18.73%
20.06%
21.39%
22.71%
24.04%
25.37%
26.70%
28.03%
29.36%
30.68%
32.01%
33.34%
34.67%
36.00%
37.32%
Erro
r pro
babi
lity
Error
Error distribution using the heuristic over 500 samples. Nvidia K40m
Error distribution
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 13 / 23
Validation of the heuristic
Heuristic is convenient if a single sample returns accurate estimation. Wecan model power with low energy footprint!
ValidationI Sample power for 500 random (b, w) pairs, calculate αC (b,w)
I try to predict power for random (b’, w’) using the achieved αC (b,w)
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
0.11
0.12
0.13
0.14
0.15
-15.81%
-14.48%
-13.15%
-11.82%
-10.49%
-9.16%-7.84%
-6.51%-5.18%
-3.85%-2.52%
-1.19%0.13%
1.46%2.79%
4.12%5.45%
6.78%8.10%
9.43%10.76%
12.09%
13.42%
14.74%
16.07%
17.40%
18.73%
20.06%
21.39%
22.71%
24.04%
25.37%
26.70%
28.03%
29.36%
30.68%
32.01%
33.34%
34.67%
36.00%
37.32%
Erro
r pro
babi
lity
Error
Error distribution using the heuristic over 500 samples. Nvidia K40m
Error distribution
Less than 10% error with high probability (over 85% of the cases), lessthan 15% error with probability over 95%!Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 13 / 23
Estimating GPU energy consumption
Energy = PC (b,w)×T (b,w)
T (b,w) = d Nb×w×32e×TGPU
f ' T (1,1)b×w
I Error below 15% using a single sample with over 95% probability!
I Same for other computations and devices until threads map to CUDA cores.
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 14 / 23
Estimating GPU energy consumption
Energy = PC (b,w)×T (b,w)
T (b,w) = d Nb×w×32e×TGPU
f ' T (1,1)b×w
I Error below 15% using a single sample with over 95% probability!
I Same for other computations and devices until threads map to CUDA cores.
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 14 / 23
Estimating GPU energy consumption
Energy = PC (b,w)×T (b,w)
T (b,w) = d Nb×w×32e×TGPU
f ' T (1,1)b×w
0 0.01
0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09
0.1 0.11 0.12
0.13 0.14 0.15 0.16
-14.82%
-12.34%
-9.86%-7.38%
-4.90%-2.43%
0.05%2.53%
5.01%7.49%
9.96%12.44%
14.92%
17.40%
19.87%
22.35%
24.83%
27.31%
29.79%
32.26%
34.74%
Erro
r pro
babi
lity
Error
Error in energy estimation distribution, using the heuristic over 500 samples. Nvidia K40m
Error distribution
I Error below 15% using a single sample with over 95% probability!
I Same for other computations and devices until threads map to CUDA cores.
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 14 / 23
Estimating GPU energy consumption
Energy = PC (b,w)×T (b,w)
T (b,w) = d Nb×w×32e×TGPU
f ' T (1,1)b×w
0 0.01
0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09
0.1 0.11 0.12
0.13 0.14 0.15 0.16
-14.82%
-12.34%
-9.86%-7.38%
-4.90%-2.43%
0.05%2.53%
5.01%7.49%
9.96%12.44%
14.92%
17.40%
19.87%
22.35%
24.83%
27.31%
29.79%
32.26%
34.74%
Erro
r pro
babi
lity
Error
Error in energy estimation distribution, using the heuristic over 500 samples. Nvidia K40m
Error distribution
I Error below 15% using a single sample with over 95% probability!
I Same for other computations and devices until threads map to CUDA cores.
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 14 / 23
Estimating GPU energy consumption
Energy = PC (b,w)×T (b,w)
T (b,w) = d Nb×w×32e×TGPU
f ' T (1,1)b×w
0 0.01
0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09
0.1 0.11 0.12
0.13 0.14 0.15 0.16
-14.82%
-12.34%
-9.86%-7.38%
-4.90%-2.43%
0.05%2.53%
5.01%7.49%
9.96%12.44%
14.92%
17.40%
19.87%
22.35%
24.83%
27.31%
29.79%
32.26%
34.74%
Erro
r pro
babi
lity
Error
Error in energy estimation distribution, using the heuristic over 500 samples. Nvidia K40m
Error distribution
I Error below 15% using a single sample with over 95% probability!
I Same for other computations and devices until threads map to CUDA cores.
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 14 / 23
Section 3
CPU Energy Model
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 15 / 23
Predicting power of a map computation
Again: Energy = (Average) Power × TimeI Explanatory variable for power is n, number of cores usedI Power grows linearly in the number of workers used in the computation
40
50
60
70
80
90
100
110
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Pow
er (W
)
Workers
Average power varying the number of processing units allocated to different map computations
Vector AddMap Atan
Matrix add
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 16 / 23
Predicting power of a map computation
Again: Energy = (Average) Power × TimeI Explanatory variable for power is n, number of cores usedI Power grows linearly in the number of workers used in the computation
40
50
60
70
80
90
100
110
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Pow
er (W
)
Workers
Average power varying the number of processing units allocated to different map computations
Vector AddMap Atan
Matrix add
Alessandro Lenzi Energy models in data parallel CPU/GPU computations December 4, 2015 16 / 23
Energy consumption using regression
Power predictor (with regression):
Pc(n) = β0+β1×n
No need for an heuristic!
I More precise measures, no data transferI 2 executions with different parallelism degrees are enough to find