INSTITUTE OF COMPUTING TECHNOLOGY BigDataBench: a Big Data Benchmark Suite from Internet Services Lei Wang, Jianfeng Zhan, Chunjie Luo, Yuqing Zhu, Qiang Yang, Yongqiang He, Wanling Gao, Zhen Jia, Yingjie Shi, Shujie Zhang, Gang Lu, Kent Zhang, Xiaona Li, and Bizhu Qiu HPCA 2014 1
38
Embed
INSTITUTE OF COMPUTING TECHNOLOGY BigDataBench: a Big Data Benchmark Suite from Internet Services Lei Wang, Jianfeng Zhan, Chunjie Luo, Yuqing Zhu, Qiang.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
INS
TIT
UTE O
F C
OM
PU
TIN
G
TEC
HN
OLO
GY
BigDataBench: a Big Data Benchmark Suite from Internet Services
Floating Point Operation Intensity (E5310)Total number of floating point instructions divided by total number of memory access bytes in a run of workload.
Very low floating point operation intensity : two orders of magnitude lower than in the traditional workloads
Data Analytics Services
CPU Type Intel CPU Core
Intel Xeon E5310 4 Cores @ 1.6G
L1 Cache L1 Cache L2 Cache L3 Cache
4*32KB 4*32KB 2*4MB None
Orlando, 2014.2.18 HPCA 2014
Sort
Grep
Wordcount
BFS
PageRankIndex
Kmeans
Connected Components
Collaborative
Filterin
g
Naive Baye
s
Select
Query
Join Query
Aggregate Query
Nutch Se
rver
Olio Se
rver
Rubis Serve
rRead
Write
Scan
Avg_BigData
Avg_HPCC
Avg_Parse
c
SPECFP
SPECIN
T0.001
0.01
0.1
1
10E5310
E5645
Floa
ting
Poin
t Ope
ratio
ns p
er B
yte
Floating Point Operation Intensity
Data Analytics Services
Floating point operation intensity on E5645 is higher than that on E5310
• Integer operation intensity is in the same order like the traditional workloads
Integer operation intensity on E5645 is higher than that on E5310• L3 Cache is effective & Bandwidth improvement
Orlando, 2014.2.18 HPCA 2014
Possible reasons (Xeon E5645 vs. Xeon E5310)
• More cores in one processor
• Deeper cache hierarchy level: L1~L3 vs. L1~L2
• Larger bandwidth in Front Side Bus
Sixe cores in Xeon E5645 vs. four cores in Xeon E5310
L3 cache is effective in decreasing memory access traffic for big data workloads
Xeon E5645 adopts Intel QuickPath Interconnect (QPI) to eliminate bottlenecks in Front Side Bus [ASPLOS 2012]
• Hyperthreading technology
Hyperthreading can improve performance by factors of 1.3~1.6 times for scale-out workloads
Technique improvements of Xeon E5645:
Orlando, 2014.2.18 HPCA 2014
Sort
Grep
Word
count
BFS
PageR
ank
Index
Kmeans
Connected
Collaborati
ve Fi
lterin
g
Naïve b
ayes
Selec
t Query
Aggreg
ate Q
uery
Join Query
Nutch Se
rver
Olio Se
rver
Rubis Serv
erRea
dW
rite
Scan
Avg_B
igData
Avg_H
PCC
Avg_P
arsec
SPEC
FP
SPEC
INT0
5
10
15
20
25
30
35
40
L1I Misses L2 Misses L3 Misses
Miss
es P
KICache Behaviors
Higher L1I Cache misses than the traditional workloads Data analytic workloads have better L2 Cache behaviors than service
workloads with the exception of BFS Good L3 Cache behaviors
Data Analytics Services56 74 83
Orlando, 2014.2.18 HPCA 2014
TLB Behaviors
data analysis service
14 5
Higher ITBL misses than the traditional workloads
Orlando, 2014.2.18 HPCA 2014
Computation intensity (integer operations)
Integer Operations per Byte (Receiving from networks)
Integer Operations per Byte (Memory Accesses)
X axis : (total number of integer instructions)/(total memory access bytes) Higher : execute more integer operations between two memory accesses
Y axis : (total number of integer instructions)/(total bytes receiving from networks) Higher : execute more integer operations on the same receiving bytes
Orlando, 2014.2.18 HPCA 2014
Big Workloads Characterization Summary Data movement dominated computing
Low computation intensity
Cache Behaviors (Xeon E5645) Very high L1I MPKI L3 Cache is effective
Diverse workload behaviors Computation/communication vs. computation/memory
accesses
Orlando, 2014.2.18 HPCA 2014
Outline
Benchmarking Methodology and Decision
Big Data Workload Characterization
Evaluating Hardware Systems with Big Data Y. Shi, S. A. McKee et al. Performance and Energy Efficiency
Implications from Evaluating Four Big Data Systems, Submitted to IEEE Micro.
Conclusion
3
3
Orlando, 2014.2.18 HPCA 2014
State-of-art Big Data System Architectures
Wimpy many-core processors
Wimpy multi-core processors
Brawny-core processors
Big Data System & Architecture Trends
Hardware Designers: What are the best big data system and architectures in terms of both performance and energy efficiency?
Data Center Administrators: How to choose appropriate hardware for big data applications?
Deploy the systems with the same network and disk configurations Provide about 1GB memory for each hardware thread /
core Adjust the Hadoop parameters to optimize performance
Orlando, 2014.2.18 HPCA 2014
Metrics
Performance : Data processed per second (DPS) Energy Efficiency : Data processed per joule(DPJ)
Report DPS and DPJ per processor
Orlando, 2014.2.18 HPCA 2014
General Observations
The Average DPS Comparison The Average DPJ Comparison
I/O intensive workload (Sort):many-core TileGx36 achieves the best performance and energy efficiency, The brawny-core processors do not provide performance advantages.
CPU-intensive and floating point operation dominated workloads (Bayes & K-means) : brawny-core processors show obvious performance advantages with close energy efficiency to wimpy-core processors.
Other workloads: no platform consistently wins in terms of both performance and energy efficiency.
Report the average number only when the data sizes bigger than 8GB (not fully utilized on small data sizes).
Orlando, 2014.2.18 HPCA 2014
Improvements from Scaling-out the Wimpy Core (TileGx36 vs. Atom D510)
• The core of TileGx36 is more wimpy than Atom D510
• TileGx36 integrates more cores on the NOC(Network on Chip)
Adopts MIPS-derived VLIW instruction set.Does not support hyperthreading. Less stages in the pipeline depth.Does not have dedicated floating point units.
36 cores in TileGx36 vs. 4 cores Atom D510
Orlando, 2014.2.18 HPCA 2014
Improvements from Scaling-out the Wimpy Core (TileGx36 vs. Atom D510)
The DPS Comparison The DPJ Comparison
I/O intensive workload (Sort): TileGx36 shows 4.1 times performance improvement, 1.01 times energy improvement (on average).
500M 2G 8G 32G 64G 128G0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
SortGrepWordcountBayesKemansAggregationJoinSelect
DPJ
Nor
mal
ized
to
Ato
m D
510
CPU-intensive and floating point operation dominated workloads(Bayes & K-means): TileGx36 shows 2.5 times performance advantage and 0.7 times energy efficiency (on average).
Other workloads: TileGx36 shows 2.5 times performance improvement, 1.03 times energy improvement (on average).
500M 2G 8G 32G 64G 128G0
1
2
3
4
5
6
7
DPS
Nor
mal
ized
to
Ato
m D
510
500M 2G 8G 32G 64G 128G0
1
2
3
4
5
6
7
DPS
Nor
mal
ized
to
Ato
m D
510
500M 2G 8G 32G 64G 128G0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
SortGrepWordcountBayesKemansAggregationJoinSelect
DPJ
Nor
mal
ized
to
Ato
m D
510
500M 2G 8G 32G 64G 128G0
1
2
3
4
5
6
7
DPS
Nor
mal
ized
to
Ato
m D
510
500M 2G 8G 32G 64G 128G0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
SortGrepWordcountBayesKemansAggregationJoinSelect
DPJ
Nor
mal
ized
to
Ato
m D
510
Orlando, 2014.2.18 HPCA 2014
Improvements from Scaling-out the Wimpy Core (TileGx36 vs. Atom D510)
• The core of TileGx36 is more wimpy than Atom D510
• TileGx36 integrates more cores on the NOC(Network on Chip)
Adopts MIPS-derived VLIW instruction set.Does not support hyperthreading. Less stages in the pipeline depth.Does not have dedicated floating point units.
36 cores in TileGx36 vs. 4 cores Atom D510
Scaling out the wimpy core can bring performance advantage by improving execution parallelism.
Simplifying the wimpy cores and integrating more cores on the NOC is an option for Big Data workloads.
Orlando, 2014.2.18 HPCA 2014
Scale-up the Brawny Core(Xeon E5645) vs. Scale-out the Wimpy Core (TileGx36)
The DPS Comparison The DPJ Comparison
I/O intensive workload (Sort): TileGx36 shows 1.2 times performance improvement, 1.9 times energy improvement (on average).
500M 2G 8G 32G 64G 128G0
1
2
3
4
5
6
7
DPS
Nor
mal
ized
to
Tile
Gx3
6
500M 2G 8G 32G 64G 128G0
0.5
1
1.5
2
2.5
3
3.5
4
SortGrepWordcountBayesKemansAggregationJoinSelect
DPJ
Nor
mal
ized
to
Tile
Gx3
6
CPU-intensive and floating point operation dominated workloads (Bayes & K-means): E5645 shows 4.2 times performance improvement, 2.0 times energy improvement (on average).
Other workloads: E5645 shows performance advantage, but with no consistent energy improvement.