Datacenter Simulation Methodologies Case Studiesleebcc/tutorial/dsm15... · Datacenter Simulation Methodologies Case Studies Tamara Silbergleit Lehman, Qiuyun Wang, Seyed Majid Zahedi

Datacenter Simulation MethodologiesCase Studies

Tamara Silbergleit Lehman, Qiuyun Wang, Seyed Majid Zahediand Benjamin C. Lee

This work is supported by NSF grants CCF-1149252, CCF-1337215, and STARnet, a SemiconductorResearch Corporation Program, sponsored by MARCO and DARPA.

Tutorial Schedule

Time Topic

09:00 - 09:30 Introduction09:30 - 10:30 Setting up MARSSx86 and DRAMSim210:30 - 11:00 Break11:00 - 12:00 Spark simulation12:00 - 13:00 Lunch13:00 - 13:30 Spark continued13:30 - 14:30 GraphLab simulation14:30 - 15:00 Break15:00 - 16:15 Web search simulation16:15 - 17:00 Case studies

2 / 45

Big Computing and its Challenges

Big data demands big computing, yet we face challenges...

• Architecture Design

• Systems Management

• Research Coordination

3 / 45

Toward Energy-Efficient Datacenters

Heterogeneity

• Tailors hardware to software, reducing energy

• Complicates resource allocation and scheduling

• Introduces risk

Sharing

• Divides hardware over software, amortizing energy

• Complicates task placement and co-location

• Introduces risk

4 / 45

Datacenter Design and Management

Heterogeneity for Efficiency

Heterogeneous datacenters deploy mix of server- and mobile-class hardware

• “Web search using mobile cores” [ISCA’10]• “Towards energy-proportional datacenter memory with mobile DRAM” [ISCA’12]

Heterogeneity and MarketsAgents bid for heterogeneous hardware in a market that maximizes welfare

• “Navigating heterogeneous processors with market mechanisms” [HPCA’13]• “Strategies for anticipating risk in heterogeneous datacenter design” [HPCA’14]

Sharing and Game TheoryAgents share multiprocessors with game-theoretic fairness guarantees

• “REF: Resource elasticity fairness with sharing incentives for multiprocessors” [ASPLOS’14]

5 / 45

Mobile versus Server Processors

• Simpler Datapath

• Issue fewer instructions per cycle

• Speculate less often

• Smaller Caches

• Provide less capacity

• Provide lower associativity

• Slower Clock

• Reduce processor frequency

Atom v. Xeon [Intel]

6 / 45

Applications in Transition

Conventional Enterprise

• Independent requests• Memory-, I/O-intensive• Ex: web or file server

Emerging Datacenter

• Inference, analytics• Compute-intensive• Ex: neural network

Reddi et al., “Web search using mobile cores” [ISCA’10]

7 / 45

Web Search

• Distribute web pagesamong index servers

• Distribute queries amongindex servers

• Rank indexed pages withneural network

Bing.com [Microsoft]

8 / 45

Query Efficiency

• Joules per second :: ↓ 10× on Atom versus Xeon• Queries per second :: ↓ 2×• Queries per Joule :: ↑ 5×


9 / 45

Case for Processor Heterogeneity

Mobile Core Efficiency

• Queries per Joule ↑ 5×

Mobile Core Latency

• 10% queries exceed cut-off• Complex queries suffer

Heterogeneity

• Small cores for simple queries• Big cores for complex queries


10 / 45

Memory Architecture and Applications

Conventional Enterprise

• High bandwidth• Ex: transaction processing

Emerging Datacenter

• Low bandwidth (< 6% DDR3 peak)• Ex: search [Microsoft],

memcached [Facebook]

11 / 45

Memory Capacity vs Bandwidth

• Online Services• Use < 6% bandwidth, 65-97% capacity

• Ex: Microsoft mail, map-reduce, search [Kansal+]

• Memory Caching• 75% of Facebook data in memory

• Ex: memcached, RAMCloud [Ousterhout+]

• Capacity-Bandwidth Bundles• Server with 4 sockets, 8 channels

• Ex: 32GB capacity, >100GB/s bandwidth

12 / 45

Mobile-class Memory

• Operating Parameters

• Lower active current (130mA vs 250mA)

• Lower standby current (20mA vs 70mA)

• Low-power Interfaces

• No delay-locked loops, on-die termination

• Lower bus frequency (400 vs 800MHz)

• Lower peak bandwidth (6.4 vs 12.8GBps)

LP-DDR2 vs DDR3 [Micron]

13 / 45

Source of Disproportionality

Activity Example

• 16% DDR3 peak

Energy per Bit

• Large power overheads• High cost per bit

“Calculating memory system power for DDR3” [Micron]

14 / 45

Case for Memory Heterogeneity

Mobile Memory Efficiency

• Bits / Joule ↑ 5×

Mobile Memory Bandwidth

• Peak B/W ↓ 0.5×

Heterogeneity

• LPDDR for search, memcached• DDR for databases, HPC

Malladi et al., “Towards energy-proportional datacenter memory with mobile DRAM” [ISCA’12]

15 / 45






• “Navigating heterogeneous processors with market mechanisms” [HPCA’13]• “Strategies for anticipating risk in heterogeneous system design” [HPCA’14]



16 / 45

Datacenter Heterogeneity

Systems are heterogeneous

• Virtual machines are sized• Physical machines are diverse

Heterogeneity is exposed

• Users assess machine price• Users select machine type

Burden is prohibitive

• Users must understandhardware-software interactions

Elastic Compute Cloud (EC2) [Amazon]

17 / 45

Managing Performance Risk

Risk: the possibility that something bad will happen

Understand Heterogeneity and Risk

• What types of hardware?• How many of each type?• What allocation to users?

Mitigate Risk with Market Allocation

• Ensure service quality• Hide hardware complexity• Trade-off performance and power

18 / 45

Market Mechanism

• User specifies value for performance• Market shields user from heterogeneity

Guevara et al. “Navigating heterogeneous processors with market mechanisms” [HPCA’13]

19 / 45

Proxy Bidding

User Provides...

• Task stream• Service-level agreement

Proxy Provides...

• µarchitectural insight• Performance profiles• Bids for hardware


Wu and Lee, “Inferred models for dynamic and sparse hardware-software spaces” [MICRO’12]

20 / 45

Visualizing Heterogeneity (2 Processor Types)

Low−powerHigh−performance

Heterogeneous

Homogeneous 0%

1%

2%

3%

4%

5%

6%

7%

8%• Ellipses represent

hardware types

• Points are combinationsof processor types

• Colors showQoS violations


21 / 45

Further Heterogeneity (4 Processor Types)

oo6w24

io4w24 io1w24

io1w10

A

B C

D

E

F

G

H

I

J

K

L M

N

O

2%

4%

6%

8%

10%

12%

14%

16%

18% • Best configuration isheterogeneous

• QoS violations fall16% → 2%

• Trade-offs motivatedesign for manageability


Guevara et al. “Strategies for anticipating risk in heterogeneous datacenter design” [HPCA’14]

22 / 45






• “Navigating heterogeneous processors with market mechanisms” [HPCA’13]• “Strategies for anticipating risk in heterogeneous datacenter design” [HPCA’14]



23 / 45

Case for Sharing

Big Servers

• Hardware is under-utilized• Sharing amortized power

Heterogeneous Users

• Tasks are diverse• Users are complementary• Users prefer flexibility

Sharing Challenges

• Allocate multiple resources• Ensure fairness

Image: Intel Sandy Bridge E die [www.anandtech.com]

24 / 45

Motivation

• Alice and Bob are working on research papers

• Each has $10K to buy computers

• Alice and Bob have different types of tasks

• Alice and Bob have different paper deadlines

25 / 45

Strategic Behavior

• Alice and Bob are strategic

• Which is better?• Small, separate clusters• Large, shared cluster

• Suppose Alice and Bob share• Is allocation fair?• Is lying beneficial?

Image: [www.websavers.org]

26 / 45

Conventional Wisdom in Computer Architecture

Users must share

• Overlooks strategic behavior

Fairness policy is equal slowdown

• Fails to encourage envious users to share

Heuristic mechanisms enforce equal slowdown

• Fail to give provable guarantees

27 / 45

Rethinking Fairness

”If an allocation is both equitable and Pareto efficient,... it is fair.” [Varian, Journal of Economic Theory (1974)]

28 / 45

Resource Elasticity Fairness (REF)

REF is an allocation mechanism that guarantees game-theoreticdesiderata for shared chip multiprocessors

• Sharing IncentivesUsers perform no worse than under equal division

• Envy-FreeNo user envies another’s allocation

• Pareto-EfficientNo other allocation improves utility without harming others

• Strategy-ProofNo user benefits from lying

Zahedi et al. “REF: Resource elasticity fairness with sharing incentives for multiprocessors” [ASPLOS’14]

29 / 45

Cobb-Douglas Utility

u(x) =∏R

r=1 xαrr

u utility (e.g., performance)xr allocation for resource r (e.g., cache size)αr elasticity for resource r

• Cobb-Douglas fits preferences in computer architecture

• Exponents model diminishing marginal returns

• Products model substitution effects


30 / 45

Example Utilities

u1 = x0.61 y

0.41 u2 = x

0.22 y

0.82

u1,u2 performancex1, x2 allocated memory bandwidthy1, y2 allocated cache size


31 / 45

Possible Allocations

• 2 users

• 12MB cache

• 24GB/s bandwidth

Memory Bandwidth

CacheSize

0 5 10 15 20 24

0510152024

0

2

4

6

8

10

12 0

2

4

6

8

10

12

user2

user1


32 / 45

Envy-Free (EF) Allocations

• Identify EF allocationsfor each user

• u1(A1) ≥ u1(A2)

• u2(A2) ≥ u2(A1)

Memory Bandwidth

Cac

he

Siz

e

0 5 10 15 20 24

0510152024

0

2

4

6

8

10

12 0

2

4

6

8

10

12

EF Region

Memory Bandwidth

Cac

he

Siz

e

0 5 10 15 20 24

0510152024

0

2

4

6

8

10

12 0

2

4

6

8

10

12

EF Region


33 / 45

Pareto-Efficient (PE) Allocations

• No other allocation improves utility without harming others

Memory Bandwidth

Cac

he

Siz

e

0 5 10 15 20 24

0510152024

0

2

4

6

8

10

12 0

2

4

6

8

10

12

Contract Curve


34 / 45

Fair Allocations

Fairness = Envy-freeness + Pareto efficiency

Memory Bandwidth

Cac

he

Siz

e

0 5 10 15 20 24

0510152024

0

2

4

6

8

10

12 0

2

4

6

8

10

12

Fair Allocations

Many possible fair allocations!


35 / 45

Mechanism for Resource Elasticity Fairness

Profile preferences

Fit utility function

Normalize elasticities

Allocate proportionally

Guarantees desiderata

• Sharing incentives• Envy-freeness• Pareto efficiency• Strategy-proofness


36 / 45

Profiling for REF

Profile preferences




Off-line profiling

• Synthetic benchmarks

Off-line simulations

• Various hardware

Machine learning

• α = 0.5, then update


37 / 45

Fitting Utilities

Profile preferences




• u =∏R

r=1 xαrr

• log(u) =∑R

r=1 αr log(xr)

• Use linear regression to find αr


38 / 45

Cobb-Douglas Accuracy

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Ferret Sim. Ferret Est.

Fmm Sim. Fmm Est.

IPC

• Utility is instructions per cycle

• Resources are cache size, memory bandwidth


39 / 45

Normalizing Utilities

Profile preferences




• Compare users’ elasticitieson same scale

• u = x0.2y0.3 → u = x0.4y0.6


40 / 45

Allocating Proportional Shares

Profile preferences




u1 = x0.61 y

0.41 u2 = x

0.22 y

0.82

x1 =(

0.60.6+0.2

)× 24 = 18GB/s

x2 =(

0.20.6+0.2

)× 24 = 6GB/s


41 / 45

Equal Slowdown versus REFR

esourc

e A

lloca

tion

(% o

f To

tal C

apaci

ty)

• Equal slow-down providesneither SI nor EF

• Canneal receives < half ofcache, memory

Reso

urc

e A

lloca

tion

(% o

f To

tal C

apaci

ty)

• Resource elasticity fairnessprovides both SI and EF

• Canneal receives morecache, less memory


42 / 45

Equal Slowdown versus REFR

esourc

e A

lloca

tion

(% o

f To

tal C

apaci

ty)

• Equal slow-down providesneither SI nor EF

• Canneal receives < half ofcache, memory

Reso

urc

e A

llocatio

n(%

of To

tal C

apaci

ty)

• Resource elasticity fairnessprovides both SI and EF

• Canneal receives morecache, less memory


42 / 45

Performance versus Fairness

WD1 (4C) WD2 (2C-2M) WD3 (4M) WD4 (3C-1M) WD5 (1C-3M)0

0.5

1

1.5

2

2.5

3

3.5

4

Max Welfare w/ FairnessProportional Elasticity w/ Fairness

Max Welfare w/o FairnessEqual Slowdown w/o Fairness

Wei

ghte

d S

yste

m T

hro

ughp

ut

• Measure weighted instruction throughput

• REF incurs < 10% penalty


43 / 45






• “Navigating heterogeneous processors with market mechanisms” [HPCA’13]• “Strategies for anticipating risk in heterogeneous system design” [HPCA’14]



44 / 45




• Processors – hardware counters for CPI stack• Memories – simulator for cache, bandwidth activity


• Processors – simulator for core performance• Server Racks – queueing models (e.g., M/M/1)


• Memories – simulator for cache, bandwidth utility

45 / 45

Datacenter Simulation Methodologies Case Studiesleebcc/tutorial/dsm15... · Datacenter Simulation Methodologies Case Studies Tamara Silbergleit Lehman, Qiuyun Wang, Seyed Majid Zahedi

Documents