Testing adaptive workload management Harumi Kuno HP Labs Stefan Krompass (TUM), Kevin Wilkinson, Umeshwar Dayal, Goetz Graefe, Janet Wiener
Testing adaptive workload management
Harumi KunoHP Labs
Stefan Krompass (TUM), Kevin Wilkinson, Umeshwar Dayal, Goetz Graefe, Janet Wiener
Compile-timeQuery
Optimization
Run-timeQuery
Execution
QueryPlan
Performance
Actualconditions
(work + resources)
Expectedconditions
(work + resources)
2
Controlling resources for a complex (dynamic mixed) workload is hard
Traditional solution is to isolate components, by partitioning work across hardware or time multiplexing.
3
Grand Challenge (1):Managing dynamic mixed workloads
• Ignore it • Avoid problem through isolating systems or
using time multiplexing• Provide rich tools to be used with manual
workload management
• Adaptive workload management
4
Dynamic mixed workloads are difficult to manage because resource contention
… changes resource requirements
Disk and memory usage needed to execute a sort changes with the amount of available memory
Goetz Graefe, Harumi Kuno, Janet Wiener. Visualizing the Robustness of Query Execution. Proc. Conference on Innovative Data Systems Research (CIDR). January 4-7, 2009.
5
Dynamic mixed workloads are difficult to manage because resource contention
… changes performance
Throughput/MPL for a single homogenous workload varies with different cache hit rates
6
Janet L. Wiener, Harumi Kuno, Goetz Graefe. Benchmarking Query Execution Robustness. First TPC Technology Conference on Performance Evaluation and Benchmarking (http://www.tpc.org/tpctc2009), held in conjunction with VLDB 2009.
Dynamic mixed workloads are difficult to manage because resource contention
… and is difficult to predict
Throughput/MPL for OLTP queries changes as various report queries run
Report 1
Report 3Report 2
7Stefan Krompass, Harumi Kuno, Janet L.Wiener, Kevin Wilkinson, Umeshwar Dayal, Alfons Kemper. A Testbed for Managing Dynamic Mixed Workloads. Demonstration at VLDB 2009.
Can static workload management policies handle unreliable cost estimates?
none
Admission control threshold
0
20
40
60
80
100
120
140
160
Weig
hted
mak
espa
n (i
n th
ousa
nds o
f sim
ulat
or ti
me u
nits)
3A3C
0A0C
3A3C
3A2C3A0C
3A0C
0A0C0A0C0A0C
0A0C
1.0m
absolute 12000
penalty absolute 5000none absolute 5000,
progress <30%relative 1.2x
under-informed admission control and scheduling decisions
8Stefan Krompass, Harumi Kuno, Janet L.Wiener, Kevin Wilkinson, Umeshwar Dayal, Alfons Kemper. A Testbed for Managing Dynamic Mixed Workloads. Demonstration at VLDB 2009.
Can static workload management policies handleunobserved resource contention?
monitored resource not the source of contention
9Stefan Krompass, Harumi Kuno, Janet Wiener, Kevin Wilkinson, Umeshwar Dayal, Alfons Kemper. Managing Long-Running Queries. Proc. EDBT 2009.
Can static workload management policies handle system overload?
0
20
40
60
80
100
120
140
160
Weig
hted
mak
espa
n(in
thou
sand
s of s
imul
ator
tim
e uni
ts)
2A2C 2A0C2A0C
2A0C2A0C 2A0C
2A0C
2A0C
2A0C
2A2C
MPL 4 MPL 10 (overload)
penalty
relative 1.2xabsolute 12000none
absolute 5000absolute 5000,progress <30%
No. Challenge: Static policies have a hard time correcting overload situations.
10Stefan Krompass, Harumi Kuno, Janet Wiener, Kevin Wilkinson, Umeshwar Dayal, Alfons Kemper. Managing Long-Running Queries. Proc. EDBT 2009.
Adaptive Workload Management Hypothesis
We can build a system that uses feedback to add a policy control loop.
11‹#›Stefan Krompass, Harumi Kuno, Janet Wiener, Kevin Wilkinson, Umeshwar Dayal, Alfons Kemper. Managing Long-Running Queries. Proc. EDBT 2009.
Testing Adaptive Workload Management
• Functionality of management tools• Configuration for a particular anticipated workload
• How it handles the unexpected.
12
Grand Challenge (2)
Underlying challenges• Characterize workloads and service classes: queries,
mega-queries (graphs of queries considered as a single unit), loads, continuous inserts, etc. Plus usual features: estimated arrival rates, execution times, resource usage, etc., together with SLOs.
• Map the components of the workload to service classes.
• Develop recommendations for policies for the query control loop and policy control loop.
• Consider elasticity (dynamic scale out of resources) and outage avoidance.
• Evaluate responses to the unexpected. 13‹#›
You’d like to make decisions are made based on expected performance, but….
expected unexpected
unex
pect
ed
Resource Requirements (e.g., degree of skew)
Reso
urce
Ava
ilabi
lity
expe
cted
Performance predictable
Performance harder to predict
Performance harder to predict
Runtime performance really, really hard to predict
• BI queries -> skew + complex queries
• Skew + complex queries -> unexpected amount of work.
• More work -> more resource usage.
• Multiple queries with unexpected resource usage -> unexpected resource availability.
14‹#›