BEYOND AVERAGESDan Kuebrich / @dankosaur
Beyond Averages
• Abstraction: summary statistics for performance data
• Why performance data is hard
• Visualizing data distributions
A few of my favorite abstractions
A few of my favorite abstractions• Abstraction lets us trade information for
actionability
A few of my favorite abstractions• Abstraction lets us trade information for
actionability
• Min, max, average (“mean”), quantiles, stdev
A few of my favorite abstractions• Abstraction lets us trade information for
actionability
• Min, max, average (“mean”), quantiles, stdev
• That’s a great trade! • ... right?
Averages: average at best
Averages: average at best
Averages: average at best
Averages: average at best
Averages vs Percentiles
[1, 16, 17, 19, 13, 5, 20, 3, 10, 14, 8]
Averages vs Percentiles
[1, 3, 5, 8, 10, 13, 14, 16, 17, 19, 20]
Averages vs Percentiles
[1, 3, 5, 8, 10, 13, 14, 16, 17, 19, 20]
Average (Mean): 11.54
Median (50th Percentile): 13 90th Percentile: 19
Percentiles: 1 of 100 slices95%
X
Percentiles: 2 of 100 slices95%
10%
X
Y
Percentiles: 2 of 100 slices95%
10%
Percentiles: 2 of 100 slices95%
10%
Percentiles: 2 of 100 slices95%
10%
Computers are hard• Rarely do we have a single distribution underlying the
data
• Different users, different requests, different resources, different instances, different times
Percentiles vs Distributions
http://en.wikipedia.org/wiki/Percentile
Percentiles vs Distributions
[13, 13, 13, 13, 13, 13, 19, 19, 19, 19, 19]
Median (50th Percentile): 13 90th Percentile: 19
Rarely do we have a single normal distribution underlying the data
Median Mean 90th
Median Mean 90th
The Log-Normal Distribution
(source: http://www.geo.mtu.edu/volcanoes/vc_web/background/S_chem.html)
Log-Normal Distribution
(source: http://en.wikipedia.org/wiki/File:Comparison_mean_median_mode.svg)
Log-Normal Distribution
(source: http://en.wikipedia.org/wiki/File:Comparison_mean_median_mode.svg)
Log-Normal Distribution
Is there a place between Averageland and “A Beautiful Mind”?
http://now-here-this.timeout.com/2012/10/07/crazy-walls-of-clues-from-tv-film-reviewed-by-carrie-from-homeland/
Histograms
Freq
uenc
y (e
g. #
of c
alls
)
Value (eg. latency)
Populations revisited95%
10%
HistogramsFr
eque
ncy
(eg.
# o
f cal
ls)
Value (eg. latency)
Populations re-revisited95%
10%?
3d Histograms?
Freq
uenc
y (e
g. #
of c
alls
)
Value (eg. latency)
3d Histograms?Fr
eque
ncy
(eg.
# o
f cal
ls)
Value (eg. latency)
Time
Heatmaps
Freq
uenc
y (e
g. #
of c
alls
)
Value (eg. latency)
HeatmapsFr
eque
ncy
(eg.
# o
f cal
ls)
Value (eg. latency)
HeatmapsFr
eque
ncy
(eg.
# o
f cal
ls)
Value (eg. latency)
HeatmapsVa
lue
(eg.
late
ncy)
Time
HeatmapsVa
lue
(eg.
late
ncy)
Time
Latency in the wild…
http://sciencefiction.com/2013/10/24/throwback-thursday-jurassic-park/
Average, or Absolute?
Average, or Absolute?
Multi-modal Data
Multi-modal Data
Multi-modal Drill-down
Multi-modal Drill-down
Multi-modal Drill-down
Long Tails and Outliers
bottom 98%
Long Tails and Outliers
bottom 98%
Long Tails and Outliers
all of it
Long Tails and Outliers
Added Population
Added Population
Added Population
Thanks!
Sign up for a free TraceView account at appneta.com/products/traceview-free-account