Top Banner
An Approach to Automated Application Tuning Mike Huang, Sr. Performance Architect Performance and Reliability Engineering
29

An Approach to Automated Application Tuning

Jan 15, 2017

Download

Software

Mike Huang
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: An Approach to Automated Application Tuning

An Approach toAutomated Application Tuning

Mike Huang, Sr. Performance ArchitectPerformance and Reliability Engineering

Page 2: An Approach to Automated Application Tuning

UNNECESSARY EXPOSITION

Prologue.

Page 3: An Approach to Automated Application Tuning

Copy What Someone

Else Is Doing

Tweak Until Things “Work”

Forget About It

Something Changes

Something Breaks

Tuning. Ugh.

Page 4: An Approach to Automated Application Tuning

Why?●Insufficient defaults●One-size-fits-all myth●Unfamiliar technology●Complex interactions●Time

Page 5: An Approach to Automated Application Tuning

Nirvana

●Optimal●Adaptive●Automatic

Page 6: An Approach to Automated Application Tuning

Construct a Model1. Frame the problem2. Understand interactions3. Define success4. Narrow focus5. Validate data and behavior6. Refine

Page 7: An Approach to Automated Application Tuning

ORIGIN STORY

Sceneone.

Page 8: An Approach to Automated Application Tuning
Page 9: An Approach to Automated Application Tuning

Spilled Milk

Page 10: An Approach to Automated Application Tuning

GoalImprove resiliency to downstream failures

Page 11: An Approach to Automated Application Tuning

UNLOCKING MYSTERIES

Scenetwo.

Page 12: An Approach to Automated Application Tuning

Dependency Framework

TimeoutsFallbacksRejectionsShort-circuits

TimeoutsRetriesPoolingBlacklists

Page 13: An Approach to Automated Application Tuning

Understanding Interactions

Hystrix

Service B

EVCache

EVC Client

Ribbon

Ribbon

/all/the/moviesService A

Hystrix

EVC Client

/moar/movies

Page 14: An Approach to Automated Application Tuning

Hystrix

Hystrix

Hystrix

Understanding Interactions

Hystrix

Service B

EVCache

EVC Client

Ribbon

Ribbon

/all/the/moviesService A

Hystrix

EVC Client

/moar/movies

Page 15: An Approach to Automated Application Tuning

Hystrix Workflow

Page 16: An Approach to Automated Application Tuning

Define SuccessUnder normal operations:●Expire only the most egregious requests●No concurrency constraint●No premature fallback

Under duress:●Fail fast!

Page 17: An Approach to Automated Application Tuning

Assumptions●Spectrum of desirable behavior●You can’t win them all● It’s okay to fail

Page 18: An Approach to Automated Application Tuning

Key Tunables

●Timeout●Error Threshold●Isolation Strategy

●Thread Pool Size●Queue Size●Batch Size

Page 19: An Approach to Automated Application Tuning

THE TWIST

Scenethree.

Page 20: An Approach to Automated Application Tuning

True Lies●Recommendations are

only as good as the data

●Data is only good if you understand it

Page 21: An Approach to Automated Application Tuning

Things Are Not Always As They Seem

Page 22: An Approach to Automated Application Tuning

Pick a Number, Any Number

Page 23: An Approach to Automated Application Tuning

How Low Can You Go?

Page 24: An Approach to Automated Application Tuning

THE FINAL ACT

Scenefour.

Page 25: An Approach to Automated Application Tuning

Automation? Piece of Cake

Page 26: An Approach to Automated Application Tuning

Evaluate

RecommendApply

The Cake Is a Lie!

Page 27: An Approach to Automated Application Tuning

[email protected]

@gnauHekiM

Page 28: An Approach to Automated Application Tuning

GO HOME – THERE ARE NO STINGERS HERE

Credits.

Page 29: An Approach to Automated Application Tuning

Acknowldgements (Please Don’t Sue Me)

Slide: 4 – Parks and Recreation (Season 2, Episode 4) 5 – http://www.musiclipse.com/wp-content/uploads/2013/09/nirvana-hd-photo-band-members.jpg 6 (C) – https://rtfitch.files.wordpress.com/2014/08/bojack-horseman.jpg 6 (BoJack) – http://www.serieslyawesome.tv/wp-content/uploads/2015/05/BoJack-Horseman-Season-2.jpg 9 (Drew Barrymore) – The Wedding Singer 9 (Angel of Grief) –

https://en.wikipedia.org/wiki/Angel_of_Grief#/media/File:Emelyn_Story_Tomba_(Cimitero_Acattolico_Roma).jpg 9 (Angry Cat) – http://theeconomiccollapseblog.com/wp-content/uploads/2011/01/Angry-Cat.jpg 12 (Ribbon logo) – http://3.bp.blogspot.com/-

8OQzOCw1rXY/VdE2mxvnybI/AAAAAAAABoE/ARq3Pa4v9yI/s1600/ribbon.png 12, 18 (Hystrix logo) – https://raw.githubusercontent.com/wiki/Netflix/Hystrix/images/hystrix-logo-tagline-640.png 15 – https://github.com/Netflix/Hystrix/wiki/How-it-Works/images/hystrix-flow-chart-original.png 20 – https://fanart.tv/fanart/movies/36955/moviebackground/true-lies-51036cf5a0251.jpg 25 – http://www.fmvmagazine.com/wp-content/uploads/2012/07/Portal-cake.jpg 26 – http://theportalwiki.com/wiki/File:The_bots.png 28 – Mike Huang