Checking Performance along Your Build Pipeline

T11 Continuous Delivery

10/16/2014 11:15:00 AM

Checking Performance along Your Build Pipeline

Presented by:

Andreas Grabner

Compuware

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com

Andreas Grabner

Compuware Andreas Grabner has more than fifteen years of experience as an architect and developer in Java, .NET, and Web 2.0 with a strong focus on application performance. At Compuware APM (formerly dynaTrace) Andi is a technology strategist, helping companies improve their applications’ performance across the development lifecycle by embracing ideas of continuous delivery and DevOps. He is a frequent speaker on software performance, testing, and architecture topics at technology conferences including Velocity, STAREAST, STARWEST, and JavaOne. Andi regularly publishes articles and blogs on apmblog.compuware.com. Before dynaTrace, Andi was an engineer and product manager for Segue Software and Borland on their Silk Product Line.

1"

Checking)Performance)Along)your)Build)Pipeline)

)@grabnerandi)

)

h9p://apmblog.compuware.com)

2"

h%p://starwest.techwell.com/sessions/starwest62014/checking6performance6along6your6build6pipeline"

Do"you"consider"the"performance"impact"when"adding"a"new"JavaScript"file,"a"single"AJAX"call,"or"a"new"database"query"to"your"app?"Negligible,"you"say?"I"disagree―and"so"should"you."Andreas"Grabner"demonstrates"the"severe"impact"small"changes"can"have"on"performance"and"scalability."Many"small"changes"will"have"an"even"bigger"impact"so"it"is"important"to"catch"them"early."If"you"are"working"with"a"delivery"pipeline,"make"sure"to"look"into"performance,"scalability,"and"architectural"metrics"such"as"the"number"of"resources"on"your"page,"size"of"resources,"number"of"requests"hiPng"your"web"servers,"database"statements"executed,"and"log"messages"created."Monitoring"these"allows"you"to"add"a"new"quality"gate"to"your"delivery"pipeline"and"prevents"major"problems."Andi"shares"a"handful"of"metrics"to"teach"to"your"developers,"testers,"and"operaSons"folks,"and"explains"why"they"are"important"to"performance."

3"

4"

5"

6"

7"

And"just"very"recently"@"Euro"Song"Contest"

8"

9"

What"you"should"check"out"…"

10"

Commit"Stage"•  Compile(

•  Execute(Unit(Test(•  Code(Analysis(•  Build(installers(

Automated""Acceptance""TesSng"

Automated""Capacity""TesSng"

Manual"tesSng"•  Key(showcases(

•  Exploratory(tes;ng(

""

Release"

11"

12"

Who"is"doing"it?"How"many"successful"deployments"can"they"do?"

300"Deployments"/"Year"

50660"Deployments"/"Day"

10+"Deployments"/"Day"

Every"11.6$seconds$

13"

More"on"Amazons"Story"

75%$fewer"outages"since"2006"

90%$fewer"outage"minutes"

~0.001%$of"deployments"cause"a"problem"

Instantaneous$automaSc"rollback"

Deploying"every"11.6s"

14"

15"

16"

17"

TesSng"is"Important"–"and"gives"Confidence"

18"

But"are"we"ready"for"“The"Real”"world?"

19"

Measure"Performance"during"the"game"

Ball"Possession:" "40":"60)Fouls:""""""""""""""""""""""""0":"0"Score:""""""""""""""""""" """0":"0"

Minute"1"6"5"

20"

Measure"Performance"during"the"game"

Minute"6"6"35"

Ball"Possession:" "80":"20)Fouls:""""""""""""""""""""""""2":"12)Score:""""""""""""""""""" """0":"0"

21"

Deep"Dive"Analysis"

22"

OpSons"“To"Fix”"the"situaSon"

23"

Not"always"a"happy"ending"!"

Minute"90"

Ball"Possession:" "80":"20)Fouls:""""""""""""""""""""""""4":"25)Score:""""""""""""""""""" """3":"0"

24"

FRUSTRATED"FANS!!"

24"

25"

How)does"that"relate)to)SoGware?$

26"

From"Deploy"to"…"

Deploy Promotion/Event Problems Ops Playbook War Room

Timeline"

27"

The"“War"Room”"–"back"then""'Houston,"we"have"a"problem‘"NASA"Mission"Control"Center,"Apollo"13,"1970"

28"

The"“War"Room”"–"NOW""Facebook"–"December"2012"

29"

4)SituaJons)on)WHY)this)happens,)HOW)to)avoid)it$

30"

31"

“Blindly”)(Re)use)ExisJng)

Components)

32"

Requirement:"We"need"a"report"

33"

Using"Hibernate"results"in"4k+"SQL"Statements"to"display"3"items!"

Hibernate"Executes"4k+"Statements"

Individual"ExecuSon"VERY"

FAST"

But"Total"SUM"takes"6s$

34"

Requirement:"We"need"a"fancy"UI"

35"

Using"Telerik"Controls"Results"in"9s"for"Data6Binding"of"UI"Controls"

#1:$Slow$Stored$Procedure$Depending"on"Request"execuSon"Sme"of"this"SP"varies"between"1"and"7.5s"

#2:$240!$Similar$SQL$Statements$Most"of"these"240!"Statements"

are"not"prepared"and"just"differ"in"things"like"Column"Names"

36"

Metrics:"#"Total"SQLs"#"SQLs"/"Web"Request"#"Same"SQLs"/"Request"

Transferred"Rows"

Test:"With"realisSc"Data"

Dev:"“Learn”"Frameworks"

37"

38"

ImplementaJon)Flaws)

39"

Business"Impact"requires"AcSon!"

40"

SoluSon:"Cache"to"the"RESCUE!!"

41"

ImplementaSon"and"Rollout"

Implemented"InMemory"Cache"

Worked"well"in"Load"TesSng"

42"

Result:"Out"of"Memory"Crashes!!"

SSll"crashes"

Problem)fixed!)Fixed)Version)Deployed)

43"

Metrics:"Heap"Size,""#"Objects"Allocated,"#"Objects"in"Cache"Cache"Hit"RaSo"

Test:"With"realisSc"Data"

44"

45"

#No)“Agile”)Deployment)

46"

Ad)on)air)

Load"Spike"resulted"in"Unavailability"

47"

AlternaSve:"“GoDaddy"goes"DevOps”"

1h"before""SuperBowl"KickOff"

1h"auer""Game"ended"

48"

Behind"the"Scenes"

49"

Metrics:"Availability"Page"Size,"#"Objects"

#"Hosts,"#"ConnecSons"

DevOps:"“Feature”"Switches"

50"

51"

#Push))without)a)Plan)

52"

Mobile"Landing"Page"of"Super"Bowl"Ad"

434$Resources$in"total"on"that"page:"230"JPEGs,"75"PNGs,"50"GIFs,"…"

Total"size"of"~"20MB$

53"

m.store.com"redirects"to"www.store.com"

ALL"CSS"and"JS"files"are"redirected"to"the"www"domain"

This"is"a"lot"of"Sme"“wasted”"especially"on"high"latency"mobile"

connecSons"

54"

Metrics:"Load"Time,""#"Resources"(Images,"…),""#"HTTP"3xx,"4xx,"5xx"

Dev:"Build"for"Mobile"

Test:"Test"on"Mobile"

Ops:"Monitor"Mobile"

55"

56"

#"of"Requests"/"User"

#"of"Log"Messages"

#"of"ExcepSons"

#"Objects"Allocated"

#"Objects"In"Cache"

Cache"Hit"RaSo"

#"of"Images"

#"of"SQLs"

#"SQLs"per"Request"Availability"

#"HTTP"3xx,"4xx"

Page"Size"

57"

Commit"Stage"•  Compile(

•  Execute(Unit(Test(•  Code(Analysis(•  Build(installers(

Automated""Acceptance""TesSng"

Automated""Capacity""TesSng"

Manual"tesSng"•  Key(showcases(•  Exploratory(tes;ng(

""

Release"

Unit)&)IntegraJon)Tests)

FuncJonal)Tests)

Performance)Tests) ProducJon)Monitoring)

FuncJonal)Tests)

58"58"

59"

60"

If"we"do"all"that"

61"

Which"gives"you"more"Sme"for"the"real"important"things"in"life"…"

62"

Want"MORE"of"these"and"more"details?"

h%p://apmblog.compuware.com""

63"

FREE"Products"&"More"Info"

• dynaTrace$Enterprise$–  Full"End6to6End"Visibility"in"your"Java,".NET,"PHP"Apps"–  Sign"up"for"a"15"Days"Free"Trial"on"h%p://goo.gl/1ZFQZ0""

• dynaTrace$AJAX$EdiQon$–  Browser"DiagnosScs"for"IE"+"FF"–  Download"@"h%p://ajax.dynatrace.com""

• Our$Blog:"h%p://apmblog.compuware.com"