Testing in Agile with Coaching Agile Journeys and XBOSoft

Tes$nginAgile(agile)

Philip Lew

©XBOSo4,Inc.2017AllRightsReserved

1 @xboso4@philiplew

AgileTes$ng–WhattoDo?

•  Today,we’llcoverwhatyoudoin‘AgileTes$ng’,whichisamaQeroffocusand$ming

•  Asopposedtowaterfallwhereso4warewas‘readyfortes$ng’,weare‘readyfortes+ng’(doingtes$ngac$vi$es)allthe$me,butwhatexactlyarewedoingwhen?

©XBOSo4,Inc.2017AllRightsReserved 2@xboso4@philiplew

A4erWorkingWithManyofOurClientsinBothWaterfallandAgile

•  I’vefoundthatfocusand$mingarethekeydifferences– Eitherintheirsuccess– Orintheirfailure

•  Notrocketscience•  Mostlynochangesintechnology,butchangesinfocusand$ming

©XBOSo4,Inc.2017AllRightsReserved 3

“Having the Right People In the Right Place Doing the Right Thing At the Right Time.”

@xboso4@philiplew

Meet Your Instructor

4©XBOSo4,Inc.2017AllRightsReserved

Phil Lew •  Telecommunications consultant and network designer •  Team Lead, Data warehousing product development •  Software product manager, BI product •  COO, large IT services company •  CEO, XBOSoft, software QA and testing services

Relevant specialties/Research •  Software quality process improvement •  Software process measurement •  Software quality in use / UX design

Where to find me •  @philiplew •  @xbosoft •  www.xbosoft.com •  [email protected]

@xboso4@philiplew

TodayisAllAboutYou


•  Everwonderhowdotes$ngprac$cesdifferwhetheryouronanagileversusawaterfallteam?

•  Doyous$llhavestep-by-steptestscriptsorjusthavegeneraltestscenariosoruserstories.

•  Doyouhaveatestplan?•  Whatdoyoudoasand‘agile’tester?•  Whyareyouhere?

@xboso4@philiplew

Se_ngExpecta$ons…

•  Interac$ve-askques$onsviaGoto-WebinarPanel

•  Iwon’treadtheslides…•  Slidesforyouasatake-away

– Sessionisrecordedandavailablea4erwards

6©XBOSo4,Inc.2017AllRightsReserved@xboso4@philiplew

So4wareisNew•  So4waredevelopmentisnew–notlongenoughtoknowhowtodoit.

•  Wearenotbuildingdoorsormakingtoilets.•  Whenwedonotknowwhatwearedoing,thebeststrategyistodo‘liQlebyliQle’approachandgetfeedbackassoonaspossible.

•  Weusethesametechniqueinoureverydaylives,andAgileisanapplica$onofitinso4ware.


WhatGotUsHere

•  Smallerteams•  Fasteritera$ons•  Listeningtotheuser

– Con$nuousbeta– Datacollec$on&analy$cs


•  Communica$on•  Workingsmarter•  Analysis,adap$onandimprovement

1.  Changes in technology (mobile, cloud) 2.  Changes in business models 3.  Many failures…

@xboso4@philiplew

AgileProblems

ResistancetoChange

DIstrust

RequirementsChurn

FrozenRequirements

AgileDoing–NotBeing

ProcessInconsistency LackTest

Automa$on

Retrospec$vesNotValuable

AgileFall

LackCustomer-User

Understanding

AgileFailures–Why?


AgileSuccess

Execu$veSupport

UserInvolvement

ScopedValue

SkilledPlayers

AgileProcess

Proficiency

ClearBusinessObjec$ves HighUseof

Automa$on

ConsistentDev,Integra$onandReleasePrac$ces

Collabora$veBehaviors

FullFocus

AgileSuccess–Why?



Be-Do-HaveParadigm

Be

Do

Have@xboso4@philiplew


AgileBe-Do-Have

Be

Do

Have

©XBOSo4,Inc.2017AllRightsReserved

Process •  Iterative (sprints) •  Daily standups •  Face to face communication •  Post mortem – end of sprint •  Delivery meeting – end of sprint •  Planning meeting – before sprint •  Self organizing

People •  Communicative •  Collaborative/Cooperative •  Flexible and willing •  Knowledgeable-multi •  Initiative/responsible •  Responsive

Results •  Speed •  Quality

@xboso4@philiplew 13

AgileQuality

•  Nopolice•  Sharedteamresponsibility•  Tes$ngisdonebyeveryone?

– Tes$ngisdonebyqualifiedteammembers– Developerscouldtakeonsometes$ngtasks

•  Developmentisdonebyeveryone?– Developmentisdonebyqualifiedteammembers– Testerscouldtakeonsomedevelopmenttasks


Team members do what they can do and what is needed


OneoftheMainAgileFlavors:Scrum

1 2

3

@xboso4@philiplew

AGILEPLANNINGThingstothinkaboutanddoduringthiscri$calphase…


1

@xboso4@philiplew

TesterPairingandSwapping•  WithTestSpecialists

– Security– Performance– Usability– Automa$on

•  WithDomainExperts– Businessperspec$ve– Auto-correct/complete/verifystories– Benefitsinbothdirec$ons


1

@xboso4@philiplew

Tes$ngisCri$calPartofthePlan

•  Tes$ngisnotana4erthought– Clarifystorieswithcustomerandenduser

•  Thinkfromtheirpointofview–  Iden$fytes$ngac$vi$estosupportdevelopment

• Whiteboxtes$ng•  Integra$ontes$ng

– Backloggrooming•  Improveandclarifystories


1

@xboso4@philiplew

WorkloadManagement

•  “Sustainable”-thisAgileprinciplehelpstopreventtestteamburnout

•  Sometes$ngtaskscanbeoutsidethesprints(suchastestautoma$oncodedevelopment),–  sometesterscanworkoutsidethesprintsontheselong-termprojects

•  Properes$ma$onofworkavoidspressureontestersduringtheendoftheitera$on

•  Properes$ma$onalsoincludeses$ma$onoftes$ng

1


SizetheJobProperlyandPriori$ze•  SomeAgileprojectsritualizesprint$medeadlines,whilenotfollowingotherrulesrelatedtosustainableworkload

•  Whentheteamcon$nuallyover-commits,thetestteamgetssqueezedonthelastweekendofeverysprint

•  Resolvethischallengethroughrisk-basedtes$ng•  Reducethescopeoftes$ngbasedonrisk

– Movelow-riskteststonextsprintor– Don’tdoatall

1

©XBOSo4,Inc.2017AllRightsReserved@xboso4@philiplew 20

PlanningforTechnicalDebt

•  Agileteamsshouldhavea“fixbugsfirst” approachtopreventalargebacklogofbugs– Reduce‘$metofix’beyondthesprintlength

•  Periodicstabiliza$onor‘fixdefects’sprintsshouldbeplanned

•  Withpropermetrics,tes$ngcanhelpmeasureandmanagetechnicaldebt

•  Plan$mefordefectverifica$on

1


UnderstandingTestBasis•  Testbasisisconstantlychangingeachitera$onandsome$meswithin– Stories– Acceptancetests– Environment– Productusage

•  Youmayhaveadifferentfocusindifferentitera$ons


1

@xboso4@philiplew

SprintBacklog–SprintPlanning•  Eachbacklogentrybrokendownintotasks

•  Tasksincludedevelopment,tes$ng,environmentconfigura$ons,dataprepara$on,anddocumenta$on

•  Allplannedtogether


1

@xboso4@philiplew

UserStories

•  Ideallyreviewedbybusinessandtechnicalstakeholdersearlyinthesprint

•  Testersshouldbeinvolvedinthesereviews•  Highqualityuserstorieswithclearexpectedresultsleadtoearlydefectremovalanddefectpreven$on

•  Testersshouldinspectforcompletenessandclarity– DoIunderstandwhatI’dbetes$ngfor?

1


StoryGrooming•  Clarifydescrip$onvialanguageandexamples

•  Predictandavoidalternateinterpreta$ons•  Elicitacceptancetestsfromproductownerorcustomer/enduser

•  Predictalternateorerroneousflows•  Relatenewstoriestoalreadyunderstoodconceptsandfunc$ons


1

@xboso4@philiplew

Infrastructure

•  Developingandmaintainingtakes$me,some$mesoverlookedandnotsmall– Tools– Frameworks– Data– Environments


1

@xboso4@philiplew

TestDataandEnvironments

•  Automa$ngtestdatachoresiscri$cal– Generateandcapture– Maintainandorganize– Validate– Changeproceduresandaccess– Regulatorycompliance


1

@xboso4@philiplew

TestWhatonWhatEnvironment•  Developmentenvironment

–  Unit/component–  Sta$ccodeanalysis

•  Integra$onenvironment–  Incrementalintegra$on–  Regression


1

@xboso4@philiplew

•  QAenvironment–  Smoke–  Func$onal–  Usability–  Regression–  UA

•  Stagingenvironment–  Performance,stressandload–  EndtoEnd–  System–  Security

Func$onalTes$ng

•  Canshi4focusdependingontheitera$onandfocusoftheitera$on– Stories

• Crossfunc$onal• Storyinterac$ons• Experiencebasedandexploratory


1

@xboso4@philiplew

NonFunc$onalTes$ng•  O4enforgoQenun$ltheend

– Usability– Security– Performance– Sta$c-Whitebox– Inspec$ons


1

@xboso4@philiplew

ReviewTests

•  Inaddi$ontorequirements,businessandtechnicalstakeholderscanreviewtestcondi$ons(acceptancecriteria)– Reducesthenumberoffalseposi$vesandfalsenega$vesassociatedwithtes$ng

–  Improvestestresults–  Increasesconfidenceinquality– Gaininsight/reviewwhat’simportantandwhat’snot

1


PlanningforChange

•  Agiletes$ngrequireschangeinmindsetandchangeinfocusintechnicalskills

•  Testersarebestequippedwhentheyknowhowtoprogramoratleastunderstandthelanguagebeingused

•  Duetoshortitera$onsandcompressedregression,Agiletes$ngrequirestestautoma$on

•  Testersarebestequippediftheycaneffec$velyimplementtestautoma$on

1


DURINGTHESPRINT


2

@xboso4@philiplew

Gowhereyou’rewantedANDneeded…

DuringtheSprint–WhatDoYouDo?

•  Pairing•  TestDrivenDevelopment•  AutomatedTes$ng•  ExploratoryTes$ng•  Con$nuousIntegra$on


2

@xboso4@philiplew

TesterPairing–HeatoftheSprint•  Withtestdomainexperts

– Security– Performance– Usability– Automa$on

•  Withothertesters– “ExtremeTes$ng”--XT(XP)

•  Withprogrammers– Whitebox– Programminglogicversustes$nglogic


2

@xboso4@philiplew

TesterPairing–HeatoftheSprint•  Withdevelopers

– Whitebox– Programminglogicversustes$nglogic– Helpdeveloperslearnhowtowritegoodunittestsforcode


2

@xboso4@philiplew

TestDrivenDevelopment•  Definethetest

– Thisiswheretesterscanhelpindesignandcri$queofdatadriventestsandunittests

– AcceptanceTestDrivenDevelopment– BehaviorDrivenDevelopment

•  Design/Implementthetest•  Developcodetopassthetest


2

@xboso4@philiplew

AutomatedUnitTes$ng

•  Automatedunittestslimitregressionrisk•  Canberunaso4enaseveryfewhours,oronce/day

•  Combineautomatedunittestswithsta$ccodeanalysisandcodecoveragetomaximizeeffec$venessandcompleteness– Don’tusuallyfinddefects

2


AutomatedFunc$onalTes$ng

•  Automatedfunc$onaltes$ngcanbeintegratedwithcon$nuousintegra$on,automatedunittes$ng,codecoverage,andsta$canalysisframeworks

•  Furtherminimizesregressionriskandincreasestes$ng‘velocity’.

•  Selenium,amongstmanyotheropensourcetoolscandothis.

2


Sta$cAnalysis

•  Sta$ccodeanalysiscaniden$fymanyproblemspriortotes$ng– Complexity– Deadcode– Reusability

•  Somesta$canalysiscaniden$fysecurityandmaintainabilityproblemsthattes$ngcannotiden$fyeasily

2


ExploratoryTes$ng•  Notfreeformdowhateveryouwant•  Structuredexperiment•  Time-boxed•  Purposefulandfocused

– Charter


Design

Execu$onLearning

2

@xboso4@philiplew

ExploratoryTes$ngCharters•  Mission,whatyousetouttodo•  Whatyouwanttoaccomplish,learn/find•  Ensuresharedunderstandingofscope•  Examples:

–  Failures,errors,andrecovery–  Endtoendsystem–  Onefunc$on/workflow–  Emergingbehaviors(analy$cs)–  Non-func$onal

•  Security–aQacks•  Performance


2

@xboso4@philiplew

ExploratoryTes$ngTime-Boxes•  Varieddependingonscopeandmission

– Typical–2hourstohalfday– Short–1hourorless– Long–1dayormore


2

@xboso4@philiplew

ExploratoryTes$ng–MaximizingOutput

•  Duringthesession,takenoteson– Decisionsyoumade– Whatyoudid– Whatyoudiscovered

•  A4erthesession– Stopatthedesignated$me-box– Reviewfindings

• Whatworkedandwhatdidn’t• Whatdidwelearnandshouldwedoanothersame/diff


2

@xboso4@philiplew

Con$nuousIntegra$on

•  Detectintegra$onproblemsimmediatelya4ercodecheck-in

•  Brokenbuildscanbereducedoreliminated•  Combinecon$nuousintegra$onwithautomatedunittes$ngandautomatedfunc$onaltes$ngasthefounda$onofyour‘smoke’regression.

2


THERETROSPECTIVE


3

@xboso4@philiplew

Tes$ngImprovements

• Wherewas$mewasted?• Whatwasnottestable?• Whattasksfellofftheplateandwhy?• Whattasksweretotallyoverlooked?•  Isautoma$ondoingwhatwewant?

3


AnalyzeDefects

• Whatnewdefectswereintroduced– Whatarethesourcesofdefects– Wherewerethedefectsintroduced

•  Commoncauses•  Howtoavoid?

3


Collabora$on

• Withclient/user-Didwedeliverwhatwasexpected?

• Withdevelopers• Withbusinessanalysts• Withotherteams• Whatinforma$onwasmissed/notcommunicatedandwhy?

3


AgileTrust

•  EveryonetalksabouthowAgilerequires‘trust’.

•  Trustisespeciallyneededinretrospec$veswhenyoutalkaboutyourdirtylaundry.

•  Butwhatistrust?•  Whatdoestrustmeantoyou?•  Howdoyoucreateanenvironmentfortrust?

@xboso4@philiplew©XBOSo4,Inc.2017AllRightsReserved 50

3

@xboso4@philiplew©XBOSo4,Inc.2017AllRightsReserved 51

1 2

3

TheEnd4

Successfully‘GoingAgile’•  Agileisnotatechniqueortechnology•  Youcanbeagilewithoutabook,withoutacoach,justlikeyoucanloseweightifyouwantto,withoutWeightWatchers

•  Withadiet,youwantalifestylechange– Notjusta3weekeffort,butforlongtermhealth– Some$mesyouhavesetbacks

•  WithAgile,youwantaworkstylechange


Summary–Shi4YourThinking•  Agilepresentsasignificantshi4in$mingofwhenandwhatyoudoasatester

•  Testersshouldembracethesechanges(opportuni$es)– Learnnewskills– Takeonadifferentmindsetincollabora$on– “Everyoneisresponsibleforquality”


Summary-AsaTester1.  Agiledoesn’tmean‘notestplan’asyou’ve

seen,butit’sadifferenttestplan.2.  Beflexibleintheheatofthesprint,followthe

planand“Gowhereyouarewantedandneeded.”

3.  Agilegivesyoutheopportunityforates$ngretrospec$vethatyoudidnothavebefore.

–  Useittoimprove.


Thanks! For those knowledge seekers, download:

Agile Metrics Agile Test Plan Agile 7 Habits

https://xbosoft.com/resources/white-papers/

Philip Lew @philiplew [email protected]
