Agile Testing FAQs and Mythbuster - Software Testing Atlanta Conference 2015

@yuvalyeret#STAC15

AgileTes5ngFAQsandMythbuster

YuvalYeret-AgileSparks#STAC15

1

@yuvalyeret#STAC15

Agile Team2

Agile Team1

Ongoing

Done Features Backlog

Develop

Feature/ Sprint

in Progress

Story-level Test & Fix

Deployment

Done Ongoing Done

Stories Backlog

Ongoing

Specify

Done

A

B

C

G

H

J K

L

M

H.6 H.0

H.1 H.2

H.3

H.4 H.5 D2

D3

T2 T1

D1 P1

H.7

T2

Implement Feature by Stories

How does an agile project/program looks like? (by the book…)

@yuvalyeret#STAC15

Butwhenthetypicaldev/testgroupmeetsAgile…

SeeKentBeck’sideaasdescribedbyMarkusGartnerathKp://www.shino.de/2010/11/04/soQware-g-forces-the-effects-of-accelera5on/

Agile Team2

Agile Team1

Ongoing

Done Features Backlog

Develop

Feature/ Sprint

in Progress


Deployment

Done Ongoing Done

Stories Backlog

End of Release Testing

Ongoing Done Ongoing

Specify

Done

D

E

F

A

B

C

G

H

J K

L

M

H.6 H.0

H.1 H.2

H.3

H.4 H.5 D2

D3

T2 T1

D1 P1

H.7

T2


UAT Regression

Performance Security

Functional Progression

Exploratory

ATDD

Auto

Long wait for the endgame

Real Life… Not so agile testing...

“Sprint is too short for everything we need to

achieve DONE”

“Let’s leave the serious testing for the release

testing phase”

The 2-level Test Strategy Pyramid – Story + Release-level

Platform Matrix

Real Network

@yuvalyeret#STAC15

The result - only a limited amount of feedback is early and effective

0

20

40

60

80

100

120

1 2 3 4 5 6 7 8 9101112131415161718192021222324

done

donedone

40% 60%

Iden5fycoveragethathappensoutsidethesprint.Whichisthebiggestqualityriskiffoundlate?

Agile Team2

Agile Team1

Ongoing

Done FeaturesBacklog

Develop

Feature/Sprint

in Progress


Deployment

Done Ongoing Done

Stories Backlog

Ready For

Feature Test

Test Feature-

level Ongoing Done



Specify

Done

D

E T F T

T

A

B

C

G

H

J K

L

M

H.6 H.0

H.1 H.2

H.3

H.4 H.5 D2

D3

T2 T1

D1 P1

H.7 T2


UAT

Regression

Performance Security

Functional Progression

Exploratory

Try - Slowly shifting left towards Continuous Quality and adding the Feature-level testing stage in the pipeline

Add Feature/Epic level testing

1.  Add Feature/Epic/ Iteration level

testing.

2. left-shift more and more testing through automation, enabling teams using

environments/tools/knowhow and more.

@yuvalyeret#STAC15

Roleoftes5ngistofinddefectstoassurequality

ElaborateRequirements

TechnicalDesign

Coding/UnitTes5ng

TestExecu5on+Fixing Done

TestDesign TestAutoma5on

MythorBestPra

c5ce?

@yuvalyeret#STAC158

@yuvalyeret#STAC15

• Mostexpensiveautoma5ontodevelop,run&maintain,sominimize!!!

• MovemajorityofE2Etes5ngcoveragetoService/APIlayer

• QTP/UFT/Selenium/PerfectoMobile/etc.

UI

• “TheWorkhorse”ofenterpriseagiletes5ng

• Createdbytesters&developersonagileteamssupportedbyframeworks/guidancebyAutoma5onCoE

• soapUI,etc.

Acceptance(Service/API)

• LeverageAgileTeamsdevelopertes5ngtoreducecoverageneeds

• Abilitytoautoma5callydetect(throughcoveragetoolsetc.)whatiscovered

UnitTes5ng

Manual

hKp://www.mountaingoatsoQware.com/blog/the-forgoKen-layer-of-the-test-automa5on-pyramid

Ifweacceptthetestautoma5onpyramidwhatdoesitmeanforTesterSkills?

MythorBestPrac5ce?

10

Credit-UdiWeinberg–HPSoQware

@yuvalyeret#STAC15

BestPrac5ce–AgileTestersDRIVEquality

@yuvalyeret#STAC15

ATDD=AcceptanceTestDrivenDevelopment-BuildQualityIntoDesign–preven5ngdefects

ratherthanjustfindingthem

ElaborateRequirements TestDesign Technical

DesignCoding/Unit

Tes5ngTestExecu5on

+Fixing Done

1.   ATDDThinkingUsetestscenariostoguidedesign(whichcanthenhappenusingTDD,ornot…)

hKp://www.agilesparks.com/test-first-reading-list

TestAutoma5on

ATDD/BDD-T

heEnterprise/

Real-worldwr

apperforTDD

…

@yuvalyeret#STAC15

Automation is a USEFUL skill for Testers/ Quality

Engineers. NOT their core skill!

It IS a core team

capability for enabling Continuous Quality

@yuvalyeret#STAC15

TheFuture?*

*AgileSparkspredic5on

2005 2010 2015 2020

SWDevelopers

Testers

SWEngineers

Automa5onExperts

TestEngineers/Experts

@yuvalyeret#STAC15

UserStory1578•  AcceptanceCriteria:

–  Scenario1–  Scenario2–  Scenario3–  Scenario4–  Scenario5–  Scenario6–  Scenario7–  Scenario8–  Scenario9–  Scenario10–  Scenario11

15

Whatdoyouthink?Isth

isagoodstory?

Whatdoyoulikeabout

it?

Howwouldyouimprove?

Agile Team2

Agile Team1

Ongoing

Done FeaturesBacklog

Develop

Feature/Sprint

in Progress


Deployment

Done Ongoing Done

Stories Backlog

Ready For

Feature Test

Test Feature-

level Ongoing Done



Specify

Done

D

E T F T

T

A

B

C

G

H

J K

L

M

H.6 H.0

H.1 H.2

H.3 H.4

H.5

D2

D3

T2

T1 H.7

T2


Specify/Groom using Acceptance Tests

Backlog Grooming / Acceptance-Tests Specification Workshop •  Identify acceptance criteria/tests for the next stories in the

backlog. •  Use acceptance tests as guidance for slicing stories smaller to

enable more effective agile collaboration

@yuvalyeret#STAC15

Acceptance Tests/Mini-

Stories Stories MMFs Features

Feature

MMF H

Story H1 Test for H1

Mini-story H4

Mini-story H5

Story H2

Story H3

MMF I

MMF J

MMF K

Testers are key to effective

grooming together with the PO

and the Devs!!!

@yuvalyeret#STAC15

Ques5ons?•  hKp://www.agilesparks.com/agile-tes5ng-reading-list•  hKp://www.agilesparks.com/Services/AgileTes5ng•  Bloggingatyuvalyeret.com•  Emailmeques5ons/feedback/sugges5onsat

[email protected]•  UnlockmyAgileTes5ngWorkshopinyourcity!

•  TheseslideswillbeupathKp://www.slideshare.net/yyeretsoon.Myothertalksarealreadythere…

18

@yuvalyeret#STAC15

DevopsTools

Tradi5onalProcesses

IdealBatchw/oCon5nuousQuality

Investment

ReduceBatchSizeFurtherWITH

Greatautoma5onandleanerprocesses

Con5nuousQualityisajourneywithlotsofvalueateachstep