Testing in Agile Development

Hariprakash Agrawal [email protected]

8th BSPIN Conference on 18-19, September 2014

http://testoptimizer.com

Testing in Agile Development

Services Experience

About Me

Serving as CEO at OpCord –  Software Development, Test

Automation, Agile Consulting, Products (RideAlly, TestOptimizer)

Agile / Kanban Coach & Trainer (CSM) –  Agile Transformations

Email: [email protected]

2

Enterprise Agile Coaching & Transformation

Building Agile Culture and values

Training Scrum, Kanban, XP

Implementing SAFe Framework

Tools expert: Rally

An Entrepreneur and Agile Coach


The Problem

http://testoptimizer.com 3

Individuals and interactions over Processes and Tools

Working Product over Comprehensive Documentation

Customer Collaboration over Contract Negotiation

Responding to change over Following a plan

Agile Manifesto

That is, while there is value in the items on the right, we value the items on the left more.

Courtesy: www.agilemanifesto.org 4

Agile Principles

5

Our highest priority is to satisfy the customer through early and continuous delivery

Welcome changing requirements, Agile processes harness change for the customer's competitive advantage.

Deliver working product frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

Business people and developers must work together daily throughout the project.

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

Working product is the primary measure of progress.

Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

Continuous attention to technical excellence and good design enhances agility.

Simplicity--the art of maximizing the amount of work not done--is essential.

The best architectures, requirements, and designs emerge from self-organizing teams.

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

Courtesy: www.agilemanifesto.org

Courtesy: http://smartbear.com/products/qa-tools/what-is-agile-testing/ 6

Scrum Life Cycle!

7 http://testoptimizer.com

Agile Testing with Scrum

Courtesy: https://www.imbus.de/veranstaltungen/trends-in-testing-2012/ 8

Definition of Done

9

is usually a clear and concise list of requirements that a software increment must adhere to, for the team to call it complete

checklist of valuable activities required to produce software

primary reporting mechanism for team members

not static

an auditable checklist

DoD can be at various levels •  for a feature (story or product backlog item) •  for a sprint (collection of features developed within a sprint) •  for a release (potentially shippable state

Courtesy: http://www.scrumalliance.org/community/articles/2008/september/what-is-definition-of-done-%28dod%29

Sample Definition of Done

User Story Level

Code checked-in the integration branch and merged

Code review done and comments incorporated (Coding guidelines checked)

Automated Unit tests run

code coverage tools run Static analysis is at 100% pass rate

No open defects unless deferred

Product owner demo completed

Acceptance criteria met

10

Sprint or Release Level

Required user documentation updated

No open defects unless deferred

Test cases documented, automated and run over night

Functional testing is at 100% pass rate (Positive and Negative)

Usability guidelines incorporated User-facing story is reviewed by UE designers (where applicable)

Product owner demo completed

Acceptance criteria met


Extreme Programming!

User Stories

Release Planning

Sprint Planning

On Site Customer

Pair Programming

Test-first Design

Refactoring

Continuous Integration

Collective Ownership

Coding Standard

40 hour week

11 Courtesy: http://xprogramming.com/what-is-extreme-programming/

Agile testing is a software testing practice that follows the principles of agile software development#

Courtesy: http://en.wikipedia.org/wiki/Agile_testing

–  involves all members of a cross-functional agile team#–  with special expertise contributed by testers#–  to ensure delivering the business value desired at frequent intervals#–  working at a sustainable pace #–  specification by example is used to capture examples of desired and

undesired behavior and guide coding.#–  not a separate phase, but an integral part of software development, along

with coding#–  "whole-team" approach to "baking quality in" to the software product#–  testing and coding are done incrementally and iteratively#–  covers all types of testing#

12

Traditional Vs. Agile Testing

Courtesy: http://www.agilecoachjournal.com/index.php/2014-01-28/testing-2/the-agile-testing-pyramid/ 13

Principles of Agile Testing

Testing moves the project forward

Testing is NOT a Phase

Everybody Tests

Reduce feedback latency – Agile teams test early and often

Test represent expectations

Bugs don’t hang around – Keep code clean & Fix bugs fast

Reduce test documentation overhead

Tested is part of “Done”

From test last to test driven

Courtesy: http://testobsessed.com/wp-content/uploads/2011/04/AgileTestingOverview.pdf 14

Practices of Agile Testing

Automated Unit/Integration Tests


Test Driven Development

Automated System Level Regression Tests

Acceptance Test Driven Development

Behavior Driven Development

Exploratory Testing

Courtesy: http://testobsessed.com/wp-content/uploads/2011/04/AgileTestingOverview.pdf 15

7 deadly sins of Agile Testing

Courtesy: http://properosolutions.com/2011/03/the-seven-deadly-sins-of-agile-testing/

Water scrumming

Lack of TDD and CI

Testing is one Sprint behind

Coding

Unbalanced Testing

Quadrants Ignoring Test

Failures

Separate QA Team

Requirements and Tests are

Separate

16

Exercise

Predict the quadrants


Q2# Q3#

Q1# Q4##

Business Facing#Su

ppor

ting

the

Team

#Critique Product#

Technology Facing#

17

Agile Testing Quadrants

Courtesy: http://swansegarconsulting.com/learn-agile/agile-roles-titles/quality-assurance-control/ 18

Have we decided to automate tests?


Why Automate Tests?


Test Automation is the use of special software to control the execution of tests and the comparison of actual outcomes with predicted outcomes.

Courtesy: http://en.wikipedia.org/wiki/Test_automation

Code-driven testing The public (usually) interfaces to classes, modules or libraries are tested with a variety of input arguments to validate that the results that are returned are correct.

Graphical user interface testing Generates user interface events such as keystrokes and mouse clicks, and observes the changes that result in the user interface, to validate that the observable behavior of the program is correct.

API driven testing Uses programming interface of the application to validate, the behaviour under test. Typically API driven testing bypasses application user interface altogether.

21

Test Automation Life Cycle - 1

Courtesy: http://www.nordixsoft.com/AutomatedTesting/AutomationMethodlogy/ 22

Test Automation Life Cycle - 2

Courtesy: http://geek4eva.com/2009/08/28/test-automation-and-roi/ 23

Types of Automation Frameworks

Courtesy: http://www.softwaretestinghelp.com/qtp-tutorial-17-test-automation-frameworks/ 24

Linear Framework

Verify the creation of a new account in gmail!#Steps are:##a) Open gmail.comb) Click on ‘Create Account’ c) Enter the details d) Verify the detailse) Create the account##

#1 'Open GMail#2 SystemUtil.Run "iexplore.exe", "http://www.gmail.com”#3 'Page Sync#4 Browser("Gmail").Page("Gmail").Sync#5 ‘Click on create account#6 Browser("Gmail").Page("Gmail").WebLink(“Create Account”).Click#7 ‘Enter the details#8 Browser("Gmail").Page("Google Accounts").WebEdit(“First Name”).Set “Swati”#9 Browser("Gmail").Page("Google Accounts").WebEdit(“Last Name”).Set “test”#10 ‘Fill in several other details#11 ‘Submit#12  Browser("Gmail").Page("Google

Accounts").WebButton(“Next Step”).click#

25 Courtesy: http://www.softwaretestinghelp.com/qtp-tutorial-17-test-automation-frameworks/

Keywords Driven - 1

Function gotoGoogleAccount()#'Open Gmail#SystemUtil.Run "iexplore.exe", "http://www.gmail.com"#'Page Sync#Browser("Gmail").Page("Gmail").Sync#‘Click on create account#Browser("Gmail").Page("Gmail").WebLink(“Create Account”).Click#‘Enter the details#End Function#!

Function SubmitToCreate()#‘Submit#Browser("Gmail").Page("Google

Accounts").WebButton(“Next Step”).click#

End Function!!##Function EnterDetails()#Browser("Gmail").Page("Google

Accounts").WebEdit(“First Name”).Set “Swati”#

Browser("Gmail").Page("Google Accounts").WebEdit(“Last Name”).Set “test”#

‘Fill in several other details#End Function#


Keywords Driven - 2

'Open GMail gotoGoogleAccount() ‘Enter the details EnterDetails() ‘Submit SubmitToCreate() It helps in achieving readability, modularity and if in case another program wants to use the login function, we can surely reuse it.


Data Driven

Courtesy: http://www.softwaretestinghelp.com/qtp-tutorial-18-data-driven-and-hybrid-frameworks/

Browser("Gmail: Email from Google").Page("GoogleAccounts").WebEdit("FirstName").Set "swati"#Browser("Gmail: Email from Google").Page("Google Accounts").WebEdit("LastName").Set "s"#Browser("Gmail: Email from Google").Page("Google Accounts").WebEdit("GmailAddress").Set "test”########Browser("Gmail: Email from Google").Page("Google Accounts").WebEdit("FirstName").Set DataTable("G_First_Name", dtGlobalSheet)#Browser("Gmail: Email from Google").Page("Google Accounts").WebEdit("LastName").Set DataTable("G_Last_Name", dtGlobalSheet)#Browser("Gmail: Email from Google").Page("Google Accounts").WebEdit("GmailAddress").Set DataTable("gmail_address", dtGlobalSheet) ###

28

Exercise

Take any scenario and write an approach (or code) keeping Hybrid framework in mind


Distributed Test Automation

Courtesy: http://www.testandtry.com/2009/02/19/distributed-test-automation-infrastructure-plan-1/ 30


31

Maintain a code repository

Automate the build

Make the build self-testing

Every commit (to baseline) should be built

Keep the build fast

Test in a clone of the production environment

Make it easy to get the latest deliverables

Everyone can see the results of the latest build

Run regression test in automated way over night

Automate deployment


Test Driven Development (TDD)

Courtesy: http://en.wikipedia.org/wiki/Test-driven_development 32


What ATDD takes?

34

Executable requirements

Product Owner perspective

Continuous refactoring

Tests drive product design

Automated acceptance tests

Example: Selenium


Behaviour Driven Development (BDD)

Courtesy: http://blog.bughuntress.com/automated-testing/automated-testing-with-behavior-driven-testing 35

Behaviour Driven Development

Courtesy: http://behaviordrivendevelopment.wikispaces.com/Overview 36

BDD in Nutshell

Courtesy: http://www.hascode.com/2011/05/oh-jbehave-baby-behaviour-driven-development-using-jbehave/ 37

Factors to consider for test automation tool


Extensible & Customizable (Open APIs)

Runs with automated

build over night (batches)

Customizable Reporting

Easy Debugging & Logging

Email Notifications

Keywords & Data Driven Framework

Take Screenshots

for failed testcases

Multi Browser & OS

Support

Platform Independence

Supports Distributed Execution

Environment

38

Test Automation Tools – Open Source

Courtesy: http://www.opensourcetesting.org

Opensourcetesting.org lists many open source testing tools

Functional 129 Performance 53 Test management 29 Unit testing Java 72, C/C++ 39, .Net 20, PHP 10

39

Tools as per Quadrants

Courtesy: http://www.slideshare.net/AndrewDzynia/effective-testing-in-agile?qid=ec8c463b-4b6a-41ea-80c9-064b7b4d5bd1&v=qf1&b=&from_search=6

Story tests, Checklists, Mind Maps, Flow Diagrams#

#Automation Tools: BDD

(Fitnesse etc) and UI Tools (Selenium etc)#

Q2 Tools, Monitoring Tools#Data Generation tools#

Exploratory testing tools (Rapid Reported, Session Tester)#

Code Management: SVN, GIT#IDE: Eclipse, NetBeans#

Build/CI: Jenkins, Ant, Maven#UT: xUnit, Mocks#

Loadrunner, Silk Performer,#Jmeter#

Profilers##

Business Facing#Su

ppor

ting

the

Team

#Critique Product#

Technology Facing#

40

More information about tools

Tool Type # #WHO # # #URL ##BDD # # #Cucumber #http://cukes.info/##GUI UT# # #Abbot # #http://abbot.sourceforge.net/doc/overview.shtml#GUI UT# # #SWTBot #http://www.eclipse.org/swtbot/#GUI UT# # #TestNG# #http://testng.org/doc/index.html##UT # # #EasyMock #http://www.easymock.org/#UT # # #Junit for Java #http://www.junit.org/##UT # # #Nunit for .NET #http://www.nunit.org/#UT # # #PyUnit for Python #http://pyunit.sourceforge.net/#UT # # #Ruby/Mock#https://www.ruby-toolbox.com/categories/mocking#UT # # #Perl and Ruby #http://test-unit.rubyforge.org/#

http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks ##


Recent Trends

Test Automation is possible directly from testcases in Excel Test Driven Development is possible without coding or any knowledge of programming

Simplified BDD

Anybody can write/modify Testcases (in plain English) based on user stories in Excel Anybody can run/execute them by clicking a button

Ease of Use

No dependency on specific skills or automation engineers. Even Management, Business analysts can change/run testcases Run testcases on any OS/Browser and its versions

No Dependency

Smaller Test Cycles Low License cost Only basic training needed, No learning curve

Cost


TestOptimizer helps you automate Functional and UI Test Cases without writing a single line of code

Business Analyst or Testers write testcases

as usual in Excel or spreadsheets

Testers also write test steps to enable

automation in Excel itself

Install TestOptimizer and store test cases

sheet in defined location

Just click on TestOptimizer icon to

execute testcases in automated way

Choose browsers on which you wish to run

testcases

Get screenshots for failed testcases and get results with pass/fail in

same Excel

Get test reports in xls or html format

Get email with consolidated test execution reports

That’s All !!! Automation done !!!


OpCord Consultancy Services Private Limited, 2nd Floor, #417, Above People Showroom, 27th Main, Sector -1, HSR Layout, Bangalore – 560102

Voice: 91 – 9620887298, Email: [email protected], Web: http://opcord.com

Let’s Opcord

Let’s optimize with coordination