Test Tooling in Visual Studio 2012 an overview

TechNightClemens Reijnen

Continuous Quality

‘ility’ testingAlpha/ BetaComponent TestsExamplesExploratory TestingFunctional TestingLoad TestingPerformance PrototypesScenariosSecurity TestingSimulationsStory TestsUnit TestsUsability TestingUser Acceptance Testing

Master Test Plan

Managing the Total Test ProcessWhat will be tested for each test level

Based on Business Driven aspects of Result, Risks, Time and Cost.

BusinessDevelopment Test Operations

Every test must be able to rely on previous testsTest goals and depth of test coverage per phase are made compatible with each other using a Master Test Plan.

All test types are supported by Visual Studio 2012Functional Testing, Examples, Story Tests, Prototypes, Simulations, Exploratory Testing, Scenarios, Usability Testing, User Acceptance Testing, Alpha/ Beta, Unit Tests, Component Tests, Performance & Load Testing, Security Testing, ‘ility’ testing.

Test Levels, Phases and Types

Rely

Support

http://www.tmap.net/en/tmap/4-essentials/structured-test-process/master-test-plan

Performance & Load TestingSecurity Testing‘ility’ testing

Unit TestsComponent Tests

Exploratory TestingScenariosUsability TestingUser Acceptance TestingAlpha/ Beta

Functional TestingExamplesStory TestsPrototypesSimulations

Tests in the top quadrants proof business needs.

Test in the bottom quadrants support the implementation.

Agile Testing Quadrants

Performance & Load TestingSecurity Testing‘ility’ testing

Unit TestsComponent Tests

Exploratory TestingScenariosUsability TestingUser Acceptance TestingAlpha/ Beta

Functional TestingExamplesStory TestsPrototypesSimulations

Tests in the right-hand quadrants help the product.

Agile Testing QuadrantsTests in the

left-hand quadrants

help the team.

Performance & Load TestingSecurity Testing‘ility’ testing

Unit TestsComponent Tests

Exploratory TestingScenariosUsability TestingUser Acceptance TestingAlpha/ Beta

Functional TestingExamplesStory TestsPrototypesSimulations

Visual Studio Support

Supporting the Team and Implementation

Unit TestsComponent Tests

Isolate each part of the program and show that the individual parts are correct.

Crafted during the sprint, executed continuously.

Unit Tests

DEMOUnit test

Developer Focused Testing

Red Green Refactor

Pluggable test frameworks

Run test after build

Unit Test Explorer

Run from Unit Test Explorer

Code Coverage

Isolating code for better testing

Shims – run-time interception lets you replace callStubs – concrete implementations of interfaces or abstract classes

Fakes

Strive to refactor the code so you don’t need the Shims anymore

Supporting the Team and Implementation

Unit TestsComponent Tests

But Wait, There's

More!

Find semantically similar code

Code Clone

Understanding Class Coupling

Code Analysis

Check coding rules

Code Analysis

Code Performance

Supporting the Team and Business

Functional TestingExamplesStory TestsPrototypesSimulations

Story Boarding

Prototypes Simulations

Beginning of the sprint make pbi’s clear.

DEMOPowerPoint Storyboarding

Specify Functional Tests

Test Runner

Action Recording

Execute Test Cases

Coded UI

DEMOManual test case specification and execution with Microsoft Test ManagerCreate and execute Coded UI test

TIP for get testing done in a sprint

Test Automation Levels0

1

2

3

4

No automation

Shared Steps with action recordingTest Cases with action recordingGenerated Coded UI

Customized Coded UI

Test Automation Levels

Number of test cases per level in a

sprint.

Supporting the Product and Implementation

Performance & Load TestingSecurity Testing‘ility’ testing

Performance test: To determine or validate speed, scalability, and/or stability.

Load test:To verify application behavior under normal and peak load conditions.

Performance & Load Testing

Supporting the Product

proof business needs

Automated

Web Performance Test

Load Test

DEMOWeb Performance and Load Test

Supporting the Product and Business

Exploratory TestingScenariosUsability TestingUser Acceptance TestingAlpha/ Beta

Simultaneous test design and execution.

Test working software.

Exploratory Tests

support the implementation

Supporting the Product

Automated

3 Components of Testing

time spent during exploratory testing.(courtesy Jon Bach)

test setupbug fillingtesting

DEMOExploratory Test

The Planning Phase

Formulating a cohesive and broadly supported approach with which the test assignment can be successfully executed. An important part of the planning phase is the creation of the test plan, for the purpose of informing the client and other stakeholders concerning the approach, schedule, budget, activities and the (end) products to be delivered in relation to the test process. If an overall master test plan exists, the test plan should be derived from it.

http://www.tmap.net/en/tmap/4-essentials/structured-test-process/acceptance-and-system/planning-phase

Planning the test effort

Master Test Plan

Product Backlog Sprint Backlog Scrum Board

Product Backlog

Value, risk, priority, and necessity.

Logical Test Cases

Risk Classification Business Value

Acceptance Criteria

Grooming Backlog

Acceptance Criteria

Title / Description(describing the story)Narrative:As a [role] I want [feature] So that [benefit]

Acceptance Criteria: (presented as scenarios)Given [context]  And [some more context]...When  [event]Then  [outcome]  And [another outcome]...

Acceptance Test Driven

Logical Test CasesThe logical test cases consist of the combining of the test situations derived from the scenarios and data.

Test CoverageA test design technique is used to derive the necessary test cases that achieve the required coverage from a specific test basis.

The Done List

"Are you done yet?“Development Teams deliver an Increment of product functionality every Sprint. This Increment is useable, so a Product Owner may choose to immediately release it.

Shared UnderstandingEveryone must understand what “Done” means.

Testing in a Sprint

Get a TeamPeople work together, also with testers.

Get Testing knowledgeIt is a must for planning the developmenteffort.

Have Test TasksTest is part of the ‘Definition of Done’.

Collaborative cultureTrust

Reward

Multi disciplinary

Test Specification Tasks

Define Test Cases based on the PCT2 Test Design Technique

Path combinations for decision point 1: (1,2) and (1,3)Path combinations for decision point 2: (2,4) and (3,4)

Based on PCT path combinations for test measure 2 are 1-2-4 and 1-3-4 for these paths test cases should be created.

Test Lifecycle

Every phase has it’s specific test tasksPlanning, Control, Preparing, Specification, Execution, Completion, Control, Infrastructure.

Implementation Sequence

The PBI isn’t ready to be tested till coding is done.

DEMOPlanning the Sprint

www.ClemensReijnen.nl@ClemensReijnen