Software Development Conference 2007 IBM RATIONAL TOOLS FOR TESTING Presentation: Vítek Linduška Demo: Václav Novotný.

Software Development Conference 2007

IBM RATIONAL TOOLS FOR TESTING

Presentation: Vítek LinduškaDemo: Václav Novotný


2

Agenda

Manual Tester

Functional Tester

Performance Tester

ClearQuest


3

Manual Tester


4

Why Teams use Manual Testing?

Recruitment of subject-matter experts and end-users to test

Poor or non-existent test automation-ready interfaces

Strict regulatory or process requirements aren’t met with automation tools


5

Challenges of Manual TestingChallenges of Manual Testing

Labor Intensive Too much time is spent creating

and maintaining manual test documentation



“More time is spent maintaining test

documentation than on executing tests.”

“We struggle to document even a

minimum set of tests.”


6

Challenges of Manual TestingChallenges of Manual Testing



Unreliable

Inadvertent human-error often skews test results



Unreliable

Inadvertent human-error often skews test results

“Our results are unreliable due to

human error.”

“Our tests aren’t detailed enough for the testers to consistently

execute them.”


7

Challenge 1: Labor Intensive

Challenge Creating and maintaining manual tests is labor intensive and difficult

with tools such as Microsoft Word or Excel.

Challenge Creating and maintaining manual tests is labor intensive and difficult

with tools such as Microsoft Word or Excel.

Resolution Implement a solution that

promotes the best practice of modular tests which can share common content

Leverage existing Word or Excel test documents to quick-start test development

Resolution Implement a solution that

promotes the best practice of modular tests which can share common content

Leverage existing Word or Excel test documents to quick-start test development

Share test

content


8

Challenge Human error often skews tests results

Challenge Human error often skews tests results

Resolution Assist testers with the tasks

most prone to error – data entry and data verification

Build manual tests that are clear, concise and easy for the tester to follow

Resolution Assist testers with the tasks

most prone to error – data entry and data verification

Build manual tests that are clear, concise and easy for the tester to follow

Challenge 2: UnreliableChallenge 2: Unreliable


9

Leverage Existing Test Documents Import from Word or Excel Microsoft Word

documents

Microsoft Excel spreadsheets

Jump-start adoption of Rational Manual Tester by leveraging existing assets

Bulk import manual tests documented in Word or Excel

Jump-start adoption of Rational Manual Tester by leveraging existing assets

Bulk import manual tests documented in Word or Excel

Rational Manual Tester test


10

Assisted Data Entry and Data VerificationReduces human error

Store critical data used during the test as part of the test instructions Store critical data used during the test as part of the test instructions

Test execution window


11


Store critical data used during the test as part of the test instructions

Automate data entry during test execution to speed entry and reduce errors



Test execution window

Application under test


12




Automate data validation during test execution to ensure accurate results



Automate data validation during test execution to ensure accurate results


13

Enables Shared Content Across Multiple TestsReduces test maintenance and ensure consistency

Create commonly used test statements once

Reuse statements across multiple tests through simple drag and drop

Maintenance of shared content is efficient with single-point updating of shared content

Ensures consistency and reduces test maintenance

Create commonly used test statements once

Reuse statements across multiple tests through simple drag and drop

Maintenance of shared content is efficient with single-point updating of shared content

Ensures consistency and reduces test maintenance


14

On-Screen Prompted Test ExecutionImmediate and accurate results collection

On-screen prompting of test steps

Immediately record test results and attach images or files for clarity

Tool will assist with data entry and data verification

Results recorded in permanent log

On-screen prompting of test steps

Immediately record test results and attach images or files for clarity

Tool will assist with data entry and data verification

Results recorded in permanent log


15

Spread-Sheet Ready Test ResultsUse with any 3rd party reporting tool

Results easily exported to common file format

Import into any 3rd party tool for evaluation and consolidation with other test data

Results easily exported to common file format

Import into any 3rd party tool for evaluation and consolidation with other test data


16

Functional Tester


Manual testing can’t keep pace!Manual testing

can’t keep pace!Rerun the first tests and...

Time

Tests

...take more time

...take more time

...the new tests...

Build 1 Build 2 Build 3, 4, 5 …

Challenge of Manual Testing with Short Test Cycles

Manual testing alone can’t keep pace with the rapid development of applications Quality will be sacrificed to keep on schedule


18

Challenges of Functional TestingChallenges of Functional Testing

Controlling cost of test automation Inexperienced testers face steep

learning curve for tools

Test maintenance outpaces testing

Controlling cost of test automation Inexperienced testers face steep


Test maintenance outpaces testing

“Our testers are not experienced in building

automated tests.”

“Application UI changes are always

breaking our test scripts.”


19


Control cost of test automation Inexperienced testers face steep


Test maintenance grows with project

Testing Tools Lack Flexibility Power users hands are tied by tool

limitations





limitations

“I need one tool for both my expert and

novice testers.”

“Our script language limits our ability to overcome some

automation obstacles.”


20






limitations

Test Team Integration Managing script changes with

parallel development

Sharing test assets with the larger software team





limitations

Test Team Integration Managing script changes with

parallel development

Sharing test assets with the larger software team

“How do I keep my team in connected with

the project team?”

“We’re geographically distributed and we’re worried about parallel script development .”


21

Challenge Overcome the complexity of test automation to ensure teams spend more

time on high value quality assessment rather than on low return test maintenance.

Challenge Overcome the complexity of test automation to ensure teams spend more

time on high value quality assessment rather than on low return test maintenance.

Resolution Provide technologies to enable

novice testers to craft complex tests with no programming Data driven wizard

Dynamic data verification

Reduce script maintenance Wizard driven global updates to

centralized object map

Resolution Provide technologies to enable

novice testers to craft complex tests with no programming Data driven wizard

Dynamic data verification

Reduce script maintenance Wizard driven global updates to

centralized object map

Challenge 1: Controlling the cost of AutomationChallenge 1: Controlling the cost of Automation


22

Challenge Fulfill both standard test needs as well as expert

requirements to ensure usage limited only by the imagination of the project team.

Challenge Fulfill both standard test needs as well as expert

requirements to ensure usage limited only by the imagination of the project team.

Resolution Provide two test script

development languages Java

Visual Basic .NET

Give professional testers the flexibility to overcome testing challenges in the language of their choice

Resolution Provide two test script

development languages Java

Visual Basic .NET

Give professional testers the flexibility to overcome testing challenges in the language of their choice

Challenge 2: Tools Lack FlexibilityChallenge 2: Tools Lack Flexibility


23

Challenge 3: Test Team IntegrationChallenge 3: Test Team Integration

Challenge Accommodate the requirements of both dedicated,

independent QA teams as well as integrated project teams to ensure high operational integrity.

Challenge Accommodate the requirements of both dedicated,

independent QA teams as well as integrated project teams to ensure high operational integrity.

Resolution Provide version control

support in the box with ClearCase LT

Provide integrations for standard version control features

Check In/Out

Compare withPrevious Versions

Show change history

Resolution Provide version control

support in the box with ClearCase LT

Provide integrations for standard version control features

Check In/Out

Compare withPrevious Versions

Show change history


24

Reduce Test Script MaintenanceReliable Playback with Script Assure

DeterminesMatch

Version 1.0 Version 2.0

TesterSees

ToolSees

No User No User Intervention Intervention

Required With Required With ScriptAssureScriptAssureTMTM


25

Performance Tester


26

Challenges of Performance TestingChallenges of Performance Testing

Complex Tools & Complex Task Tool complexity and lack of

experience intimidates many first time users



“We don’t have a lot of experience building

good load tests.”

“Most load test tools are much more

complicated than we require.”


27

Challenges of Performance TestingChallenges of Performance Testing




limitations




limitations

“We don’t have a good understanding of what

exactly is occurring during a load test.”

“We have highly technical performance

testers that need to customize their tests.”


28

Challenge 1: Complex Tools & Complex TaskChallenge 1: Complex Tools & Complex Task

Challenge Tool complexity and lack of experience intimidates many first time

users

Challenge Tool complexity and lack of experience intimidates many first time

users

Resolution Remove the need for

programming

Tests are represented in a tree view as a sequential flow web pages

User interface enables

Data pooling

Data correlation

Looping

Conditional events

Resolution Remove the need for

programming

Tests are represented in a tree view as a sequential flow web pages

User interface enables

Data pooling

Data correlation

Looping

Conditional events


29

ChallengePower users hands are tied by tool limitations

ChallengePower users hands are tied by tool limitations

Resolution Tests may call custom Java

code provided by the user Offers flexibility for power users

to perform complex tasks

Expose behind the scenes details which are hidden from novice users by default Provide the flexibility to

overcome any testing obstacles

Resolution Tests may call custom Java

code provided by the user Offers flexibility for power users

to perform complex tasks

Expose behind the scenes details which are hidden from novice users by default Provide the flexibility to

overcome any testing obstacles

Challenge 2: Tools Lack FlexibilityChallenge 2: Tools Lack Flexibility


30

Performance Problem Identification During Test

Performance Testing finds bottlenecks Next logical question is Why?

Root Cause Analysis provides to tools to answer this question

Page Performance ReportShows average response time

per page.

Highest bar = Performance Problem


31

Performance Problem Identification During Test

Performance Testing finds bottlenecks Next logical question is Why?

Root Cause Analysis provides to tools to answer this question


32

Response Time Breakdown

Feature: Breakdown page response times into composite element response time

Benefit: Breakdown data drills down to highlight slowest components of a given page.

Breakdown into tiers and components is a unique competitive advantage for RPT.

Response data broken down by tier and by transaction component (JDBC, JSP,

Servlet, etc…)


33

Deep Diagnostic DataProviding the details you need to solve the performance problem mystery

UML Sequence diagram showing method calls between

classes with timing information. User can drill

down to source code.

Execution Statistics Shows response time for

every method


34

ClearQuest


35

Eclipse Based Test Management Joins Functional and Performance

Tester to complete transition of testing tools to eclipse platform

Native and web clients also available

Key Functional Areas Test Planning

Create test plans, test cases and configured test cases

Test Execution Create suites or run individual

test cases

Test Reporting Extensive query and reporting

facilities to measure test progress

Test Management with ClearQuestEclipse, Windows and Web Clients for test project tracking


36

Lifecycle TraceabilityQuery one source for all aspects of project progress

Query to find Test results associated to requirements

Defects associated to test results

Defects associated to tests associated to requirements

Traceability from requirements to tests and defects closes the loop on the software development cycle

Requirements Tests Defects


37

Test Case Trending

Test Verdicts

Lifecycle TraceabilityQuery one source for all aspects of project progress

Leverage existing reports

orCreate new reports with Crystal Reports

To provide the information you need for decision making


38

ClearQuest Web InterfaceDistributed access for test planning and reporting

Instant access for remote users

Access for alternative platforms

No additional licenses required

Perfect for test planning and reporting


39

`

ClearQuest MultiSiteReplication and synchronization for geographically distributed teams

Automatic replication and synchronization of ClearQuest repositories

Access current information, regardless of location

Automatic recovery of repositories in the event of system failure maintains data integrity

Transmission of only incremental changes for improved network performance


40

Centralized User ManagementLDAP authentication for common user id support

Reduce administration with centralized user management – maintain only one list of user ids

LDAP authentication supports common user ids for ClearQuest

and RequisitePro

Rational RequisitePro Authorization Rational

RequisitePro

Rational ClearQuest

LDAP 3.0 Compliant Server

Rational ClearQuest

Authorization

Rational ClearQuest User DBLogon Dialog

LDAP


41

Compliance and ControlUsing Audit Trails and eSignatures to track change

Audit Trails can track changes to individual records Provides immutable change log of all

record change activity

e-Signature requirement can be added to any event on any artifact

Will record who changed a record, when and how the record was changed

e-Signature provides immutable evidence of process completion

ValidateResults

TestExecutionTest CaseRequirement Approve

e-Signature Approval


42

Configure and Enforce Your ProcessCustomize appearance and behavior of all forms and processes

Customize everything Modify forms

Configure State Transition

Add fields and control behavior

Extend your system with custom coded hooks

Extend functionality using pre-defined Packages eSignatures

Audit Trail

Many, many more…


43

Extensible Test EcosystemIntegrated with Rational, open source and 3rd party test automation tools

Collect and report on results from Rational test execution engines Rational Manual Tester

Rational Functional Tester

Rational Performance Tester

Collect and report on results from Eclipse Test and Performance Tools Project jUnit tests

Manual tests

URL tests

Open API Extensibility SPI Dynamics QAInspect


44

Rational Software Quality Solutions

Developer Test Functional Test

Automated Manual

Rational RequisitePro

Rational ClearQuest

Rational ClearQuest

Defects

Project Dashboards

Detailed Test Results

Quality Reports

Performance Test

SOFTWARE QUALITY SOLUTIONS

Test and Change Management

Test Automation

Quality Metrics

DE

VE

LO

PM

EN

T

OP

ER

AT

OIN

S

BUSINESS

Rational ClearQuest

Requirements Test Change

Rational PurifyPlus

Rational Test RealTime

Rational Functional Tester Plus

Rational Functional Tester

Rational Robot

Rational Manual Tester

Rational Performance Tester


45

WAKE UP, IT’S DEMO TIME!

Software Development Conference 2007 IBM RATIONAL TOOLS FOR TESTING Presentation: Vítek Linduška Demo: Václav Novotný.

Documents

manual tester slide

test development resolution

excel test documents

test results labor intensive

tests arent

unreliable slide

labor intensive challenge

challenge human error