Top Banner
Where Value & Innovation co-exist Testing Automation CORE and TAM Road Map October , 2005 Raja Ramesh
32

Automation Presentation

Apr 11, 2015

Download

Documents

api-3698888
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Automation Presentation

Where Value & Innovation co-exist

Testing Automation

CORE and TAM Road MapOctober , 2005

Raja Ramesh

Page 2: Automation Presentation

Where Value & Innovation co-exist

April 11, 2023 2

Agenda

Why do we need test automation? What can be test automated? Need for automation at SHPS Automation objectives Automation script architecture Automation MANAGER CORE automation plans TAM automation plans Challenges on hand Automation demo Discussion

Page 3: Automation Presentation

Where Value & Innovation co-exist

April 11, 2023 3

Why do we need test automation?

Speed up testing to accelerate releases Allow testing to happen more frequently Reduce costs of testing by reducing manual labor Improve test coverage Ensure consistency Improve the reliability of testing Allow testing to be done by staff with less skill Define the testing process and reduce dependence on the few who know it Make testing more interesting Develop programming skills

Page 4: Automation Presentation

Where Value & Innovation co-exist

April 11, 2023 4

What can be test automated?

Regression testing Tests that can run on every build are excellent candidates for automation

Configuration tests If testing required for different platforms, automation may help

Test bed setup Automating setup procedures will save time

Non-GUI testing It’s almost always easier to automate command line and API tests than it

is to automate GUIs

Build readiness QA time can be better utilized if unstable builds can be identified through

automation

Page 5: Automation Presentation

Where Value & Innovation co-exist

April 11, 2023 5

Need for automation at SHPS

# Item CORE TAM

1 Total Applications 41 18 *

2 Test Cases 2006 2055

3 Hours for Manual Execution 171 Man Hours 740 Man Hours

4 Frequency of Regression Testing 3 to 4 time/release ** 1 time/release

5 Sanity testing (Man Hours) 2-3 3-4

6 Frequency of Sanity Testing Every Build Every Build

* TAM CSR is executed for 23 clients and TAM PPWEB executed for 18 client.** Few application are executed only once a release. These applications occupy 40-50 Man hours of CORE regression and 10-12 hours of TAM regression.

Page 6: Automation Presentation

Where Value & Innovation co-exist

April 11, 2023 6

Need for automation SHPS cont…

Multiple regressions is not possible CORE takes 21 man-days for one iteration TAM takes 93 man-days for one iteration

Cost of labor Reasonable labor is being spent on manual testing

Limitation on aggressive schedules Short release cycles not possible Parallel releases not possible

No tool/mode for confirming build readiness Manual cursory testing 3-4 hours effort

Prone to skip test scenarios Due to repeated testing of test cases, it is possible for the tester to

overlook/assume correctness of some scenarios

Page 7: Automation Presentation

Where Value & Innovation co-exist

April 11, 2023 7

Automation objectives

Reduce the regression testing time by at least 60-70% Provide quick sanity testing facility at onsite Improve the regression testing execution frequency Develop scripts with minimal maintenance time Provide facility for everyone to run automation scripts Provide facility to receive automation test results through email

Page 8: Automation Presentation

Where Value & Innovation co-exist

Automation overview

Page 9: Automation Presentation

Where Value & Innovation co-exist

April 11, 2023 9

Script architecture - Guidelines

External data source Generic code Flexibility Controllability Maintainability Operability Reporting ability

Page 10: Automation Presentation

Where Value & Innovation co-exist

April 11, 2023 10

Script architecture – Baseline suite

Start

Driver Script

Get GUI FilePath / Exception

Data

Database

.INIFile

ApplicationScript

GenerateReport

Get Test Data

Database

Invoke theApplication

Navigate to theScreen

GUIVerify?

Functional TestCase execution

No

Yes

ChecksResults

Results

Each Test Case

Next

Database

Stop

GUI Verification

Page 11: Automation Presentation

Where Value & Innovation co-exist

April 11, 2023 11

Script architecture – Participant WEB

Start

Load all Clients(Active)

STAGE 1: DB Module

Get Active Links forPartcipant Menu and

DirectorMenu

Each Client

SQADatabase

Insert Data

NextCORE DB

STAGE 2: GUI Module

Each Client

GenerateGUI Map file

SQADatabase

Path

Next

STAGE 3: Integrated Module

Each Client

Next

Load GUI Map

SQADatabase

Link Checker

Update

Generate ReportResult

(email, excel)

Stop

Page 12: Automation Presentation

Where Value & Innovation co-exist

April 11, 2023 12

Script architecture – Cursory testing

Start

ExecuteDriver Script

For eachModul/App.

Launch theModule/App.

Get Test CasesExecute theParent TC

Pass?Execute rest of the Test Cases

Yes

Next

No

SQADatabase

Genearte Report

Report(Email, Excel) Stop

Page 13: Automation Presentation

Where Value & Innovation co-exist

April 11, 2023 13

Automation MANAGER

What is Automation Manager A utility to ease the operation folks jobs Can be executed by staff with less testing skills No dependency on offshore SQA

ON-FLY configure Applications that needs to be tested Test cases that needs to be executed (single, multiple or a range) Test results communication (email, results in shared folder etc)

Page 14: Automation Presentation

Where Value & Innovation co-exist

April 11, 2023 14

Automation MANAGER – process flow

Start

InitiateAutomation

Manager

Choose theApplication to

execute(Single / Multiple)

Choose the TestCases for the app.(Single / Multiple /

Range)

Enter emailaddress for

receiving testresults

SCOPE: User / Tester SCOPE: Automation Manager

LaunchPscExec(Remote)

Update

.INI file(Remote file)

SCOPE: WinRunner Server

TriggerPsExec

Batch file(Remote)

TriggerWinRunnerBatch file

Driver Script Path

LoadDriver Script

LoadApplication Script

RetreiveTest Data

SQADatabase

ExecuteTest Cases

Each Application

Result

Next

Generate Report

Report(email / Excel) Stop

Page 15: Automation Presentation

Where Value & Innovation co-exist

April 11, 2023 15

Automation MANAGER - prototype

Continued…

Page 16: Automation Presentation

Where Value & Innovation co-exist

April 11, 2023 16

Automation manager – prototype cont…

Page 17: Automation Presentation

Where Value & Innovation co-exist

CORE Automation

Page 18: Automation Presentation

Where Value & Innovation co-exist

April 11, 2023 18

Current automation status - CORE

# Application Total Cases Cases Automated

Hours for Manual Execution

Manual Hours with current Automation

1 CSR 173+30 48 10 7

2 HR Access 224+143 59 12 8

3 Case Setup * 119+126 103 22 NA

* For Case Setup, scripting is completed. Test data needs to be added.

1 Total regression test cases 2006*

2 Total test cases automated 107 **

3 Effort required to perform total regression manually (Man hours) 170

4 Effort required to perform regression with current automation (Man hours) 163

* Includes functional & end-to-end cases for all the 41 CORE applications.

** Does not include 103 case setup test cases for which, test data still needs to be added.

Page 19: Automation Presentation

Where Value & Innovation co-exist

April 11, 2023 19

Automation scope - CORE

# Regression Testing Cursory Testing

1 CSR Case Setup

2 HR Access CSR

3 Case Setup HR Access

4 Participant Web * Participant Web

5 Load Passwords CallDoc

6 CallDoc Template Setup

7 Knowledge Base Database checks

8 STD Load

9 AddrPBSI

10 Load PBSI

11 LoadEmailAddress

12 Template Setup

13 Survey Setup

Page 20: Automation Presentation

Where Value & Innovation co-exist

April 11, 2023 20

Proposed automation road map - CORE

VII

[TBD]

VI

[End of Nov 05]

V

[End of Nov 05]

IV

[TBD]

III

[End of Oct 05]

II

[Mid of Nov 05]

PHASE I

- Sanity Test Automation

- Save 3 hours per buildI

[ Mid of Oct 05]

PHASE II

- Case Setup & Template Setup

- Save 18 hours per build

PHASE III

- Participant WEB

- Save 07 hours per build

PHASE IV

- Client Specific

- Save XX hours per build

PHASE V

- End to End cases Automation

- Save 21 hours per build

PHASE VI

- CallDoc, KB, STD Load,

- Load Password, Address PBSI,

- Load Email Address, Load PBSI,

- Survey Setup, Template Setup

- Save 27 hours per build

PHASE VII

- Test Cases Version Control

- Test Data Automation

Page 21: Automation Presentation

Where Value & Innovation co-exist

April 11, 2023 21

Road map in detail – CORE

* Often, multiple phases are executed in parallel. Team is already working on Phase2, Phase3 & Phase6

# Phase Applications Cases Included

Expected Hours

Start Date End Date

1 Phase1 Cursory tests 30-40 420 9/26/05 10/12/05

2 Phase2 Case Setup 298 192 10/13/05 11/15/05

3 Phase3 Participant Web 160 600 9/12/05 10/31/05

4 Phase4 Client Specific tests TBI TBI TBD TBD

5 Phase5 End-to-end testing 200 480 10/13/05 11/23/05

6 Phase6 CallDoc, KB, STD Load, Load Password, Address PBSI, Load Email Address, Load PBSI, Survey Setup, Template Setup

220 608 9/19/05 11/28/05

7 Phase7 Miscellaneous – version control & test data reconstruction

TBI TBI TBD TBD

Page 22: Automation Presentation

Where Value & Innovation co-exist

April 11, 2023 22

SUMMARY: CORE automation

Total regression cases (all applications functional & end-to-end) 2006

Total time required if regression is done manually (Man Hours) 170

# Phase Total Cases Automated

Total Hours Required Automate

Manual efforts still required to complete regression (Man Hours)

1 Before Phase1

107 NA 163

2 Phase1 30-40 420 NA

3 Phase2 298 192 145

4 Phase3 160 600 138

5 Phase4 TBI TBI NA

6 Phase5 200 480 117

7 Phase6 220 608 90 *

8 Phase7 TBI TBI NA

* There are about 30 applications that are executed only once a release. These applications occupy 40-50 hours of total regression time.

Page 23: Automation Presentation

Where Value & Innovation co-exist

TAM Automation

Page 24: Automation Presentation

Where Value & Innovation co-exist

April 11, 2023 24

Current automation status - TAM

# Application Total Cases

Cases Automated

Cases still need to be Automated

Hours for Manual Execution

Manual Hours with current Automation

1 TAM CSR 537 395 54 342 123.2

2 Participant Web 362 342 10 188 31

4 TAM Case Setup 705 00 705 11 NA

* CSR cases are executed for 23 clients and case setup cases are executed for 18 clients

1 Total regression test cases 2055*

2 Total test cases automated 737

3 Effort required to perform total regression manually (Man hours) 740

4 Effort required to perform regression with current automation (Man hours) 364

* Includes functional & end-to-end cases for all the 41 CORE applications.

Page 25: Automation Presentation

Where Value & Innovation co-exist

April 11, 2023 25

Automation scope – TAM

# Cursory Testing Regression Testing

1 Core Case Setup 11 STD Daily TAM CSR

2 TAM Case Setup 12 STD Daily Cigna TAM Case setup

3 TAM CSR 13 STD Daily CNA TAM Participant Web

4 TAM Batch 14 STD Daily LEA

5 Auto Close 15 Intel TAM Out

6 Calculations 16 Participant Web

7 Email Batch 17 HR Access

8 Letter Batch

9 Printed Letters

10 TAM Termination Batch

Page 26: Automation Presentation

Where Value & Innovation co-exist

April 11, 2023 26

Proposed automation road map - TAM

V

IV

III

II

I

Current

Current Status

- TAM CSR – 70% case automated

- Participant WEB – 95% cases automated

- Saving 376 hours per build

PHASE I

- Sanity Test Automation

- Save 3-4 hours per build

PHASE II

- TAM Case Setup

- Save 13 hours per build

PHASE III

- End to End cases Automation

- Save 4 hours per build

PHASE IV

-Test Data Automation

-Save script maintenance time

PHASE V

- Test Cases Version Control

- Save 40-50 hours of test result review time per build

Page 27: Automation Presentation

Where Value & Innovation co-exist

April 11, 2023 27

Road map in detail – TAM

# Phase Applications Cases Included

Expected Hours

Start Date End Date

1 Phase1 Cursory tests 40-50 250

2 Phase2 CSR, Case Setup & HR Access

769 416

3 Phase3 End-to-end testing 32 150-170

4 Phase4 Test data reconstruction All TBD

5 Phase5 Miscellaneous TBI TBD

* Often, multiple phases are executed in parallel.

Page 28: Automation Presentation

Where Value & Innovation co-exist

April 11, 2023 28

SUMMARY: TAM automation

Total regression cases (all applications functional & end-to-end) 2055

Total time required if regression is done manually (Man Hours) 740

# Phase Total Cases Automated

Total Hours Required Automate

Manual efforts still required to complete regression (Man Hours)

1 Before Phase1

737 NA 364

2 Phase1 40-50 250 NA

3 Phase2 769 416 351

4 Phase3 32 150-170 347 *

5 Phase4 TBI TBD NA

6 Phase5 TBI TBD 300-310

* On average, 200 man hours are spent on testing letters & emails application, which can not be automated. Also, there are about 10 applications that are executed only once a while. These applications occupy 10-12 hours of total regression time.

Page 29: Automation Presentation

Where Value & Innovation co-exist

April 11, 2023 29

Challenges on hand

Research Extensive research is required to implement Phase5 & Phase6 (CORE)

Automatic Test Data Writing queries to pick test data from database requires a lot of additional study on the

system and tool.

Support from other teams Phase4 (CORE) requires support from Business development and Tech OPS teams

Prioritization of application automation SQA team needs SHPS support to finalize the list of applications for sanity testing

WinRunner instances Require multiple instances of WinRunner so that multiple tests can be executed

simultaneously

QuickTest Pro tool availability QuickTest Pro, a new regression testing tool is required to automate Knowledge Base

Page 30: Automation Presentation

Where Value & Innovation co-exist

Automation DEMO

Page 31: Automation Presentation

Where Value & Innovation co-exist

Discussion

Page 32: Automation Presentation

Where Value & Innovation co-exist

Thank you!