Sigist 2012 Managing your Automation for successful ...legacy17.sela.co.il/s/sigist12/_AllPresentations/19July/... · Sigist 2012 – Managing your Automation for successful implementation

Sigist 2012 –

Managing your

Automation for successful

implementation

Erez Shacham

Automation & Tools

Director

Agenda

Information Security Level 2 – Sensitive

© 2012 – Proprietary and Confidential Information of Amdocs 2

Test Automation introduction

Automation Methodology Life Cycle

Automation Implementation approach

Case study

Test Automation objectives

Shorten time to market (TTM)

Reduce cost of the project

Increase test coverage – Enhance testing scope

Improve quality

Increase efficiency

Improve quality

Shorter TTM

Reduce cost

The Challenge of Automation

More

Automation

Higher

Return

On

Investment

The Challenge of Automation – Cont.

● Automation scope definition

● How to reduce Automation development effort

● ROI Calculation

● Involvement of manual testing team in Automation

activities

● Automation package readiness for execution

● Use of Automation packages during execution

Agenda






Case study




Assessment Questioner



● Request information on manual procedures

● Identify pain points

● Share repetitive work

● Understand technology

● Which tools should we interact with?

● Should we support Java, Power Builder Etc?

● Web Testing

● Mobile Device Testing

Estimations and ROI



● Evaluate effort and potential ROI

● Based on complexity

● Create and use golden numbers and best

practice experience

Plan Automation - Dashboard



● Commit to timeline

● Full transparence to the teams you work with, one centralized file which covers all aspects

● Scope under test

● Progress

● Due dates

● Downtime

● Open Issues

Design and Preparation Phase



● Preparation

● Install \ Validate Tool Automation and AddIns

● Install Framework / ASAP

● Test automation environment configuration

● Connect to management testing tool

● Review each test case, within the scope, define the following:

● Prerequisite

● Script Parameters

● Error handling and recovery mechanism

● Internal script algorithm (Structure and Flow)

● Scripts dependencies

● Data table structure and content

● Verifications list

● Test Data structure preparations

Development Phase



● Skeleton Scripts – recorded and/or coded

● Tool Automation Coding Standards (Documentation, Naming Conventions, Declarations, Functions)

● Error Handling

● Synchronization Points

● Verification Points

● Common Data Files

● Reporting Results

Unit Test Phase



● Should be performed for each test script/function separately

● Each script is unit tested in Tool Automation debug mode

● Scripts are tested with all relevant input data options

● The execution report is reviewed

● Both negative & positive cases are checked

Integration Test Phase



● Check the integration between the scripts (i.e. scripts that share the same data) to make sure that the test set runs as one suite

● Support intermediate failures (this is important for the overnight run)

● Verify test environment

● Integration testing execution is done from Test Management Tool

● Review entire results report

Knowledge Transfer



● Train Manual team

● Present developed scripts and excel – Data Prep

● Walkthrough shared folder structure

● Configuration files

● BUS and GUI per version

● Log folder

● Review user guide

● Login to Test Management tool

● Create New calendar

● Configure Driver Script

● Initiate execution

● Trouble Shooting

● Maintenance ID SKIP TEST_NAME PARAM0 PARAM1 PARAM2 PARAM3 PARAM4 PARAM5

1 Create Contact Customer with Billing Profile ACTION CUSTOMER_TYPE PRODUCT_TYPE NEW_CUST_INDTITLE FIRST_NAME

2 fBusCreateCustomer Contact Customer PostPaid Y Mrs. @FIRST_NAME_CUST1

3 Create Account Customer with Billing Profile ACTION CUSTOMER_TYPE PRODUCT_TYPE NEW_CUST_INDTITLE FIRST_NAME

4 fBusCreateCustomer Account Customer PostPaid Y Mrs. @FIRST_NAME_CUST2

5 Create Contact Customer without Billing Profile ACTION CUSTOMER_TYPE PRODUCT_TYPE NEW_CUST_INDTITLE FIRST_NAME

6 fBusCreateCustomer Contact Customer PrePaid Y Mrs. @FIRST_NAME_CUST3

7 Create Account Customer without Billing Profile ACTION CUSTOMER_TYPE PRODUCT_TYPE NEW_CUST_INDTITLE FIRST_NAME

8 fBusCreateCustomer Account Customer PrePaid Y Mrs. @FIRST_NAME_CUST4

Knowledge Transfer - Execution

Test Execution

Pre Execution

• Preparations

Execution

• Run Test

Post Execution

• Analysis

Data Preparation -Test Case Example ● TC Name: “Search for a subscriber”

1. Log in Successfully

2. Subscriber phone # found

3. Details found

• Payment Options and User Information

Execution



● Execute complete packages

● Identify any intermediate failures

● Investigate the failure

● Update excel to resolve

● Re run entire flow

● Raise defects

● Review entire results report

Automation - Manual



Man

ual

Au

tom

ati

on

Scope Analysis

Plan Design &

Prerequisite Development & Unit Test

Integration Knowledge

Transfer Execution Maintenance

Share

• Details on

pain

points

• Repetitive

work

• Manual

effort

• Test

Cases

Update with

• Roadmap

• Timeline

Provide

• Business Knowledge

• Walk through on tested flows

• Env.

Participate

In status meetings

Be part

of scripts validation

OR

Participate

In status meetings

Learn How to

• Execute automated packages

• Investigate failures and resolve

• Maintain

Execute

Automated packages

Update Scripts

OR

Involve

Automation expert

Contributor

Participant

Depend – Customer’s decision which mode

Agenda






Case study

Automation Implementation Approach Overview



Gradual Development

with periodic delivery

to manual testing team

> Environment Checkers

> Sanity – Minimal

Acceptance Test

> Data Generation

> Front End; BackEnd /

SOA automation for

Core Regression

Comprehensive Coverage of

Core Regression

Main Business Activities

Entity Creation

Common Functions Library

APIs

Data

Management

Shared

Package

Initialization

Tools

Version

Control

Reports

Interfaces

ASAP- Amdocs Shared Automation Package Amdocs Layers approach

Application Under Test

Automation Tool

ASAP

Amdocs Shared Automation Package

GUI Functions \ BPT - Business components

ATI = Application Test Interface

BPT- Business Process

22

ASAP – In house developed framework Structure and Methodology

● A collection of foundation scripts and solutions developed over the years to support Amdocs automation testing teams in different projects all over the world

> Calendar file consists of

primarily 2 tabs

> MAIN – Consists of test

cases along with variable

names and fixed data

● KEEP_REFER – Consists of

all the variable data which

changes as per ENV and also

data which is created at run

time like: BAN number,

Subscriber number

Amdocs Automation Framework Parameterization and flexible design

Agenda






Case study

Automation

PSO

Manual

Tier 1 Quad Play - Case Study

Raise the needs

Brainstorming with

Manual team

Analysis of the

challenges and

search for

available solution

Solution validation

and

Implementation

Engage

Automation team,

Define

requirements

Utilities and

automation scripts

Development

Work with client

business and

testing tool team

Support

Tier 1 Quad Play - Case study – Cont.

Activity Original

Effort

New

Effort

Savings

Execution 157 73 84

157 MM

73 MM

0

20

40

60

80

100

120

140

160

180

Original New

Ma

n M

on

ths

52% Efficiency

•Dry run before testing

cycle

•184 defects were detected

during automated

execution

•Significant Automation

coverage for 1st

implementation

Thank You



Automation Implementation Approach

Example



Activity Task September October November December

Regression

AMDOCS and

non Amdocs

Design

Development

Deployment

Execution Sup

Environment

Checker +

Sanity

Design

Development

Deployment

Execution Sup

Data

Preparation

Design

Development

Deployment

Execution Sup

Sigist 2012 Managing your Automation for successful ...legacy17.sela.co.il/s/sigist12/_AllPresentations/19July/... · Sigist 2012 – Managing your Automation for successful implementation

Documents