Qa for ace infoway april 13-2010

Tuesday, April 18, 2023 SIX SIGMA BLACK BELT Ace Infoway Pvt Lmt 1

Define

Parent Project Title:

QA Process,Standards & Automation introduced to improve Development of websites and web-services ( Black Belt) Project By Ms Meenakshi Ganeriwala .

Start Date: 01/07/2009

Estimated End Date: 01/06/2010

Actual End Date:

Project Scope:

IS IS NOT

Critical to ACE Infoway production

Client support center of ACE Infoway

Critical to ACE Management Team

Mock ups for Clients

Critical to ACE Development Team

Data entry sector of ACE Infoway

Critical to ACE Design Team Does not involve creativity for designs

By Ms Meenakshi Ganeriwala

2

Related Projects

1. Yellow Belt by Ms Meenakshi Ganeriwala ACE Infoway Pvt Lmt

2. Black Belt Project by Mr Sandeep Kulkarni , Sr. Manager @DFS Austin Texas USA

3. Black Belt by Vision Raval

Key Learnings;

4. Development inaccuracy and instability with higher loads and varied technologies like PHP, Joomla, ,NET, Java etc

5. Absence of proper QA cycle reduces market value leading to loss of revenue. ( This is captured from one of the meetings with Dr Anu . He referred to Clients who deducted amount from payments due to increased no of defects existing in the deliverables)

6. Proactive vs. reactive monitoring of Client Deliverables

7. Data analysis and statistical significance of data from reports and charts

Tuesday, April 18, 2023 SIX SIGMA BLACK BELT Ace Infoway Pvt Lmt

Define


3Tuesday, April 18, 2023 SIX SIGMA BLACK BELT Ace Infoway Pvt Lmt

Team:

Project Lead :

Ms Meenakshi Ganeriwala ACE Infoway QA and R &D Manager

Project ID: 2426250

Team Members :

Kirtan Gajjar ACE Infoway Sr QA Analyst

Manish Varsani ACE Infoway Manager QA

Rushabh Patel ACE Infoway Jr QA Analyst

Shital Chavda ACE Infoway QA Analalyst

Exec Sponsor :

Dr P M Anarvartham ACE Infoway Vice President

BPI Mentor :

Sandeep Kulkarni DFS BPI Area Champion USA

Vision Raval Vision Enhancer

Define


Roles and Responsibilities

Manish : Execute all QA process laid out in Ahmedabad in Rajkot for QA

Kirtan : Regular updates to Blog, Status report Updates PAS allocation, All QA activities

Research on New QA Tools

Rushab: Updating Defect reports, Status report Updates

PAS allocation, All QA activities

Research on New QA Tools

4

Project Definition

Setting up Quality Assurance Center to enable quality testing of ACE applications , improving the existing level of “timely response” of development and quality of deliverables to the Clients, to meet the SLA of Clients.

Business Reason

The number and various types of applications under development in Ace Infoway is increasing rapidly . This is resulting in the increase of delivery issues for both quality and SLA agreement. This project of setting up a department which incorporated both Quality Assurance and Research was initiated to support the different technology to be used for QA and to scale up the performance of development and deliverables at ACE Infoway and there in work to resolve /eliminate the evident causes of development failures and issues. This would eliminate / reduce Clients insistence to alter pricing after delivery.

Desired State

All End to End applications to under go QA cycle with usage of all QA Tools and Standards declared at ACE Infoway, while improving application development by reduction in the number of defects of applications. To train all resources in the QA Tools and standard followed at ACE Infoway Pvt Lmt.


Define


5

Customer Output Process Input Supplier

Design Department

QA comparison with Client scope

Design QA process

Design HTML pages

Project Manager /Design

HeadDevelopment Mantis /Test

cases/QA feedback Document

Running QA using Mantis,

Developed Code, Client scope,

technical details of Development environment

Project Manager/Develop

er

Development Code review /Functionality

review

Root cause analysis and automation

CSS details Development

Production /Client

Optimized Code/Documents

Tested Code and Scope

Design/Code Documents

QA Department


COPIS

Define



Define


7

Critical to Quality Tree


Project Management Defect Free Design

Fewer issues Design

No Of design defects W3C

validation

Customer Need Drivers CTQs (Metric)

Defect free Code Timely response to application issues

No Of defects Use of QA Tools

End ClientsStandard and

Optimized code delivery

Reduce test time QA Automation

Xenu, Sniffer, QTP

Reduce Development

Resource

Reuse code from repository

Reusable Code and Design SLA compliance

Timely response to application issues

Define


CTQ - Critical to Quality (CTQ) variablesThis is done in the define stage

8

Estimated Project Cost Benefits

1. With the introduction of QA as a separate Dept 2. With using Trained QA professional3. Training developers to utilize QA Tools 4. Dedicated QA environment5. Using automation in QA 6. Submit Test Cases to Clients


MeasureBy Ms Meenakshi Ganeriwala

COST BENEFIT ANAYSIS

DescriptorUnit Of

MeasureFamily of Measure

ObjectiveEstimated

ChangeSavings Typein RS Showed Improvement

Number Of QA resource Cont Financial Lowered 50% 600,000 YES

NO of QA hours savings calculated per 100 hours lowered

% Financial Lowered 50% 1,00,000

YES

No of defects % Quality Improved 30%Reflection of QA hours

reduced YES

Number of QA ed Applications Count Productivity increased 100% soft

YES

SLA complience NA Quality improved 20% soft

YES

Number of Resources trained in QA Tools

% Financial improved 60% 50,000

YES

Total 750,000

Project Costs 50000

Net savings 7,00,000


Measure


Estimated Project Cost BenefitsSupporting slide for # 8

QA started with MBG, Hemant, Raxit, Hemlata, Kirtan, Shital , Jignesh partially /replaced by manish

No of resources

Average salary /month

% improvement

Period

7 1,00,000 Pre- QA Automation

July- Dec 09

4 50,000 50 % Post QA Automation

Jan- May 10

Now QA Has MBG, Kirtan (shared), Rushab 9shared),Shital, Manish for R&D

10

Measurement System Evaluation


Before Sample Size / Plan

Defects from the Defect reports: (before application of Automation of QA dept ) –

One set of defect reports would show defects before application of Automation and standards while another set would reflect the data after the application of automated tools and standards for QA.

Status reports (prior to QA Automation) – Quarterly data was used from the status report. It maintains the status of a particular project once it enters QA

Applications – Number of Applications supported is captured from Sheet sent weekly by Beta and reports collected weekly from all PMs (Central Application Repository)

Resources – Number of development Resources is calculated from headcount reports.

Sampling Plan to ensure random and unbiased sampling, data was used from production and development

Data Collection – For calculation of defects data was extracted from three reports 1. Defect Reports 2. Status report 3. Delay report

An MSE was performed on defects that were recorded when QA dept was absent and no automation in place.The metric is an accurate reflection of the end – user /Client’s issues with the delivered website or related web services.

Measure


11

Measure

A Sample defect report Data collection from Sep 2009 till May 2010



12

Measure

Sample data was collected for a period of time From Sept to Dec 2009 and defects Were ploted against 100 spent hours

This needs improvement . Peaks need to drop indication lower no of defects



13

PROCESS FLOW ANALYSIS


Analyze


14

Analyze

Process Map Analysis

No. QA Standards like Checklist

No design and development standards

No QA Automation

No QA scheduling

No proper allocation and calculation of QA hours and work load

No Unbiased separate QA for Client deliverables

Fishbone Analysis

Too many defects

No operational reporting/review of metrics

Insufficient knowledge of deliverables

Installs to production being done with out QA


15

Analyze

Pareto analysis reveals top contributors to the defects

5 WHY’s Analysis

Why is the validations having great no of defects?- Because not one standard or checklist is being followed

Why is code and functionality having issues That is programming ?- Because not one standard or checklist is being followed

Why is the root cause of large no of defects ,not being addressed ?- Because there was no analysis being done to address defective design and coding.

Why was no analysis being performed ?- Because there was no separate unbiased group to perform QA

Why was there no department of QA ?- The process that included separate QA was not present.

WHY

Pareto Chart



WHY

WHY

WHY

WHY

The highest contributors are •Defects from Validations•Defects from Functionality•Defects from Code programming


Improve

Series of brainstorming sessions assisted the team to gather all possible solutions and to group them into main improvement headings.

Areas of Opportunity

Possible Solutions were generated through the use of Brainstorming sessions



Improve

Risks:Of the solutions selected, one needed further risk analysis. The creation of a dedicated team /Automation Tools

could lead to disruption hence we needed to ascertain the risks prior to implementation.A force field was chosen over an FMEA, ( Failure Mode Effect Analysis) Risk could be assessed easier/quicker

Key Risk Actions derived from Force Field Analysis

Involvement of Top management to propagate the existence of QA and its importance

Met and Brainstorming with individual PMs and their Teams to understand QA and its standards

Arranging training of developers on QA standards


One of the initial steps was to established a dedicated and Skilled QA Team and to support usage of Automation and standards a R &D Team was set up.


QA Manager

QA Operations Head

Segment Lead 1 Segment Lead 2

Design QA Resource :

: n

Code QA Resource :

: n

Design QA Resource :

: n

Code QA Resource :

: n

QA R&D Team

Research Resource

Improve Phase -I


Established a Dedicated QA Environment


Location 1. QA server Details1.Operating System : (Linux)Fadora core 7.52.Space 40 GB3.Xampp 1.7 Location 2.

QA Server Details:1.Operating System (Linux)Fadora Core 7.52.Xampp 1.73.Space 160 GB

Improve

We are using XAMPP 1.7 for the QA server including Apche 2.0,Mysql 5.1.3,PHP 5.2.8,Perl, Open SSl and phpmyadmin. XAMPP is a free and open source cross-platform web server package, consisting mainly of the Apache HTTP Server, MySQL database, and interpreters for scripts written in the PHP and Perl programming languages.


Creation of QA Environment

Tuesday, April 18, 2023SIX SIGMA BLACK BELT Ace Infoway Pvt Lmt

20

Sample DataHere the number like 675, 689 are project numbers.

Server

Project Number

Site

QA_feedback

date

QA (project no) (date)-ver.doc

DEV (project no) (date)-ver.docDate format (DD-MM-YY). ver is

for versioning e.g. 1,2,3. (This is QA feedback reply from development)

Date Format (DDMMMYY based on version

e.g. 09sep09, 17oct09

Complete Site (code) based onversion e.g. site, site1, site2

Date format (DD-MM-YY)ver is for versioning e.g. 1,2,3

(This is QA feedback)

Server Description: LinuxAhm : http://192.168.10.222/Qctest/

Rjt : http://192.168.52.9/

Documents related to the sitee.g. Site configuration, Requirement

Specification, database, etc.,

Directory Structure Notes

Docs

ImproveBy Ms Meenakshi Ganeriwala


Improve Phase- IBy Ms Meenakshi Ganeriwala

Enhance the Team for QA and R & D


QA Manager

QA Operations Head

Segment Lead 1 Segment Lead 2

Design QA Resource

… n

Code ReviewQA Resource… n

Design QA Resource…. n

Code ReviewQA Resource….n

R&D Team

Research Resource

ProductionSupport Lead

L1 Production Support……n

L2 Production Support…n

Improve Phase - IIBy Ms Meenakshi Ganeriwala


Improve Phase - IIBy Ms Meenakshi Ganeriwala

24

Process Diagram PAS Hours


This enables exact calculation of PAS hours ( billable hours for QA Process.

Improve

PAS hours utilization for example resource

PAS hours of every QA resource id closely monitored to have accurate Billable hours.

Attempt is made to make sure the process defined is cost effective



Improve – Phase iii


This Phase of QA is specially designed to handle errors occurring in the production environment due to non functioning of the sql statements embeded in the code that fetch data from the database. For example usage of count (*) instead of designing a sql statement that fetches only required fields.

26

Improve

After implementation of the new QA standards and Tools the Following situation was observedFewer no of defects for projects per 100 spent hours



Defects do not run more than 35 numbers

Tools used•Xenu •Zend Studio •Dead Links•Mantis•QTP•Load Runner•WinRunner

Improved range

RANGE

27

Improve

Comparison chart of data pre and post implementation of QA standards



The diagram and the spread sheet show clear improvement in defect detection and solution after the implementation of QA standards, Tools and Training to resources ( The PMs and Developers)

28

Before Implementation of QA Standards and Automaton

After Implementation of QA Standards and Automation

Improve



29

Improve



This slide is to support Slide No 30 and Slide 8, % improvement in the QA hours Utilized is Sampled out for reference

For every 100 $ spent in QA there is a reduction of approx 50 % in the cost that is rounded Rs 53620 for period six months (Oct 2009 – March2010 )year savings should be Total QA hours --- 3574 hours as per spread sheetTotal savings yearly --- 1,00,000 RsReference slides ----- PAS hours data, TITO data

30

Improve


AREARS OF IMPROVEMENT1. No actual hours of QA accounted2. No use of QA Automation3. PAS not noted

IMPROVED 1. Gain in $ amount showed2. Actual data recorded3. PAS monitored4. QA Automation used

FROM

July 09 to Dec

0909

FROM Jan 10 to May10


Improvement seen by 50 %

31

ImproveArea of improvement detected in the Analyze Phase



SIGMA IS IMPROVED from 54.46 to 8.28Process improvements resulted insignificant reduction in Mean & STD!

32



Points are more centered and the same can be observed in the above diagram

Before Implementation of QA Standards and Automaton

After Implementation of QA Standards and Automation

Test for Normality

(Weekly Defects reports - after)

33



Cp (before) = 0.0058, Cp (after) = 0.0832 ( Will it ? ) Improved potential capability by reducing variation

Cpk (before) = 0.000581, Cpk (after) = 0.38245 ( Does it ?) The new process is several times more effective at producing results within the

specification because variation was reduced and new process is centered close to target.

34

QA Scheduling

Checklist1. Checklist for design 2. Checklist for

development.3. Coding Standards as per

document in ACE

Automation Tools4. W3C validation5. Zend Studio6. Dead link7. Xenu8. PHP Codesniffer9. Mantis Bug Tracker

( referance to slide 24 )10. QTP ( Quick Test

Professional by HP )


1. Procurement: QA team receives the data of new projects (From Project Manager/ Beeta )

2. PM send report to QA Lead every Saturday for scheduling of projects

Scheduling: Based on the size of the project and go live date, the project is scheduled for QA

3. Control:QA notifies by email to Project Manager for:1) The required documents supporting project requirement to QA2) Website configuration setting to be done while uploading code on

QA server3) Implementing QA feedback4) Rescheduling QA if requested for any project by project manager5) Completion of QA

Control

QA Standards and Tools


35

QA Reports

1. QA tracks the schedule for the project Document name: QA_schedule.xls QA schedule <dd-mm-yyyy>Link: \\ace-iis\QA-Doc\QA schedule

2. QA tracks the status for the project Document name: QA_status.xlsLink: \\ace-iis\QA-Doc\QA Status

3. QA provides feedback for the defects in the project.

Document name: QA_(Project number) (date).docLink: \\ace-iis\QA-Doc\QA Feedback

4. Representation of delays Delay ReportDocument name: Delay_Report_(location)_.xlsLink: \\ace-iis\QA-Doc\QA Delay Report

5. Representation of defects Defect ReportDocument name: Defect_consolidated_(location)_ report.xlsLink: \\ace-iis\QA-Doc\QA Defect Report


Control

A sample Action log


36

A Sample delay Report :( A report which states why a particular project got delayed in entering the QA cycle.)


Control

A list of QA Documents and their available updates


37

A sample Mantis Report as a part of QA Reporting


Control


38

A sample Status Report to track the status of the project period: from Sep 2009 till May 2010


Control



Control

A list of QA Documents available at a particular locationRegular updates are published 1. Blog2. News Bulletin3. Send via Email for selective crowed


40

QA Document Repository

QA team maintains the blog highlighting the standard practices to improve the development and reduce the development time.

– Link http://192.168.10.222/developmentace/


Vision DocumentLink: \\ace-iis\QA-Doc\QA Vision

Action logLink: \\ace-iis\QA-Doc\QA Action Log

PHP standardsLink: \\ace-iis\QA-Doc\QA PHP Coding Standards

Test CasesLink: \\ace-iis\QA-Doc\QA Test Cases

QA Training for resourcesLink: \\ace-iis\QA-Doc\QA_Training Doc

Knowledge Sharing @ ACE

Control


http://192.168.10.222/developmentace/

http://192.168.10.222/developmentace/


QA Research & DevelopmentPHP Code sniffer

Link http://pear.php.net/package/PHP_CodeSniffer PHP Documenter

– Link http://phpdoc.org/MANTIS

– Link http://zerozone.com/mantisbtAdmin Tool for ACE ( out of the Box)

– Link http://192.168.10.222/Qctest/admintool/adminDead Link Tool that detects dead links

– Link http://www.dead-links.com/– Link http://home.snafu.de/tilman/xenulink.html

Zend Studio – Link http://www.dead-links.com/

QTP ( Quick Test Professional by HP)– Link http://www.hp.com/

LoadRunner– Link http://www.hp.com/

WinRunner– Link http://www.hp.com/

Control

THANKS


http://pear.php.net/package/PHP_CodeSniffer

http://phpdoc.org/

http://zerozone.com/mantisbt

http://192.168.10.222/Qctest/admintool/admin

http://www.dead-links.com/

http://home.snafu.de/tilman/xenulink.html

http://www.dead-links.com/

http://www.hp.com/

http://www.hp.com/

http://www.hp.com/

Qa for ace infoway april 13-2010

Education

qa activitiesresearch

ms meenakshi ganeriwalateam

executeall qa process

qa activities research

new qa toolsrushab

project lead

project scope

end applications