It's the 2010's. Why are you Still Manually Writing Tests?

Pre-Con Education: It's the 2010s. Why Are You Still Manually Writing Tests?

Chris Kraus

DevOps: Continuous Delivery

CA Technologies

Product Manager

DO4X81E

@ChrisKraus3

#CAWorld

2 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

For Informational Purposes Only

Terms of this Presentation

© 2015 CA. All rights reserved. All trademarks referenced herein belong to their respective companies. The presentation provided at CA

World 2015 is intended for information purposes only and does not form any type of warranty. Some of the specific slides with customer

references relate to customer's specific use and experience of CA products and solutions so actual results may vary.

Certain information in this presentation may outline CA’s general product direction. This presentation shall not serve to (i) affect the rights

and/or obligations of CA or its licensees under any existing or future license agreement or services agreement relating to any CA software

product; or (ii) amend any product documentation or specifications for any CA software product. This presentation is based on current

information and resource allocations as of November 18, 2015, and is subject to change or withdrawal by CA at any time without notice. The

development, release and timing of any features or functionality described in this presentation remain at CA’s sole discretion.

Notwithstanding anything in this presentation to the contrary, upon the general availability of any future CA product release referenced in

this presentation, CA may make such release available to new licensees in the form of a regularly scheduled major product release. Such

release may be made available to licensees of the product who are active subscribers to CA maintenance and support, on a when and if-

available basis. The information in this presentation is not deemed to be incorporated into any contract.


Abstract

What's in a test? We invoke an Application Programming Interface (API)

test hoping to validate business logic and update the system of record,

but why is it so hard to find those systems of record? This introductory

session will explain and demonstrate how CA Continuous Application

Insight (CA CAI) can observe a manual test and then generate tests for

web UI APIs by creating the database validation steps for you. We will

also show you how CA CAI can be used to generate 1000’s of tests vs.

spending days getting the validation right for a single manual test!

Chris Kraus

CA Technologies

Product Manager


Agenda

WHERE ARE THE API’?

FINDING THE APIs WITH CA CAI

WHAT ARE BASELINE, DATA DRIVEN, AND EXPANDED TESTS?

HIGH FIDELITY TESTS – REMOVE HUMAN EDITING OF API TESTS

DEMO

1

2

3

4

5


Where Are the APIs – I Know I Saw Them on a Whiteboard


Finding the APIs

ESB

Mobile applications and

Rich browser applications

Services provided by the ESB

Data being serviced up by

ESB


What Does CA Continuous Application Insight Consider an API?

Broad definition of “service” or API– Requests data from a app server where

we have a agent

– Lines between agents that follow the transactions

– REST and SOAP requests for data from browser application


CA Continuous Application Insight Common API’s button

A Common API is one that more than 1 agent has consumed

Focus the analysis on 1st call into the application not the buried “shared code”

The transactions are now from different consumers perspective


What Are These Tests Generated by CA Continuous Application Insight?

Baseline Test (Expanded)– A baseline test is the most simple replay a observed requests

– Validate the same response code returned

– Graphical Side by Side diff on XML

– A separate test is generated for each observed request and response

– A suite is generated to run all of the tests

– A alternate.config is generated for each baseline

IS THE SYSTEM BEHAVING THE SAME WAY IT DID IN THE PAST?




Invoke and Verify Test (Expanded with database validation)– Transaction path can traverse multiple agents

– Single step test 1 invoke and DB steps follow

– Database Validation generated to ensure SOR has been updated

Insert

Update

Delete

DID THE SYSTEM OF RECORD GET UPDATED? DID THE ERROR GET HIDDEN BEHIND THE UI?




Data Driven tests – Same as Baseline, but a Large Data Set (LDS) is used to drive test

– When you need to modify the generated test use this – 1 test many rows of data

– Refresh the data with RR pair generation

IS THE SYSTEM BEHAVING THE SAME WAY IT DID IN THE PAST? – BUT I NEED TO UPDATE TEST




Functional Test Cases (tests generated from document transactions)– The tests follow the API’s of the user’s journey through the application

– Tests contain multiple steps in the order a user ran them

– High Fidelity test generated

Datasets for input data

Filters for passing data between steps

– Supports invoke and verify (database validation)

API TEST FOLLOWS THE USERS JOURNEY THROUGH THE APPLICATION




Functional Test Cases from Browser (Chrome Browser plugin -no agent required)– The tests follow the API’s of the users journey through the application

– Tests contain multiple steps in the order a user ran them

– High Fidelity test generated

Datasets for input data

Filters for passing data between steps

– Supports invoke and verify (database validation) when agent is instrumenting backend

API TEST FOLLOWS THE USERS JOURNEY THROUGH THE BROWSER




High Fidelity Tests – Remove Human Editing of API Tests

Externalize connections to SUT – alternate.config

Create datasets for test input so data can be changed and expanded

Filter response data and parameterize into the next step

Reverse back JDBC calls into validation steps

Test Templates – provide test to handle login, sessions, and customization


Demos

Baseline Test FidelityInvoke and Verify

Functional Test


Q & A


Recommended Sessions

SESSION # TITLE DATE/TIME

DO3T06SThe CA Roadmap for Service Virtualization and the Dev

and Test Portfolio11/18/2015 at 2:00 pm

DO3T38TTech Talk: Extreme Automation—Creating Headless Tests

“Automagically”11/18/2015 at 3:00 pm

DO4T27TTech Talk: Building Regression Tests to Increase Velocity and Prevent Things from “Going Boom” in the Night

11/19/2015 at 11:30 am


Must See Demos

Application Test DemoCreate Better Application Tests

Theater 4

DevOps Sim

DevOps Simulation Experience

Theater 3

Continuous App InsightAutomate Tests for Better Apps

Theater 4

Service VirtualizationDeliver Better Apps Faster with SV

Theater 3


Follow On Conversations At…

Smart Bar

Theaters 3 & 4

Tech Talks

Theaters 3 & 4


For More Information

To learn more, please visit:

http://cainc.to/Nv2VOe

CA World ’15

http://cainc.to/Nv2VOe

http://bit.ly/1wbjjqX

It's the 2010's. Why are you Still Manually Writing Tests?

Technology