1 Software Testing and Quality Assurance Lecture 30 – Testing Systems
Dec 21, 2015
2
Lecture Objectives Learn how to write test cases from use
cases Learn how to quantify the quality of
testing
3
Complementary strategies for selecting test cases: Orthogonal Defect Classification (ODC)
ODC captures information about the type of faults that are present in a software system under development
4
Use cases as sources of test cases
Constructing use profiles Using scenarios to construct test cases The expected result section of a test
case
5
Use cases as sources of test cases: constructing use profiles
The construction of use profile begins with actors from the use case diagram.
After deployment the use case frequencies can be updated based on actual data and used for regression testing.
6
Use cases as sources of test cases: Using scenarios to construct test cases
The process for identifying specific values from the variables mentioned in a use case has four steps: Identify all of the values that will be supplied by the
actors contained in the use case. Identify equivalence classes of values for each input
data type. Construct tables that list combinations of values
from the various equivalence classes. Construct test cases that combine a single
permutation of values with the necessary environmental constraints.
8
Use cases as sources of test cases: Using scenarios to construct test cases— example (cont...)
Each row specifies a test.
9
Use cases as sources of test cases: the expected result section of a test case
For complex systems it is very difficult to determine the expected results from a test run. E.g. spacecraft control software.
10
Use cases as sources of test cases: the expected result section of a test case
Techniques to reduce that amount of effort required to develop expected results: Construct the results incrementally
The test cases are written to cover some subset of a use of the system. E.g. database (from 50 to 1000 records)
Grand tour test cases: results of one test case are the inputs for the next test case
Disadvantage: a failure to test case 1 we can’t run test 2.
11
Testing multiple representations Systems are written using a composite of complied
servers (c++), interpreted clients (java), and multiple data sources.
Important features of these systems from a testing perspective: Interactions between two data models. Interactions between pieces written in different
languages. Interaction between static and dynamic portions of
the program.
12
Testing multiple representations (cont...)
To address these features perform a thorough unit test process:
All pieces of the system should be exercised Test the interactions across language and representation
boundaries.
Test the interfaces between two languages to determine that a complete object is transferred.
13
Testing multiple representations (cont...)
Test the following: Be certain that each connection in either
direction is exercised Validate the results by examining the
transformed objects in detail to ensure that nothing is missing or misinterpreted
Measure coverage by covering all of the primitive types of each language
14
What needs to be tested: testing for qualitative system attributes Testing for qualitative system attributes
Translate each qualitative claim into measurable attribute
Design test cases that can detect the presence or absence of these measurable attributes
Execute the test cases and analyze the results
Aggregate these results to determine if a specific claim is justified.
15
What needs to be tested: testing for qualitative system attributes Validate performance claims:
Quantify the term acceptable performance (a number of transactions per second)
Create new data or use historical data to use in the tests.
Run the tests, collect results and timing data.
Make determination (pass or fail) for the claim.
16
What needs to be tested: testing the system deployment
Testing the system deployment: required for configurable systems and for applications that require dynamic interaction with the environment.
Deployment testing is intended to ensure that the packaging used for the system provides adequate setup and delivers a product in working condition.
17
What needs to be tested: testing the system deployment (cont...)
The initial test case if a full, complete installation. Interactions between options Certain options may not be installed Libraries or drivers may be needed for
other options
18
What needs to be tested: testing the system deployment (cont...)
Levels of installation: Typical Custom full
19
What needs to be tested: deployment testing technique
Identify categories of platforms on which the system will be deployed.
Locate at least one system of each type that has a typical environment but that has not had the system installed on it.
Install the system using the deployment mechanism.
Run a regression set of system tests and evaluate the results.
20
What needs to be tested: test system security Three categories of issues that could be
classified as security: The ability of the application to allow
authorized access and to prevent access by unauthorized persons.
The ability of the code to access all of the needed resources that it needs to execute.
The ability of the application to prevent unauthorized access to other system resources not related to the application.
21
What needs to be tested: test system security
Modularity of executables and the dynamic aspects of the code does raise security issues: Permissions for files deployed in a
directories: test one resource in each directory and one user from each security class.
22
Key points Testing the system deployment:
required for configurable systems and for applications that require dynamic interaction with the environment.