Damian Gordon. Static Testing is the testing of a component or system at a specification or implementation level without execution of the software.

Software Testing 4Damian Gordon

Static Testing is the testing of a component or system at a specification or implementation level without execution of the software.

Dynamic Testing is the testing of software by executing the software of either a component or system.

Static Testing and Dynamic Testing are complementary methods and tend to find different types of defects effectively and efficiently.

Test Approaches

Static Testing detects defects such as deviations from standards, missing requirements, design defects, non-maintainable code and inconsistent interface specifications.

Dynamic Testing detects defects such as checking if outputs from the expected values.

Test Approaches

One key approach in Static Testing is the Review Process.

Reviews can find defects, are informational, communicational, and educational.

Participants in the review learn the content of the software systems, the role of their own work, help planning for future stages of the work.

Reviews often represent milestones, and support the establishments of a baseline for the software product.

Test Approaches

The Review Process

The type and quantity of defects found during the review stage can help focus the testing process.

In some cases customers or users attend the review process and provide feedback to the developers and document authors.

Studies have shown the reviews significantly increase productivity and product quality.

Test Approaches

They can be either informal or formal. The formality of the review process is

related to factors such as the maturity of the development process, any legal or regulatory requirements or the need for an audit trail.

In practice most reviews are informal. A two-person team can conduct an

informal review, as the developer/author can get a colleague to review the code and documentation.

The Review Process

Phases of a Formal Review1. Planning2. Kick-Off3. Preparation4. Review Meeting5. Rework6. Follow-up

The Review Process

1. Planning The authors/developers request a review. A moderator is assigned – this is the leader

of the review process. The project planning needs to incorporate

time to undertake the review. The planning must start with deciding an

entry criteria to ensure that document is ready for review.

The Review Process

1. Planning Entry Criteria The set of generic and specific conditions

for permitting a process to go forward with a defined task, e.g. Test phase. The purpose of entry criteria is to prevent a task from starting which would entail more (wasted) effort compared to the effort needed to remove the failed entry criteria.

The Review Process

1. Planning Minimum set of criteria for performing entry check:

◦ A short check of a product sample by the moderator does not reveal a large number of defects, e.g. After 30 minutes of checking, no more than 3 major defects are found in a single page or fewer than 10 major defects in total in a set of 5 pages.

◦ The document to be reviewed is available with line numbers.

◦ The document has been cleaned up by running any automated checks that apply.

◦ References needed for the inspection are stable and available.

◦ The document author is prepared to join the review team and feels confident with the quality of the document.

The Review Process

1. Planning The review will focus on a few different

things:◦ Focus on higher-level documents, e.g. Does the

design comply to the requirements◦ Focus on standards, e.g. Internal consistency,

clarity, naming conventions, templates◦ Focus on related documents at the same level,

e.g. Interfaces between software functions◦ Focus on usage, e.g. For testability and

maintainablity

The Review Process

2. Kick-Off The review starts with a kick-off meeting, to

make sure everyone is on the same wavelength regarding the document under review.

The meeting consists of a short introduction to the objectives of the review and the documents.

Role assignments, checking rate, the pages to be checked, process changes and possible other questions are discussed at this meeting.

The Review Process

3. Preparation Participants identify defects, questions,

and comments, according to their understanding of the document and their role.

A checking rate is decided, which is the number of pages checked per hour, usually about 5 to 10 pages per hour, depending on complexity.

The Review Process

4. Review Meeting Usually made up of the following phases:

◦ Logging phase◦ Discussion phase◦ Decision phase

The Review Process

4. Review Meeting Logging Phase: The issues that have been identified in the

Preparation stage are logged. To ensure progress and efficiency, no real

discussion is allowed during the logging phase.

Each defect is logged with a severity:◦ Critical: defects will cause downstream damage.◦ Major: defects could cause downstream damage.◦ Minor: defects are not likely to cause downstream

damage.

The Review Process

4. Review Meeting Discussion Phase: Each of the defects that require discussion

are discussed, with a chairman preventing discussions from getting too personal.

The Review Process

4. Review Meeting Decision Phase: At the end of the discussion phase, a

decision is taken about the document under review. If the number of defects found per page exceeds a certain level, the document may need to be reworked, and reviewed again.

The Review Process

5. Rework Based on the defects detects, the author

will improve the document under review, step-by-step.

The Review Process

The Review Process

Always, every, all, none, never, … (absolutely sure?)

Certainly, therefore, clearly, obviously, customarily, most, … (persuasion lingo)

Some, sometimes, often, usually, ordinarily, customarily, most, … (vague)

etc., and so forth, and so on, such as, … (not testable)

Good, fast, cheap, efficient, small, stable, … (unquantifiable)

Handled, processed, rejected, skipped, eliminated… If … then … (missing else)

Specification Terminology Checklist

Roles and Responsibilities The moderator The author The scribe The reviewers The manager

The Review Process

Roles and Responsibilities The moderator serves as the review

leader, they determine the type of review, approach and the composition of the review team.

The moderator performs the entry check, and the follow-up on the rework.

The moderator also schedules meetings, disseminates documents, leads discussions and stores relevant data.

The Review Process

Roles and Responsibilities The author writes the original document

and seeks to improve the quality of the document by working with others.

The Review Process

Roles and Responsibilities The scribe records all of the defects

during the logging meetings.

The Review Process

Roles and Responsibilities The reviewers (also called checkers and

inspectors) check the documents for defects.

Reviewers are chosen to represent different perspectives in the review.

The Review Process

Roles and Responsibilities The manager decides on the execution of

reviews and determines whether review process objectives have been met.

The Review Process

Types of Reviews Walkthrough Technical Review Inspection

The Review Process

Types of Reviews Walkthrough The author guides the review team through

the document, to achieve a common understanding and gather feedback.

This means the author does a range of preparation, and the review team don’t need to do a detailed study before the meeting.

A walkthrough is especially useful for higher-level documents, like requirements specifications and architectural documents.

The Review Process

Types of Reviews Technical Review This approach focuses on developing a

consensus about the technical content of the document.

Defects are found by technical experts, who focus on the content of the document (as opposed to considering any relevant legislation and standards, referenced documents, and intended readership).

The Review Process

Types of Reviews Inspection This approach is the most formal review type. The document is inspected thoroughly be the

reviews before the meeting, comparing the work product with its sources and other referenced documents, and using rules and checklists.

The defects found are logged and any discussion is postponed until the discussion phase.

This makes the inspection meeting a very efficient meeting.

The Review Process

Let’s do one.

The Review Process