1.1 Aims of testing Testing is a process centered around the goal of finding defects in a system. We are currently unable to produce defect-free systems.

FundamentalsFundamentals

1.1 Aims of testingTesting is a process centered around the goal

of finding defects in a system.We are currently unable to produce defect-free

systems.Testing is an essential element in system

development – it helps to improve the quality of the system.

Every test process contains activities for planning what is needed, specifying what should be tested, and executing those test cases.

There is also a universal rule that it is impossible to find all defects and that there is never enough time (or personnel or money) to test everything.

Because some things are valid for every test process, a generic test approach can be defined providing the basic structured approach for organizing a wellcontrolled test process.

This information, preferably written, is the basis for deciding what to test and whether the result is acceptable or not.

Figure 1.1 illustrates how the above elements interact with the test process. The test process defines the necessary activities and organizes them in a lifecycle. For complex activities, specific techniques are developed that aid in performing them.

1.2 What is an embedded system?“Embedded system” is one of those terms

that do not really say what exactly it is about. All embedded systems have a common

feature in that they interact with the real physical world, controlling some specific hardware.

1.2 shows a generic layout, which is applicable to virtually all embedded systems,pointing out the typical components of an embedded system.

An embedded system interacts with the real world, receiving signals through sensors and sending output signals to actors that somehow manipulate the environment.

The environment of an embedded system, including the actors and sensors, is often referred to as the plant.

The embedded software of the system is stored in any kind of non-volatile memory (NVM).

the processing unit can only process digital signals

the environment possibly deals with analog signals

1.3 Approach to the testing of embedded systemsthere are also many similar problems, which

have similar solutions, that are involved in any test approach.

Some kind of basic test principles must apply to all embedded test projects – but somehow they must be differentiated with several specific measures to tackle the specific problems of testing particular systems.

The TEmb method

2.1 Overviewplanning the test project according a certain

lifecycle, applying standardized techniques, dedicated test environments, organizing test teams, formal reporting, etc.

They are related to the four cornerstones of structured testing: lifecycle, infrastructure, techniques, organization.(referred to as “LITO” in Figure 2.1)

This basis test approach is not yet concrete enough and needs to be enhanced with details such as: which design techniques will be applied? which tools and other infrastructure

components will be used and must, perhaps, be developed first? etc.

In the initial stages of the project, choices must be made of those specific measures that will be included in the test proach.

In the TEmb method this is called “the mechanism for assembling the dedicated test approach.”

2.2 TEmb genericLifecycle. This defines which activities have to be

performed and in what order. It gives testers and managers the desired control over the process.

Techniques. This helps with how to do things, by defining standardized ways to perform certain activities.

Infrastructure. This defines what is needed in the test environment to make it possible to perform the planned activities.

Organization. This defines the roles and required expertise of those who must perform the planned activities and the way they interact with the other disciplines.

LifecycleIn the lifecycle model, the principal test

activities are divided into five phases (see Figure 2.3). In addition to the phases of planning & control, preparation, specification, and execution, a completion phase has been defined to round off the test process satisfactorily and to formally deliver the testware to the line organization for subsequent maintenance releases.

TechniquesThis cornerstone supports the test process by

offering testers elaborate, proven,and universal working methods, as well as enabling the management (and auditors) to track the progress of the test process and evaluate the results. In principle, one or more techniques can be devised for every type of activity. New techniques are developed somewhere in the world almost every day. Whenever an activity exists which is likely to be repeated several times, the future test process would be supported by devising a specific echnique for that activity.

Infrastructurehe infrastructure for testing includes all the

facilities required for structured testing. It can be divided into facilities needed for executing the test (test nvironment), facilities that support efficient test execution (tools and test automation), and facilities for housing the staff (office environment).

OrganizationOrganization is all about people and how they

communicate. Testing is not a straightforward task that can be performed in isolation, unhindered by interferences from a demanding outside world. The involvement of many different disciplines, conflicting interests, unpredictability, shortage of expertise, and time constraints make the set-up and management of the test organization a difficult task.

2.3 Mechanism for assembling the dedicated test approachChapter 2.1 explained the principle of using

system characteristics to state what it is that makes the system special. It is important to note that the Temb method does not aim at achieving a scientifically accurate and complete taxonomy of embedded systems. Rather its purpose is entirely practical and purely from a tester’s perspective. It aims at assisting the test manager in answering the question “What makes this system special and what must be included in the test approach to tackle this?”

The following provides a useful initial set of system characteristics: safety critical systems technical-scientific algorithms autonomous systems unique system; “one-shot” development analog input and output (in general, mixed signals) hardware restrictions state-based behavior hard real-time behavior control systems extreme environmental conditions.