11.10.2004 Software Engineering 2004 Jyrki Nummenmaa 1 BACKGROUND • There is no way to generally test programs exhaustively (that is, going through all execution paths with all possible values). • Therefore testing can not guarantee correctness. • In spite of this, testing is a very important way to improve quality of programs and to eliminate errors.
23
Embed
11.10.2004Software Engineering 2004 Jyrki Nummenmaa 1 BACKGROUND There is no way to generally test programs exhaustively (that is, going through all execution.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
• OO software can be designed and built in many ways and using different kinds of modeling and documentation techniques.
• Testing should reflect these techniques and models (if, for instance, you use use cases in analysis and design, then you should use them as input for testing as well).
• A single software module (e.g. a class in OO programming) is tested.
• The modules often need other modules, and these modules may not have been implemented yet.
• Therefore, module testing in general needs a test environment, which potentially has some ”dummy implementations” of other modules – they could, for instance, just return a constant value, etc.
• Validating the implemented system against the requirements.
• At this point, use cases should be executed. • If use cases are described using sequence
diagrams or activity diagrams, it would be good to track down the executions and compare them to the diagrams (unfortunately there is not much tool support…)
• You as a software developer should initially check if you can run through the use cases.
• Testing takes a lot of time and effort. Typically a large number of tests need to run several times on software.
• Therefore, there is a need to automate testing:Make a computer program to do the following: execute a predefined test set, compare the results with expected results, and reported the deviations.
• This typically works better in module testing.• However, even GUI testing automation systems
now exist, and they provide a massive speedup in testing as compared to human testers.
• Same tests are run over a number of times and their results are compared.
• Clearly, we want these results to be comparable.• It follows that we would want the same test to give
the same result every time, if it is successful (and preferably the same result every time, when the same error is present).
• This puts limitations e.g. to randomization, where the same seed should be used every time, and extra care needs to be taken when human interaction is included in testing.
• Two ways to think about this:- How well do the tests cover the required functionalities of the application?- How well do the tests cover the implementation?
• Q: What’s the difference between these two?• A:
- The implementation usually includes things that are not required. - Sometimes some required functionality is completely missing.
• if ((a<0) && (b<a)) {… }else if ((a>=0) && (b<a)) {…} else if (b<a) { dead code }
• Sometimes dead code is intentionally written to manage ”impossible situations, just in case somebody changes the code to make the possible” or ”error management for errors that should be impossible to take place”.
• Some testing tools do bookkeeping on the lines of code that are visited and calculate a percentage of visited lines over all lines.
• Because of dead code, a 100% code coverage may not be possible.
• Note that a 100% code coverage does not mean ”complete testing”, it just means that the code on each line has been executed at least once, but minimally a line has been executed with only one combination of variable and parameter values.
• Sometimes it is even required that the software has been tested by an external testing authority (e.g. Nokia Forum requires this for programs that are traded through them)
• There are now companies, which concentrate primarily on testing.
• Testing, of course, requires a lot of programming these days.
• Sometimes the testing is even taken into another country.