Principles of Software Testing[ZZ]

original article: http://www.the-software-experts.de/e_dta-sw-test-principles.htm

 

What is not Software Testing?

• Test is not debugging. Debugging has the goal to remove errors. The existence and the approximate location of the error are known. Debugging is not documented. There is no specification and there will be no record (log) or report. Debugging is the result of testing but never a substitution for it.
• Test can never find 100% of the included errors. There will be always a rest of remaining errors which can not be found. Each kind of test will find a different kind of errors.
• Test has the goal to find errors and not their reasons. Therefore the activity of testing will not include any bug fixing or implementation of functionality. The result of testing is a test report. A tester must not modify the code he is testing, by no means! This has to be done by the developer based on the test report he receives from the tester.

What is Software Testing?

• Test is a formal activity. It involves a strategy and a systematic approach. The different stages of tests supplement each other. Tests are always specified and recorded.
• Test is a planned activity. The workflow and the expected results are specified. Therefore the duration of the activities can be estimated. The point in time where tests are executed is defined.
• Test is the formal proof of software quality.

Overview of Test Methods

Static tests
The software is not executed but analyzed offline. In this category would be code inspections (e.g. Fagan inspections), Lint checks, cross reference checks, etc.
Dynamic tests
This requires the execution of the software or parts of the software (using stubs). It can be executed in the target system, an emulator or simulator. Within the dynamic tests the state of the art distinguishes between structural and functional tests.
Structural tests
These are so called "white-box tests" because they are performed with the knowledge of the source code details. Input interfaces are stimulated with the aim to run through certain predefined branches or paths in the software. The software is stressed with critical values at the boundaries of the input values or even with illegal input values. The behavior of the output interface is recorded and compared with the expected (predefined) values.
Functional tests
These are the so called "black-box" tests. The software is regarded as a unit with unknown content. Inputs are stimulated and the values at the output results are recorded and compared to the expected and specified values.

 

Test by progressive Stages

The various tests are able to find different kinds of errors. Therefore it is not enough to rely on one kind of test and completely neglect the other. E.g. white-box tests will be able to find coding errors. To detect the same coding error in the system test is very difficult. The system malfunction which may result from the coding error will not necessarily allow conclusions about the location of the coding error. Test therefore should be progressive and supplement each other in stages in order to find each kind of error with the appropriate method.
Module test
A module is the smallest compilable unit of source code. Often it is too small to allow functional tests (black-box tests). However it is the ideal candidate for white-box tests. These have to be first of all static tests (e.g. Lint and inspections) followed by dynamic tests to check boundaries, branches and paths. This will usually require the employment of stubs and special test tools.
Component test
This is the black-box test of modules or groups of modules which represent certain functionality. There are no rules about what can be called a component. It is just what the tester defined to be a component, however it should make sense and be a testable unit. Components can be step by step integrated to bigger components and tested as such.
Integration test
The software is step by step completed and tested by tests covering a collaboration of modules or classes. The integration depends on the kind of system. E.g. the steps could be to run the operating system first and gradually add one component after the other and check if the black-box tests still run (the test cases of course will increase with every added component). The integration is still done in the laboratory. It may be done using simulators or emulators. Input signals may be stimulated.
System test
This is a black-box test of the complete software in the target system. The environmental conditions have to be realistic (complete original hardware in the destination environment).

Which Test finds which Error?

Possible error	Can be best found by	Example
Syntax errors	Compiler, Lint	Missing semicolons, Values defined but not initalized or used, order of evaluation disregarded.
Data errors	Software inspection, module tests	Overflow of variables at calculation, usage of inappropriate data types, values not initialized, values loaded with wrong data or loaded at a wrong point in time, lifetime of pointers.
Algorithm and logical errors	Software inspection, module tests	Wrong program flow, use of wrong formulas and calculations.
Interface errors	Software inspection, module tests, component tests.	Overlapping ranges, range violation (min. and max. values not observed or limited), unexpected inputs, wrong sequence of input parameters.
Operating system errors, architecture and design errors	Design inspection, integration tests	Disturbances by OS interruptions or hardware interrupts, timing problems, lifetime and duration problems.
Integration errors	Integration tests, system tests	Resource problems (runtime, stack, registers, memory, etc.)
System errors	System tests	Wrong system behaviour, specification errors