DARTT Automated Test Procedure

Following the general concept of automation at BSSE, the actual test procedure consists of only very few manual steps. User interaction only occurs in

1. providing the source files,
2. starting the test system,
3. waiting for the test to complete and
4. analysing the report document.

Test input generation

DARTT automatically generates test inputs for all Ada- and user-defined-types, including fault-injection. Test vectors are recorded in ASCII and in spreadsheet format (e.g., for import in Microsoft® Excel™).

Exception recording and categorisation

Exceptions occurring during the test are automatically recorded and categorised according to their possible reason. The code from which the exception originated is analysed and listed together with the resulting category in the exception list. This has proven very convenient in in-house use in order to get a quick overview over the state of the application under test and the type of problems occurring. The exception list is provided sorted by type, category and filename.

Coverage analysis and data monitoring

The code under test is automatically instrumented for coverage analysis and data monitoring for function parameters, static and stack data. During test the range of values assigned to variables can be recorded (minimum, maximum).

Supported code coverage metrics are:

• entry points, call returns, statements, basic blocks (extended statement coverage)
• Decision Coverage (DC)
• Condition/Decision Coverage (C/DC)
• Modifie Condition/Decision Coverage (MC/DC, DO-178B)

The test report

The test report contains:

• Recorded execution time per function
• Recorded exceptions: file, function, line of raising statement,
• Results of data monitoring (minimum, maximum)
• Coverage analysis:
  • block coverage
  • MC/DC coverage
• Source Code analysis:
  • subprograms
  • types
  • code size (source, comment and blank lines)