Overview of ABC SCAT

From Deos
Jump to navigationJump to search

Overview of ABC/SCAT

The ABC/SCAT tooling supports structural coverage analysis on the Deos™ platform, and certain other platforms.

The tooling is qualifiable as a verification tool (in the sense of DO178-B). Users of the tool must qualify it for their program. (A "tool qualification kit" is provided.)

Tools

The tooling in the ABC/SCAT toolsuite consists of:

  • An instrumentor that instruments the intermediate assembly-language files with code probes at branches and labels. (You must modify your makefiles to invoke the instrumentor during compilation.)
  • A feature now standard in hypstart.py that allows you to designate a buffer in RAM for a hitmap.
  • A means to retrieve the hitmap from a Deos target back to your test workstation. (This is either an applet which you can ftp to a target; or, in newer versions of Deos, use of a capability of the ftp server itself.)
  • A report generator which interprets the hitmap for you, displaying the executed code with execution coverage marked and highlighted.

Output

ABC/SCAT's output consists of:

  • A coverage summary which summarizes the statistics about each source file and rolls the data up for the entire software under test.
  • A coverage notes file which provides a place to record per-miss analyses.
  • For each C/C++/assembly source file, an assembly-language file annotated with the original source code, and with the executable assembly-language instructions marked to indicate whether they were executed and how branches were take/not-taken.

All of these files can be generated as HTML or ASCII text. In HTML form, the summary page lets you jump directly into the annotated files, either file by file or miss by miss.

Benefits of ABC/SCAT

  • Produces "Assembly Branch Coverage" which is equivalent to MC/DC when certain conditions are met.
  • Works both on code which executes as a Deos application (a process running under Deos), and on code which executes before Deos is all the way up (eg, boot code, PAL code, drivers, or the Deos kernel itself).
  • Works on code on certain non-Deos platforms. (Currently TI TMS C67x and Blackfin BF53x are supported.)
  • Generates compact coverage output (a hitmap) and does not rely on availability of a mechanism to continuously stream coverage data off of the target.
  • Works on C++, C, and assembly-language code.
  • Works on any hardware platform running x86 or PPC Deos. (But see below under versions.)

Limitations of ABC/SCAT

  • Works only on Deos and on particular supported non-Deos platforms.
  • Is toolchain-specific. On Deos, ABC/SCAT only works with the GNU gcc tools for x86 or PowerPC (an earlier version for x86 works with the Borland tools). On the Blackfin BF53x, it works with Visual DSP++, and on the TI C67x, it works with Code Composer Studio. And ABC/SCAT has only been tested with particular versions of those toolsets. See below under versions.
  • Perturbs both your build process and your test process. During the coverage step you must change the way you compile files: the instrumentor must be run "during compilation". During the coverage step you must also change the way you execute tests on your target, to set up a hitmap buffer and retrieve it for analysis.
  • Directly performs only ABC coverage analysis, which is equivalent to MC/DC coverage if certain coding conventions are followed and certain compiler tests are performed.
  • Requires several pages of RAM for a hitmap on the test target.
  • Insertion of code probes causes an increase in code size. (However, the increase is small enough that the entire Deos kernel can be instrumented in one batch and run.)
  • Multiple test runs are typically required for certification: (1) one run with instrumentation inserted, debugging turned on, and optimization disabled; (2) one run with instrumentation inserted, debugging turned off, and optimization enabled; (3) one run without instrumentation, with debugging off and optimization on.
  • The ABC/SCAT tooling does not yet provide automated assistance with C++ vtables.

Tool qualification

All versions of ABC/SCAT (excluding the very earliest one) are "user qualifiable". That means that the program/project using the tool is expected to do any required DO-178B tool qualification in their own environment. This ensures that the tool behaves as required in the environment where the program will actually be using it.

To make this tool qualification activity relatively easy, ABC/SCAT releases include a "qualification kit" consisting of a suite of tests, expected output from the tests, a script to automatically run the testsuite and check for problems, and an example Tool Qualification Data document. The program/project using the tool will run the testsuite (via the script), and customize the tool qualification data document for their program.

ABC/SCAT versions

Which version of ABC/SCAT to use depends on several factors. One is tool chain. The earliest versions (before September 2003) did not support gcc; the latest versions (from 4.1.0 on) do not support Borland (since no one has paid for that). The earlier versions were developed for earlier versions of Deos. (Generally you can use them with later versions, but you'll need to make tweaks to the included registries and makefiles.) Also, in the later versions, features have been added, bugs fixed, and various things made easier.

The latest versions of ABC/SCAT are included in the Deos Software Development kit, the DESK, available via the Cygwin installer.

On the SCAT product page there is a link to the release notes (which go all the way back to the first version), as well as a link to the archive page from which you can fetch older versions. For recent versions of ABC/SCAT, the release notes contain detailed information about which versions of toolchains are supported. The most detailed information about which toolchains and Deos versions are supported by the older versions are on the archive page.

Retargeting ABC/SCAT

It’s possible to retarget ABC/SCAT to a different architecture and toolchain. Note that this is not a small job. Since ABC/SCAT does most of its work at the assembly language level, all the tools and all the qualification tests are very sensitive to the assembly language (thus to the processor) and the assembly language's format (thus to the compiler/assembler toolchain).

Retrieved from "https://ddci.zapto.org/Wiki/index.php?title=Overview_of_ABC_SCAT&oldid=10745"