The test build: -fcheck, -ffpe-trap, valgrind

The three pages before this one test the numbers a program produces. This page tests the program: a source with three bugs of the kind a suite of assertions cannot see, because each of them produces a plausible value and a status of zero. Every bug is found by a different tool, and each tool is a flag or a command rather than a library.

One program, three bugs

The program takes a number and runs one of three cases. The first sums an array with a loop bound one too large, the second divides one by a zero that comes from a variable so that the compiler cannot fold it, and the third allocates a pointer array and never deallocates it. Built the ordinary way and run, all three print a number and exit 0.

program bugs use, intrinsic :: iso_fortran_env, only: dp => real64 implicit none integer :: which, i, n, zero real(dp) :: total, x, y real(dp), pointer :: big(:) real(dp) :: a(5) character(len=8) :: arg call get_command_argument(1, arg) read (arg, *) which select case (which) case (1) a = [1.0_dp, 2.0_dp, 3.0_dp, 4.0_dp, 5.0_dp] n = 5 n = n + 1 total = 0.0_dp do i = 1, n total = total + a(i) end do print '(a,f10.2)', 'total = ', total case (2) zero = 0 y = real(zero, dp) x = 1.0_dp / y print '(a,es12.3)', 'x = ', x print '(a,es12.3)', 'x - x = ', x - x case (3) allocate (big(4000)) big = 1.0_dp print '(a,f8.1)', 'sum = ', sum(big) end select end program bugs

gfortran -Wall -Wextra -std=f2018 bugs.f90 -o bugs
./bugs 1
./bugs 2
./bugs 3

total =      15.00
x =     Infinity
x - x =         NaN
sum =    4000.0

Three wrong answers that look like right ones. The total of one to five is fifteen, and the sixth element happened to be zero, so even the first case is right by luck. The infinity and the NaN are values the standard defines, not errors. And a program that runs once and exits cannot show a leak in its output at all.

The bad subscript

-fcheck=all adds a test in front of every subscript and stops at the first one out of range. The build needs -g for the frame table, though the check itself names the line:

gfortran -Wall -Wextra -std=f2018 -g -fcheck=all bugs.f90 -o bugs_check
GFORTRAN_ERROR_BACKTRACE=0 ./bugs_check 1
echo $?

At line 21 of file bugs.f90
Fortran runtime error: Index '6' of dimension 1 of array 'a' above upper bound of 5
2

The message names the index, the dimension and the bound, and the status is 2. Lesson 114 is the long version of this flag, including the checks it also enables and the bugs it cannot see.

The division by zero

-fcheck does not look at floating-point exceptions, because a division by zero is a defined operation in IEEE arithmetic. -ffpe-trap=zero turns it into a signal:

gfortran -Wall -Wextra -std=f2018 -g -ffpe-trap=zero,invalid,overflow bugs.f90 -o bugs_trap
./bugs_trap 2


Program received signal SIGFPE: Floating-point exception - erroneous arithmetic operation.

Backtrace for this error:
#0  0x7850fc279430 in ???
#1  0x7850fc278345 in ???
#2  0x7159c39aa88a in ???
  at src/signal/x86_64/restore.s:1
#3  0x5591e13cf516 in bugs
  at /tmp/t4/bugs.f90:27
#4  0x5591e13cf8ef in main
  at /tmp/t4/bugs.f90:35

The status is 136, which is 128 plus the number of SIGFPE. The line is the division at 27, and the address list is the runtime's signal handler: GFORTRAN_ERROR_BACKTRACE=0 silences the trace of a runtime error but not the one a signal prints, so the session above shows the addresses and the three lines after them are the ones that matter.

The leak

The third case allocates thirty-two kilobytes through a pointer and returns without freeing it. No flag of the compiler can see that, because the program is correct as far as the language is concerned; valgrind sees it because it watches the allocator:

gfortran -Wall -Wextra -std=f2018 -g bugs.f90 -o bugs_vg
valgrind --leak-check=full ./bugs_vg 3

==932== HEAP SUMMARY:
==932==     in use at exit: 32,000 bytes in 1 blocks
==932==   total heap usage: 28 allocs, 27 frees, 43,977 bytes allocated
==932==
==932== 32,000 bytes in 1 blocks are definitely lost in loss record 1 of 1
==932==    at 0x48B7733: malloc (vg_replace_malloc.c:446)
==932==    by 0x40016CA: MAIN__ (bugs.f90:31)
==932==   by 0x40018D5: main (bugs.f90:35)
==932==
==932== LEAK SUMMARY:
==932==    definitely lost: 32,000 bytes in 1 blocks
==932==    indirectly lost: 0 bytes in 0 blocks
==932==      possibly lost: 0 bytes in 0 blocks
==932==    still reachable: 0 bytes in 0 blocks
==932==         suppressed: 0 bytes in 0 blocks
==932==
==932== For lists of detected and suppressed errors, rerun with: -s
==932== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)

The record names the ALLOCATE at line 31 and the size of the block, as lesson 116 shows in more detail. --error-exitcode=1 turns the report into a status a runner can read.

All the flags at once

A test build carries every check at the same time. The command below adds the traps and -finit-real=snan to the checks, and runs the three cases:

gfortran -Wall -Wextra -std=f2018 -g -fcheck=all -ffpe-trap=invalid,zero,overflow -finit-real=snan bugs.f90 -o bugs_test
./bugs_test 1
./bugs_test 2
./bugs_test 3

At line 21 of file bugs.f90
Fortran runtime error: Index '6' of dimension 1 of array 'a' above upper bound of 5
Program received signal SIGFPE: Floating-point exception - erroneous arithmetic operation.
sum =    4000.0

The first two cases stop, at the subscript and at the division. The third runs to the end and prints a correct sum: the combined build cannot see the leak either, and that is the division of labour between the compiler flags and valgrind. A test suite that runs the checked build finds the first two classes of bug in the ordinary course of a test run; the leak needs the separate command, which is why it belongs in a nightly job.

The test target

A Makefile target that does all of it is four lines. The first builds the checked binary, the second runs the suite, the third runs it under valgrind and the fourth keeps the status:

check: tests ./run_tests valgrind -q --leak-check=full --error-exitcode=1 ./run_tests > /dev/null ./coverage-gate.sh tests: tests.f90 asserts.f90 gfortran -Wall -Wextra -std=f2018 -g -fcheck=all -ffpe-trap=invalid,zero,overflow \ -finit-real=snan asserts.f90 tests.f90 -o run_tests

The \ at the end of the compile line is a line continuation for make, and the recipe lines are indented with spaces here where a Makefile needs a tab. The order is deliberate: the suite first, because it is fast and it says which check failed, and valgrind second, because it is slow and it only says that something is wrong. The coverage gate is last because it is a statement about the tests, not about the program.

The release build is the same source without any of these flags, and the two exist for different questions: the test build answers "is the program wrong", and the release build answers "how fast is it when it is right". Lesson 121 has the measurements that show why the flags cannot be left on in a release.

Exercises

  1. Add a fourth case with an uninitialised accumulator and run it under the combined build. Which flag finds it, and at which line?
  2. Run ./bugs_check 3 and valgrind ./bugs_check 3. Which of the two finds the leak, and does the checked build change anything for valgrind?
  3. Build the program with -O2 and all the checks. Does the bad subscript still stop at line 21?
  4. Put the leaks behind a flag so that ./bugs_test 3 also frees its memory, and run valgrind again. What is the summary line now?

The programs on this page were built with GNU Fortran (Alpine 15.2.0) 15.2.0 and run on Alpine Linux 3.24.3 (x86_64) in a Docker container, with valgrind 3.25.1.

Article author: Arthur Isaev