gcov and coverage
A test suite proves that the code it runs is right; coverage measures how much of the code it runs at all. gfortran writes a counter for every line and every branch when it is asked, the program adds to those counters as it runs, and gcov turns them into a listing with a number beside each source line. This lesson builds a small program with one branch that never runs, reads the listing, and shows the trap in the counters.
Building with --coverage
The program classifies six numbers with an if/else-if chain. The tests are a loop over ten, twenty, ... sixty, so the first two arms of the chain have no input that reaches them.
The file cover.f90:
program cover
implicit none
integer :: i, n, total
total = 0
do i = 1, 6
n = 10 * i
total = total + classify(n)
end do
print '(a,i0)', 'total = ', total
contains
pure integer function classify(n) result(code)
integer, intent(in) :: n
if (n < 0) then
code = -1
else if (n == 0) then
code = 0
else if (n <= 30) then
code = 1
else
code = 2
end if
end function classify
end program cover
gfortran -Wall -Wextra -std=f2018 --coverage cover.f90 -o cover
./cover
ls -l *.gcda *.gcno
total = 9
232 cover.gcda
1861 cover.gcno
--coverage is the short form of -fprofile-arcs -ftest-coverage: the first writes the counters into the object and the second writes the table that maps them to lines. The .gcno file appears at compile time, the .gcda file when the program exits, and both take the name of the source file, not of the executable.
The gcov report
gcov -b cover.f90 reads the two files and writes a listing; -b adds the branch counts. Its summary is four lines, and the listing is a new file beside the source:
gcov -b cover.f90
File 'cover.f90'
Lines executed:86.67% of 15
Branches executed:100.00% of 8
Taken at least once:75.00% of 8
Calls executed:100.00% of 7
Creating 'cover.f90.gcov'
The listing is where the useful information is. Its first column is a count, the second a source line number, and the rest of the line is the source itself:
-: 0:Source:cover.f90
-: 0:Graph:cover.gcno
-: 0:Data:cover.gcda
-: 0:Runs:1
function MAIN__ called 1 returned 100% blocks executed 100%
1: 1:program cover
-: 2: implicit none
-: 3: integer :: i, n, total
-: 4:
1: 5: total = 0
7: 6: do i = 1, 6
branch 0 taken 86% (fallthrough)
branch 1 taken 14%
6: 7: n = 10 * i
7: 8: total = total + classify(n)
-: 9: end do
function main called 1 returned 100% blocks executed 100%
2: 10: print '(a,i0)', 'total = ', total
call 0 returned 100%
call 1 returned 100%
call 2 returned 100%
call 3 returned 100%
call 4 returned 100%
call 5 returned 100%
call 6 returned 100%
-: 11:contains
-: 12:
function classify.0 called 6 returned 100% blocks executed 78%
6: 13: pure integer function classify(n) result(code)
-: 14: integer, intent(in) :: n
-: 15:
6: 16: if (n < 0) then
branch 0 taken 0% (fallthrough)
branch 1 taken 100%
#####: 17: code = -1
6: 18: else if (n == 0) then
branch 0 taken 0% (fallthrough)
branch 1 taken 100%
#####: 19: code = 0
6: 20: else if (n <= 30) then
branch 0 taken 50% (fallthrough)
branch 1 taken 50%
3: 21: code = 1
-: 22: else
3: 23: code = 2
-: 24: end if
6: 25: end function classify
-: 26:
-: 27:end program cover
Reading the columns
Nine of the fifteen lines carry a number: the executable ones. A dash in the first column means the line has no code to count, such as a declaration, a blank line or contains. ##### means the line was never executed at all, and here there are two of them: code = -1 at line 17 and code = 0 at line 19, the two arms no test reaches.
The counts themselves are interesting. do i = 1, 6 reads 7 because the loop test runs once more than the body, which is how a counted loop ends. classify is called 6 times and its line 20, the third test, is reached 6 times with the two arms taken 3 times each. A branch line under an if counts the two ways the condition can go, and branch 0 taken 0% (fallthrough) is the arm that never happened.
The four summary numbers differ in what they count, and the percentages are worth reading in order. Lines executed is lines, Branches executed is branch points, Taken at least once is the branches where at least one direction was used, and Calls executed is calls. A hundred per cent of branches executed with seventy-five per cent taken at least once, as here, means that every branch was reached and some of them always went the same way: the code is exercised but not decided.
Counters accumulate
A .gcda file is not replaced by the next run, it is merged with it. Running the program a second time and calling gcov again gives:
./cover
gcov -b cover.f90
total = 9
File 'cover.f90'
Lines executed:86.67% of 15
Branches executed:100.00% of 8
Taken at least once:75.00% of 8
Calls executed:100.00% of 7
Creating 'cover.f90.gcov'
The percentage does not move, because the same lines ran again, but the counts do: the listing now says Runs: 2, the loop is 14 and the body 12. That is convenient for a suite that runs many inputs into one binary and misleading when the counters of a previous, different build are still on disk. Delete the .gcda files before a coverage run, and the numbers belong to that run alone:
rm -f *.gcda
./cover
gcov -b cover.f90
A test script does exactly that before every run, and compares the summary line with the one it expects, so a test that stopped exercising a branch is noticed as a change rather than hidden in an accumulated count.
What coverage does not say
Line coverage is a map of what ran, not a statement about what is right. The loop above could compute the wrong total with every line covered; the two dead arms are a hint that the tests never pass a negative number or a zero, and the fix is another case in the test, not a higher percentage. A branch that is 100 per cent covered with one direction missing is the more common shape of the same problem, and it is why the Taken at least once line is the one to watch.
gcovr, lcov and genhtml turn these listings into one HTML report for a whole project. None of them is installed in this image, so this page does not show their output; a project that wants them installs them on its own machine, and gcov stays the tool that produces the raw data.
Exercises
- Add a negative number to the loop and run the whole sequence again. Which line stops being #####, and what does the summary say?
- Build with --coverage -O0 and then with --coverage -O2. Are the counts the same? Which lines does the optimiser merge?
- Move the source into src/cover.f90 and compile with -o build/cover. Where do the .gcno and .gcda files go, and what does gcov -o build cover.f90 find?
- Delete only the .gcno file and run gcov. What is the message?
Questions
A new set of questions in a new order every time. Choose one answer for each, then check.
-
What does
--coveragestand for?Answer
-fprofile-arcs -ftest-coverage
Counters in the object and a table that maps them to lines.
-
When is the
.gcdafile written?Answer
When the program exits
The .gcno comes at compile time, the .gcda at exit.
-
What does
#####in the count column mean?Answer
The line was never executed
The two dead arms of the else-if chain.
-
Why does
do i = 1, 6show a count of 7?Answer
The loop test runs once more than the body
The final test is what ends the loop.
-
What does a branch line with
taken 0%say?Answer
That direction of the condition never happened
The arm is reached but never taken that way.
-
What happens to the counters on a second run of the same binary?
Answer
They are merged, so the counts double and Runs becomes 2
Delete the .gcda files for a clean run.
-
What does
Lines executed: 86.67% of 15count?Answer
Executable lines that ran at least once
Declarations and blank lines carry a dash.
-
Which summary line shows that every branch was reached but some never went both ways?
Answer
Branches executed 100% with Taken at least once 75%
Reached is not the same as decided.
-
What does a high line percentage prove?
Answer
That the lines ran, not that the results are right
Coverage is a map, not a proof.
-
Which of these is not in this image?
Answer
gcovr, lcov and genhtml
The page says so and shows no output of them.
The programs on this page were built with GNU Fortran (Alpine 15.2.0) 15.2.0 and run on Alpine Linux 3.24.3 (x86_64) in a Docker container, with gcov (Alpine 15.2.0) 15.2.0.
Article author: Arthur Isaev