Coverage with gcov
Coverage answers the question a test suite cannot answer about itself: which lines of the program did the tests never run. The image has no COBOL coverage tool, and it does not need one, because cobc compiles COBOL to C and gcc can count the lines of the C. The counts are mapped back to the COBOL source, so the report is in terms of the program a person wrote.
How coverage works here
Three steps, and the page shows all of them. cobc --coverage instruments the generated binary and leaves .gcno files beside it; the run writes .gcda files; gcov reads the pair and writes a .gcov file per source, with an execution count in front of every line.
The harness with two cases
The file is the harness of the first page cut down to two cases, A and B, with no unknown code. The unit is the second program in the same file, and only two cases exist:
IDENTIFICATION DIVISION.
PROGRAM-ID. COVHARNESS.
DATA DIVISION.
WORKING-STORAGE SECTION.
01 WS-CASES.
05 WS-CASE OCCURS 2 TIMES.
10 C-CODE PIC X(1).
10 C-AMOUNT PIC 9(7)V99.
10 C-STATUS PIC 9(2).
10 C-DISCOUNT PIC 9(7)V99.
01 WS-I PIC 9(2).
01 WS-PASSED PIC 9(3) VALUE 0.
01 WS-FAILED PIC 9(3) VALUE 0.
01 WS-OUT PIC Z(6)9.99.
01 DISCOUNT-REQUEST.
05 DR-CODE PIC X(1).
05 DR-AMOUNT PIC 9(7)V99.
01 DISCOUNT-ANSWER.
05 DA-STATUS PIC 9(2).
05 DA-DISCOUNT PIC 9(7)V99.
05 DA-MESSAGE PIC X(30).
PROCEDURE DIVISION.
MAIN-PARA.
PERFORM FILL-CASES
PERFORM VARYING WS-I FROM 1 BY 1 UNTIL WS-I > 2
PERFORM RUN-CASE
END-PERFORM
DISPLAY "passed " WS-PASSED " failed " WS-FAILED
IF WS-FAILED IS GREATER THAN 0
MOVE 1 TO RETURN-CODE
END-IF
STOP RUN.
FILL-CASES.
MOVE "A" TO C-CODE(1)
MOVE 100.00 TO C-AMOUNT(1)
MOVE 0 TO C-STATUS(1)
MOVE 10.00 TO C-DISCOUNT(1)
MOVE "B" TO C-CODE(2)
MOVE 100.00 TO C-AMOUNT(2)
MOVE 0 TO C-STATUS(2)
MOVE 5.00 TO C-DISCOUNT(2).
RUN-CASE.
MOVE C-CODE(WS-I) TO DR-CODE
MOVE C-AMOUNT(WS-I) TO DR-AMOUNT
CALL "DISCOUNT" USING DISCOUNT-REQUEST DISCOUNT-ANSWER
IF DA-STATUS NOT = C-STATUS(WS-I)
PERFORM FAIL-CASE
ELSE
IF DA-DISCOUNT NOT = C-DISCOUNT(WS-I)
PERFORM FAIL-CASE
ELSE
ADD 1 TO WS-PASSED
DISPLAY "pass case " WS-I
END-IF
END-IF.
FAIL-CASE.
ADD 1 TO WS-FAILED
MOVE DA-DISCOUNT TO WS-OUT
DISPLAY "FAIL case " WS-I " status " DA-STATUS
" discount " FUNCTION TRIM(WS-OUT)
" expected " C-STATUS(WS-I) " " C-DISCOUNT(WS-I).
IDENTIFICATION DIVISION.
PROGRAM-ID. DISCOUNT.
DATA DIVISION.
WORKING-STORAGE SECTION.
01 RATE-TABLE.
05 RATE-ENTRY PIC 9V99 OCCURS 2 TIMES.
01 WS-INDEX PIC 9.
01 WS-RATE PIC 9V99.
LINKAGE SECTION.
01 DISCOUNT-REQUEST.
05 DR-CODE PIC X(1).
05 DR-AMOUNT PIC 9(7)V99.
01 DISCOUNT-ANSWER.
05 DA-STATUS PIC 9(2).
05 DA-DISCOUNT PIC 9(7)V99.
05 DA-MESSAGE PIC X(30).
PROCEDURE DIVISION USING DISCOUNT-REQUEST DISCOUNT-ANSWER.
MOVE 0 TO DA-STATUS
MOVE 0 TO DA-DISCOUNT
MOVE SPACES TO DA-MESSAGE
MOVE 0.10 TO RATE-ENTRY(1)
MOVE 0.05 TO RATE-ENTRY(2)
EVALUATE DR-CODE
WHEN "A"
MOVE 1 TO WS-INDEX
WHEN "B"
MOVE 2 TO WS-INDEX
WHEN OTHER
MOVE 9 TO DA-STATUS
MOVE "unknown discount code" TO DA-MESSAGE
GOBACK
END-EVALUATE
MOVE RATE-ENTRY(WS-INDEX) TO WS-RATE
COMPUTE DA-DISCOUNT ROUNDED = DR-AMOUNT * WS-RATE
GOBACK.
END PROGRAM DISCOUNT.
The build, the run and gcov
cobc -x -free --coverage -o harness_cov harness_cov.cob
./harness_cov
echo $?
gcov harness_cov.gcno
pass case 01
pass case 02
passed 002 failed 000
0
File '/tmp/cob671_0.c'
Cannot open source file /tmp/cob671_0.c
Cannot open source file /tmp/cob671_0.c.l2.h
Cannot open source file /tmp/cob671_0.c.h
Lines executed:80.54% of 298
Creating 'cob671_0.c.gcov'
File '/tmp/cob671_0.c.l2.h'
Lines executed:100.00% of 2
Creating 'cob671_0.c.l2.h.gcov'
File 'harness_cov.cob'
Lines executed:83.33% of 60
Creating 'harness_cov.cob.gcov'
File '/tmp/cob671_0.c.h'
Lines executed:77.78% of 9
Creating 'cob671_0.c.h.gcov'
Lines executed:81.03% of 369
The three Cannot open source file lines are the honest part of the output: the C file that cobc wrote lives in /tmp under a name that is deleted when the compiler exits, so gcov can count its lines but cannot show them. The COBOL source is there, and that is the file a reader wants: 83.33% of 60 lines.
Reading the .gcov file
The file written by gcov is the COBOL source with three prefixes: a count, a dash for a line that is not executable, or five hashes for a line that never ran. The end of the unit inside the file, from the .gcov of the run above:
2*: 88: MOVE 0 TO DA-STATUS
2*: 89: MOVE 0 TO DA-DISCOUNT
2*: 90: MOVE SPACES TO DA-MESSAGE
2*: 91: MOVE 0.10 TO RATE-ENTRY(1)
2*: 92: MOVE 0.05 TO RATE-ENTRY(2)
2*: 93: EVALUATE DR-CODE
2*: 94: WHEN "A"
1*: 95: MOVE 1 TO WS-INDEX
1*: 96: WHEN "B"
1*: 97: MOVE 2 TO WS-INDEX
-: 98: WHEN OTHER
#####: 99: MOVE 9 TO DA-STATUS
#####: 100: MOVE "unknown discount code" TO DA-MESSAGE
#####: 101: GOBACK
Three things are visible at once. The two calls each executed the setup lines, so they carry a 2. The WHEN lines are counted as executed even though only one of them ran, which is how gcc counts a line with a condition on it. And the three lines of the WHEN OTHER branch carry #####, because no case in this harness ever sends a code that is neither A nor B.
That is the value of the page: the branch is not a line the author forgot, it is a line the tests never reached, and the report says which one it is. The failure path of the harness itself is uncovered for the same reason, and it is the harness code, not the unit, so it matters less.
The cases the harness page adds
The first page of the group builds the same file with four cases: the two here, the ten percent of two hundred and fifty, and an unknown code X. Measured the same way, its listing under --coverage gives:
File 'harness.cob'
Lines executed:89.71% of 68
Creating 'harness.cob.gcov'
The three lines of the unknown-code branch are covered now. What remains uncovered in that file is the harness\'s own failure path: the counter, the FAIL line and the status, which only a failing case executes. Coverage of a test harness is not the point of measuring, and it is worth knowing that the number includes it.
What a line count does not say
A line count is not a correctness count, and the pages of this group show why in three different ways. Line coverage says nothing about the values: a test that sends one amount exercises the multiplication and proves nothing about rounding. It says nothing about the unchecked write past a table on the runtime checks page, where every line ran and memory was still corrupted. And it says nothing about the conditions inside a line, because a line with an IF counts once whether the branch was taken or not.
What it does say is cheap and exact: these lines never ran, so no test of this program\'s behaviour has ever depended on them. A branch that never runs is a branch whose first real execution is in production, and the number on the report is the shortest argument for adding the case.
The last page of the group is the one about the tools this image does not have: what cobol-check and zUnit are, how a project would install them, and why a page about them has no output to show.
The code on this page was built with cobc (GnuCOBOL) 3.2.0, run, and measured with gcov on Alpine Linux 3.24.3 (x86_64) in a Docker container.
Article author: Arthur Isaev