A test harness in COBOL
A COBOL unit test is a program that calls another program with values it chose and compares what comes back with values it wrote down. There is no test runner in the language and none in this image, so the runner is the program: a table of cases, a loop, a count and a status. This page builds that harness for a small discount subprogram and shows it passing and failing.
One file, two programs
The file holds two programs. The first is the harness, which is the main program because it comes first; the second is the unit under test, a subprogram with a USAGE list that the harness calls by name. COBOL allows both in one source file, and a single cobc -x builds them into one executable:
IDENTIFICATION DIVISION.
PROGRAM-ID. DISCHARNESS.
DATA DIVISION.
WORKING-STORAGE SECTION.
01 WS-CASES.
05 WS-CASE OCCURS 4 TIMES.
10 C-CODE PIC X(1).
10 C-AMOUNT PIC 9(7)V99.
10 C-STATUS PIC 9(2).
10 C-DISCOUNT PIC 9(7)V99.
01 WS-I PIC 9(2).
01 WS-PASSED PIC 9(3) VALUE 0.
01 WS-FAILED PIC 9(3) VALUE 0.
01 WS-OUT PIC Z(6)9.99.
01 DISCOUNT-REQUEST.
05 DR-CODE PIC X(1).
05 DR-AMOUNT PIC 9(7)V99.
01 DISCOUNT-ANSWER.
05 DA-STATUS PIC 9(2).
05 DA-DISCOUNT PIC 9(7)V99.
05 DA-MESSAGE PIC X(30).
PROCEDURE DIVISION.
MAIN-PARA.
PERFORM FILL-CASES
PERFORM VARYING WS-I FROM 1 BY 1 UNTIL WS-I > 4
PERFORM RUN-CASE
END-PERFORM
DISPLAY "passed " WS-PASSED " failed " WS-FAILED
IF WS-FAILED IS GREATER THAN 0
MOVE 1 TO RETURN-CODE
END-IF
STOP RUN.
FILL-CASES.
MOVE "A" TO C-CODE(1)
MOVE 100.00 TO C-AMOUNT(1)
MOVE 0 TO C-STATUS(1)
MOVE 10.00 TO C-DISCOUNT(1)
MOVE "A" TO C-CODE(2)
MOVE 250.00 TO C-AMOUNT(2)
MOVE 0 TO C-STATUS(2)
MOVE 25.00 TO C-DISCOUNT(2)
MOVE "B" TO C-CODE(3)
MOVE 100.00 TO C-AMOUNT(3)
MOVE 0 TO C-STATUS(3)
MOVE 5.00 TO C-DISCOUNT(3)
MOVE "X" TO C-CODE(4)
MOVE 100.00 TO C-AMOUNT(4)
MOVE 9 TO C-STATUS(4)
MOVE 0 TO C-DISCOUNT(4).
RUN-CASE.
MOVE C-CODE(WS-I) TO DR-CODE
MOVE C-AMOUNT(WS-I) TO DR-AMOUNT
CALL "DISCOUNT" USING DISCOUNT-REQUEST DISCOUNT-ANSWER
IF DA-STATUS NOT = C-STATUS(WS-I)
PERFORM FAIL-CASE
ELSE
IF DA-DISCOUNT NOT = C-DISCOUNT(WS-I)
PERFORM FAIL-CASE
ELSE
ADD 1 TO WS-PASSED
DISPLAY "pass case " WS-I
END-IF
END-IF.
FAIL-CASE.
ADD 1 TO WS-FAILED
MOVE DA-DISCOUNT TO WS-OUT
DISPLAY "FAIL case " WS-I " status " DA-STATUS
" discount " FUNCTION TRIM(WS-OUT)
" expected " C-STATUS(WS-I) " " C-DISCOUNT(WS-I).
IDENTIFICATION DIVISION.
PROGRAM-ID. DISCOUNT.
DATA DIVISION.
WORKING-STORAGE SECTION.
01 RATE-TABLE.
05 RATE-ENTRY PIC 9V99 OCCURS 2 TIMES.
01 WS-INDEX PIC 9.
01 WS-RATE PIC 9V99.
LINKAGE SECTION.
01 DISCOUNT-REQUEST.
05 DR-CODE PIC X(1).
05 DR-AMOUNT PIC 9(7)V99.
01 DISCOUNT-ANSWER.
05 DA-STATUS PIC 9(2).
05 DA-DISCOUNT PIC 9(7)V99.
05 DA-MESSAGE PIC X(30).
PROCEDURE DIVISION USING DISCOUNT-REQUEST DISCOUNT-ANSWER.
MOVE 0 TO DA-STATUS
MOVE 0 TO DA-DISCOUNT
MOVE SPACES TO DA-MESSAGE
MOVE 0.10 TO RATE-ENTRY(1)
MOVE 0.05 TO RATE-ENTRY(2)
EVALUATE DR-CODE
WHEN "A"
MOVE 1 TO WS-INDEX
WHEN "B"
MOVE 2 TO WS-INDEX
WHEN OTHER
MOVE 9 TO DA-STATUS
MOVE "unknown discount code" TO DA-MESSAGE
GOBACK
END-EVALUATE
MOVE RATE-ENTRY(WS-INDEX) TO WS-RATE
COMPUTE DA-DISCOUNT ROUNDED = DR-AMOUNT * WS-RATE
GOBACK.
END PROGRAM DISCOUNT.
A project would keep the unit in a file of its own and add it to the build line; the modules lesson has that form. The file here carries both so that the page is one listing, one command and one run, and nothing on it is a fragment.
The unit takes a discount code and an amount and answers a status, a discount and a message. The two rates are a table of two and the code selects the entry; a code that is neither A nor B is a status 9 with a message, which is the branch the happy path never reaches.
The table of cases
Four cases: ten percent of a hundred, ten percent of two hundred and fifty, five percent of a hundred, and an unknown code. Each case carries the answer it must produce, and the loop compares both the status and the amount.
Three decisions make the harness useful rather than decorative. The expected values are constants in the program, so a change in the unit is a failure and not a new output somebody has to notice. Both fields are compared, because a unit that returns the right amount with the wrong status is broken and a harness that checks one field would pass it. And a failure prints the actual value, the expected value and the case number, because the person reading the log has neither the program nor the table in front of them.
The run that passes
cobc -x -free -Wall -o harness harness.cob
./harness
echo $?
pass case 01
pass case 02
pass case 03
pass case 04
passed 004 failed 000
0
Four passes, status 0. The build compiles the harness and the unit from one file, so a change to the unit cannot be half-built. The arithmetic is checkable by hand: ten percent of a hundred is ten, of two hundred and fifty is twenty-five, and five percent of a hundred is five.
The run that fails
A harness nobody has seen fail is a harness nobody trusts. Change one expected value and run it again:
sed 's/MOVE 10.00 TO C-DISCOUNT(1)/MOVE 10.50 TO C-DISCOUNT(1)/' \
harness.cob > harness_bad.cob
cobc -x -free -Wall -o harness_bad harness_bad.cob
./harness_bad
echo $?
FAIL case 01 status 00 discount 10.00 expected 00 0000010.50
pass case 02
pass case 03
pass case 04
passed 003 failed 001
1
One failure, status 1, and a line that names the case, the actual amount and the expected one. The copy keeps the program name DISCHARNESS because the name of the main program is not used to resolve anything: the harness calls DISCOUNT, and the unit keeps that name in both copies.
What a harness of this shape cannot see
This harness tests the unit\'s contract and nothing else. It does not test the program that reads a request file, because that program has files and arguments and needs a different kind of test; the golden file page is that kind. It does not see a write past the end of a table if the answer still comes out right, which is exactly the case on the runtime checks page. And it cannot tell whether a branch was executed, which is what coverage answers.
What it does have is the property that matters most in a COBOL shop: it is a COBOL program. The person who maintains the discount rule can read it, add a case in five lines and run it with the compiler they already use, on the machine the batch runs on. That is worth more than a framework the maintainer cannot open.
The next page wraps a program like this in a shell script over a table of cases, which is the other half of a test suite: the harness for the unit, and a runner for the programs that read arguments and print lines.
The code on this page was built with cobc (GnuCOBOL) 3.2.0 and run on Alpine Linux 3.24.3 (x86_64) in a Docker container.
Article author: Arthur Isaev