A shell runner and exit statuses

Not every COBOL program is a subprogram with a record interface. A batch step or a utility takes arguments, reads files and prints lines, and the test for it is the same test any command-line program gets: run it, compare what it printed, look at its status. In a shell, that is ten lines of sh.

A program with an argument

The program is a front end for the discount unit of the previous page: it takes a code and an amount, calls the unit and prints the discount, or the message and a status of its own. The unit is the second program in the same file, so one command builds both:

IDENTIFICATION DIVISION. PROGRAM-ID. DISCOUNTCLI. DATA DIVISION. WORKING-STORAGE SECTION. 01 WS-COUNT PIC 9(3). 01 WS-CODE PIC X(10). 01 WS-AMOUNT PIC X(20). 01 WS-OUT PIC Z(6)9.99. 01 DISCOUNT-REQUEST. 05 DR-CODE PIC X(1). 05 DR-AMOUNT PIC 9(7)V99. 01 DISCOUNT-ANSWER. 05 DA-STATUS PIC 9(2). 05 DA-DISCOUNT PIC 9(7)V99. 05 DA-MESSAGE PIC X(30). PROCEDURE DIVISION. ACCEPT WS-COUNT FROM ARGUMENT-NUMBER IF WS-COUNT NOT = 2 DISPLAY "usage: discount_cli CODE AMOUNT" UPON STDERR MOVE 2 TO RETURN-CODE STOP RUN END-IF. ACCEPT WS-CODE FROM ARGUMENT-VALUE ACCEPT WS-AMOUNT FROM ARGUMENT-VALUE IF FUNCTION TEST-NUMVAL(WS-AMOUNT) NOT = 0 DISPLAY "bad amount: " FUNCTION TRIM(WS-AMOUNT) UPON STDERR MOVE 4 TO RETURN-CODE STOP RUN END-IF. MOVE WS-CODE(1:1) TO DR-CODE COMPUTE DR-AMOUNT = FUNCTION NUMVAL(WS-AMOUNT) CALL "DISCOUNT" USING DISCOUNT-REQUEST DISCOUNT-ANSWER IF DA-STATUS = 0 MOVE DA-DISCOUNT TO WS-OUT DISPLAY FUNCTION TRIM(WS-OUT) ELSE DISPLAY FUNCTION TRIM(DA-MESSAGE) UPON STDERR END-IF. MOVE DA-STATUS TO RETURN-CODE. IDENTIFICATION DIVISION. PROGRAM-ID. DISCOUNT. DATA DIVISION. WORKING-STORAGE SECTION. 01 RATE-TABLE. 05 RATE-ENTRY PIC 9V99 OCCURS 2 TIMES. 01 WS-INDEX PIC 9. 01 WS-RATE PIC 9V99. LINKAGE SECTION. 01 DISCOUNT-REQUEST. 05 DR-CODE PIC X(1). 05 DR-AMOUNT PIC 9(7)V99. 01 DISCOUNT-ANSWER. 05 DA-STATUS PIC 9(2). 05 DA-DISCOUNT PIC 9(7)V99. 05 DA-MESSAGE PIC X(30). PROCEDURE DIVISION USING DISCOUNT-REQUEST DISCOUNT-ANSWER. MOVE 0 TO DA-STATUS MOVE 0 TO DA-DISCOUNT MOVE SPACES TO DA-MESSAGE MOVE 0.10 TO RATE-ENTRY(1) MOVE 0.05 TO RATE-ENTRY(2) EVALUATE DR-CODE WHEN "A" MOVE 1 TO WS-INDEX WHEN "B" MOVE 2 TO WS-INDEX WHEN OTHER MOVE 9 TO DA-STATUS MOVE "unknown discount code" TO DA-MESSAGE GOBACK END-EVALUATE MOVE RATE-ENTRY(WS-INDEX) TO WS-RATE COMPUTE DA-DISCOUNT ROUNDED = DR-AMOUNT * WS-RATE GOBACK. END PROGRAM DISCOUNT.

The program prints one line on success and one on failure, and it returns the unit status, so the shell can tell a wrong argument (2), a bad number (4) and an unknown code (9) apart:

cobc -x -free -Wall -o discount_cli discount_cli.cob
./discount_cli A 100.00
echo $?
./discount_cli X 100.00
echo $?

10.00
0
unknown discount code
9

The runner

The script builds the program, then reads a table of three-field rows: code, amount and the line the program must print. Each row runs the program in a command substitution, compares the two strings and counts. The last line is the status: zero when every row passed, non-zero when one did not.

#!/bin/sh set -u cobc -x -free -o discount_cli discount_cli.cob || exit 2 pass=0 fail=0 while read -r code amount expected; do got=$(./discount_cli "$code" "$amount") if [ "$got" = "$expected" ]; then pass=$((pass + 1)) echo "ok $code $amount -> $got" else fail=$((fail + 1)) echo "FAIL $code $amount -> $got expected $expected" fi done <<'CASES' A 100.00 10.00 A 250.00 25.00 B 100.00 5.00 B 9.99 0.50 CASES echo "passed $pass failed $fail" [ "$fail" -eq 0 ]

Four details are deliberate. The table is a here-document, so the run cases are in the same file as the runner and there is no second file to keep in step. set -u makes an unset variable an error, which catches a typo in the script itself. The comparison [ "$got" = "$expected" ] is exact, because the program whole contract is the line. And the last command is the test [ "$fail" -eq 0 ], so the script exit status is the result without an explicit exit.

The run that passes

chmod +x run_cases.sh
./run_cases.sh
echo $?

ok   A 100.00 -> 10.00
ok   A 250.00 -> 25.00
ok   B 100.00 -> 5.00
ok   B 9.99 -> 0.50
passed 4 failed 0
0

The fourth row is the interesting one: five percent of nine and ninety-nine is 0.4995, and the unit ROUNDED turns that into 0.50. The runner checks the rounding without knowing anything about rounding, because the expected line is the specification.

A row that fails

One changed expectation is enough to see the other branch. The second script is the first with one row edited:

sed 's/^B 100.00 5.00$/B 100.00 0.55/' run_cases.sh > run_cases_bad.sh
chmod +x run_cases_bad.sh
./run_cases_bad.sh
echo $?

ok   A 100.00 -> 10.00
ok   A 250.00 -> 25.00
FAIL B 100.00 -> 5.00 expected 0.55
ok   B 9.99 -> 0.50
passed 3 failed 1
1

One line names the input, the output and the expectation, and the script returns 1. The other three rows still ran, which is the difference between a runner and a harness that stops at the first failure: a runner that stops hides the second bug behind the first, and a run over forty cases then takes forty fixes.

The status a build server reads

A build server knows one thing about a command: whether it succeeded. The script is written so that its status is the whole result, which means it can sit in a Makefile, a CI step or a cron line without anybody parsing its output:

tests:
    ./run_cases.sh

.PHONY: tests

The output is for the person and the status is for the machine, and a test suite needs both. A script that prints FAIL lines and returns 0 is worse than no script, because the build goes green and the failure becomes a line in a log nobody reads.

The same shape works for a golden file: run the program, redirect its output to a file and compare that file with diff, whose status is 0 exactly when the two are equal. That is the next page.

The code on this page was built with cobc (GnuCOBOL) 3.2.0 and run with the sh of Alpine Linux 3.24.3 (x86_64) in a Docker container.

Article author: Arthur Isaev