Batch job patterns
A batch job runs without a person in front of it, so it has to say what it did and whether the result can be trusted. Four things make that possible and every long-running job in the world has some version of them: a run header, a control total, a restart point, and a status that says the run failed.
A job in six paragraphs
The job reads a transaction file, skips what an earlier run already did, totals the rest and writes down where it stopped. Six paragraphs, in the order the main paragraph calls them:
IDENTIFICATION DIVISION.
PROGRAM-ID. BATCHJOB.
ENVIRONMENT DIVISION.
INPUT-OUTPUT SECTION.
FILE-CONTROL.
SELECT TXN-F ASSIGN TO "txn.dat"
ORGANIZATION IS LINE SEQUENTIAL
FILE STATUS IS WS-TXN-STATUS.
SELECT RESTART-F ASSIGN TO "restart.dat"
ORGANIZATION IS LINE SEQUENTIAL
FILE STATUS IS WS-RESTART-STATUS.
DATA DIVISION.
FILE SECTION.
FD TXN-F.
01 TXN-REC.
05 TXN-ID PIC 9(4).
05 TXN-AMOUNT PIC 9(6)V99.
FD RESTART-F.
01 RESTART-REC.
05 RESTART-ID PIC 9(4).
WORKING-STORAGE SECTION.
01 WS-TXN-STATUS PIC XX.
01 WS-RESTART-STATUS PIC XX.
01 WS-EOF PIC X VALUE "N".
01 WS-RESTART PIC 9(4) VALUE 0.
01 WS-READ PIC 9(4) VALUE 0.
01 WS-SKIPPED PIC 9(4) VALUE 0.
01 WS-PROCESSED PIC 9(4) VALUE 0.
01 WS-TOTAL PIC 9(9)V99 VALUE 0.
01 WS-NOW PIC X(21).
01 WS-OUT PIC Z(8)9.99.
01 WS-I PIC 9(2).
01 WS-DEMO-AMOUNT PIC 9(6)V99.
PROCEDURE DIVISION.
MAIN-PARA.
PERFORM SETUP-INPUT
PERFORM READ-RESTART
PERFORM RUN-HEADER
PERFORM PROCESS-FILE
PERFORM RUN-TRAILER
PERFORM SAVE-RESTART
STOP RUN.
SETUP-INPUT.
OPEN INPUT TXN-F
IF WS-TXN-STATUS = "35"
OPEN OUTPUT TXN-F
PERFORM VARYING WS-I FROM 1 BY 1 UNTIL WS-I > 6
MOVE WS-I TO TXN-ID
COMPUTE WS-DEMO-AMOUNT = WS-I * 100 + 25
MOVE WS-DEMO-AMOUNT TO TXN-AMOUNT
WRITE TXN-REC
END-PERFORM
END-IF
CLOSE TXN-F.
READ-RESTART.
OPEN INPUT RESTART-F
IF WS-RESTART-STATUS = "00"
READ RESTART-F
AT END MOVE 0 TO WS-RESTART
END-READ
IF WS-RESTART-STATUS = "00"
MOVE RESTART-ID TO WS-RESTART
END-IF
END-IF
CLOSE RESTART-F.
RUN-HEADER.
MOVE FUNCTION CURRENT-DATE TO WS-NOW
DISPLAY "job BATCHJOB started " WS-NOW(1:8) " " WS-NOW(9:6)
DISPLAY "input txn.dat restart " WS-RESTART.
PROCESS-FILE.
OPEN INPUT TXN-F
PERFORM UNTIL WS-EOF = "Y"
READ TXN-F
AT END MOVE "Y" TO WS-EOF
NOT AT END
IF WS-TXN-STATUS = "00"
ADD 1 TO WS-READ
IF TXN-ID IS GREATER THAN WS-RESTART
ADD 1 TO WS-PROCESSED
ADD TXN-AMOUNT TO WS-TOTAL
ELSE
ADD 1 TO WS-SKIPPED
END-IF
ELSE
MOVE "Y" TO WS-EOF
END-IF
END-READ
END-PERFORM
CLOSE TXN-F.
RUN-TRAILER.
MOVE WS-TOTAL TO WS-OUT
DISPLAY "read " WS-READ " skipped " WS-SKIPPED
" processed " WS-PROCESSED
DISPLAY "control total " WS-OUT.
SAVE-RESTART.
MOVE WS-READ TO RESTART-ID
OPEN OUTPUT RESTART-F
WRITE RESTART-REC
CLOSE RESTART-F.
The listing is the whole program, and the order of the paragraphs is the lesson. The first one writes six demo transactions only when the file is not there, so the page is one program you can build and run; the last two paragraphs are the report and the record of where the run stopped.
The file declarations are the ones of the SELECT and FD lesson, and the loop over it is the one of sequential files. What is new here is the order of the paragraphs and the two files they keep beside the data.
The run header
RUN-HEADER.
MOVE FUNCTION CURRENT-DATE TO WS-NOW
DISPLAY "job BATCHJOB started " WS-NOW(1:8) " " WS-NOW(9:6)
DISPLAY "input txn.dat restart " WS-RESTART.
Three facts, and each one has a job. The program name says which job wrote the lines when several jobs share a log. The date and time say when, in the format the rest of the log uses. The restart value says what the run believed about the state of the file before it read a record, and it is the line to read first when a job processed nothing for no visible reason.
A header that names its input file matters more than it looks: the same job run with a test file and with the production file prints one line that tells the two runs apart, and the log then explains why the totals differ.
The control total
RUN-TRAILER.
MOVE WS-TOTAL TO WS-OUT
DISPLAY "read " WS-READ " skipped " WS-SKIPPED
" processed " WS-PROCESSED
DISPLAY "control total " WS-OUT.
A control total is a number that a second, independent count can check: the number of records read, the number processed and the sum of their amounts. The three lines of the trailer are what a person compares with the numbers from the source system, and they are what a checking program reads when nobody is looking.
The trailer prints a field with Z(8)9.99, so the total has no leading zeros and a decimal point, which is what the person reading it needs. The check program below compares the same total as a plain PIC 9(9)V99, because a check between two programs is a comparison of values and not of printed forms.
cobc -x -free -Wall -o job job.cob
./job
job BATCHJOB started 20261118 145443
input txn.dat restart 0000
read 0006 skipped 0000 processed 0006
control total 2250.00
The restart point
The restart file holds one number: the last record the job read. The run starts by reading it, and a record whose id is not greater than that number is skipped and counted:
READ-RESTART.
OPEN INPUT RESTART-F
IF WS-RESTART-STATUS = "00"
READ RESTART-F
AT END MOVE 0 TO WS-RESTART
END-READ
IF WS-RESTART-STATUS = "00"
MOVE RESTART-ID TO WS-RESTART
END-IF
END-IF
CLOSE RESTART-F.
The second run of the same command finds a restart file and does nothing, which is the whole point: a scheduler that runs the job twice by mistake does not process the same transactions twice.
./job
job BATCHJOB started 20261118 145443
input txn.dat restart 0006
read 0006 skipped 0006 processed 0000
control total 0.00
SAVE-RESTART writes WS-READ and not WS-PROCESSED, so the restart point is the last record the job has seen, not the last one it changed. The difference matters the day a record is rejected in the middle: the job stops before it writes the restart, and the next run reads the rejected record again instead of jumping over it.
A job that refuses to pass
The second program checks the control total against a file written by the job or by the scheduler. A missing file, a wrong count and a wrong total each get their own message and their own status:
IDENTIFICATION DIVISION.
PROGRAM-ID. CHECKJOB.
ENVIRONMENT DIVISION.
INPUT-OUTPUT SECTION.
FILE-CONTROL.
SELECT CONTROL-F ASSIGN TO "control.dat"
ORGANIZATION IS LINE SEQUENTIAL
FILE STATUS IS WS-STATUS.
DATA DIVISION.
FILE SECTION.
FD CONTROL-F.
01 CONTROL-REC.
05 CTR-COUNT PIC 9(4).
05 CTR-TOTAL PIC 9(9)V99.
WORKING-STORAGE SECTION.
01 WS-STATUS PIC XX.
01 WS-COUNT PIC 9(4) VALUE 6.
01 WS-TOTAL PIC 9(9)V99 VALUE 2250.00.
01 WS-OUT PIC Z(8)9.99.
PROCEDURE DIVISION.
OPEN INPUT CONTROL-F
IF WS-STATUS NOT = "00"
DISPLAY "control file missing: control.dat" UPON STDERR
MOVE 8 TO RETURN-CODE
STOP RUN
END-IF.
READ CONTROL-F
AT END
DISPLAY "control file empty" UPON STDERR
MOVE 8 TO RETURN-CODE
STOP RUN
END-READ.
CLOSE CONTROL-F.
IF CTR-COUNT NOT = WS-COUNT
DISPLAY "count mismatch: expected " CTR-COUNT
" actual " WS-COUNT UPON STDERR
MOVE 16 TO RETURN-CODE
STOP RUN
END-IF.
IF CTR-TOTAL NOT = WS-TOTAL
MOVE WS-TOTAL TO WS-OUT
DISPLAY "total mismatch: expected " CTR-TOTAL
" actual " WS-OUT UPON STDERR
MOVE 16 TO RETURN-CODE
STOP RUN
END-IF.
DISPLAY "control ok: " CTR-COUNT " records".
STOP RUN.
The two header lines of that listing are IDENTIFICATION DIVISION. and PROGRAM-ID. CHECKJOB.. The control record is PIC 9(4) and PIC 9(9)V99, so a good one is fifteen characters. The four runs, with the file created by printf between them:
cobc -x -free -Wall -o jobfail jobfail.cob
./jobfail
echo $?
printf '000600000225000\n' > control.dat
./jobfail
echo $?
printf '000700000225000\n' > control.dat
./jobfail
echo $?
printf '000600000220000\n' > control.dat
./jobfail
echo $?
control file missing: control.dat
8
control ok: 0006 records
0
count mismatch: expected 0007 actual 0006
16
total mismatch: expected 000002200.00 actual 2250.00
16
Status 8 is "a file the job needed was not there" and status 16 is "the data does not add up". A scheduler reads those two numbers differently: 8 is often worth a retry after the file arrives, and 16 never is. That is why a job does not exit with 1 for everything, and why the message names the file and the values.
Exercises
- Add a third line to the trailer: the first and last transaction id the run processed. Which paragraph has to remember them?
- Make the job exit with 12 when a record has an amount of zero, and leave the restart file untouched in that case.
- Delete restart.dat and run the job twice in a row. What does each run print, and why is the second one not a no-op?
- Give the check program a third check: the number of skipped records must be zero on a first run. Which status would you give a failure of it?
Questions
A new set of questions in a new order every time. Choose one answer for each, then check.
-
What belongs in a run header?
Answer
The program, the date and time, and the input and restart the run used
RUN-HEADER prints the name, CURRENT-DATE and the restart value.
-
Which record does SAVE-RESTART write?
Answer
The last record the job read, processed or skipped
It writes WS-READ, so a rejected record is read again next time.
-
What is a control total for?
Answer
A second, independent count that shows the run read what it should have
The trailer prints count, skipped, processed and the sum of amounts.
-
What does status 16 of the check program mean?
Answer
The data does not add up, and a retry will not help
Both a count mismatch and a total mismatch end with 16; a missing file ends with 8.
-
What does this program print?
IDENTIFICATION DIVISION. PROGRAM-ID. Q5. DATA DIVISION. WORKING-STORAGE SECTION. 01 N PIC 9(4) VALUE 0. 01 I PIC 9(2). PROCEDURE DIVISION. PERFORM VARYING I FROM 1 BY 1 UNTIL I IS GREATER THAN 4 ADD I TO N END-PERFORM. DISPLAY N. STOP RUN.Answer
0010
One plus two plus three plus four is ten.
-
What does this program print?
IDENTIFICATION DIVISION. PROGRAM-ID. Q6. DATA DIVISION. WORKING-STORAGE SECTION. 01 T PIC 9(3)V99 VALUE 250.00. 01 O PIC Z(2)9.99. PROCEDURE DIVISION. MOVE T TO O DISPLAY O. STOP RUN.Answer
250.00
Four integer digits fill the Z(2)9 part and the decimals are kept.
-
What does this program print?
IDENTIFICATION DIVISION. PROGRAM-ID. Q7. DATA DIVISION. WORKING-STORAGE SECTION. 01 W PIC X(3) VALUE "ABC". PROCEDURE DIVISION. DISPLAY W(2:1) W(1:1). STOP RUN.Answer
BA
The second character is B and the first is A.
-
What does this program print?
IDENTIFICATION DIVISION. PROGRAM-ID. Q8. DATA DIVISION. WORKING-STORAGE SECTION. 01 A PIC 9(2) VALUE 10. 01 B PIC 9(2) VALUE 4. 01 C PIC 9(3). PROCEDURE DIVISION. COMPUTE C = A * B / 2 DISPLAY C. STOP RUN.Answer
020
Ten times four is forty, and half of it is twenty.
-
Why does the job write txn.dat only when it is not there?
Answer
So a second run reads the file the first run left, which is what a restart test needs
The FILE STATUS 35 branch is the setup, and the second run skips it.
-
Why is a status of 8 worth a retry and 16 not?
Answer
A missing file may arrive later; wrong data will stay wrong
The missing control file and the mismatched total are different kinds of failure.
The code on this page was built with cobc (GnuCOBOL) 3.2.0 and run on Alpine Linux 3.24.3 (x86_64) in a Docker container.
Article author: Arthur Isaev