MERGE
MERGE is the sibling of SORT: where a sort puts one input in order, a merge interleaves several inputs that are already in that order into one. It is the verb of a nightly job that adds today's transactions to a master file, and it costs one pass over each file.
MERGE two sorted files
The shape is the one of the sort: an SD work file with the key, USING for the inputs and GIVING for the output. The input files are named in one clause, separated by spaces:
IDENTIFICATION DIVISION.
PROGRAM-ID. MG1.
ENVIRONMENT DIVISION.
INPUT-OUTPUT SECTION.
FILE-CONTROL.
SELECT M-F ASSIGN TO "mergew.dat"
FILE STATUS IS WS-MRG-ST.
SELECT A-F ASSIGN TO "a.dat"
ORGANIZATION IS LINE SEQUENTIAL
FILE STATUS IS WS-A-ST.
SELECT B-F ASSIGN TO "b.dat"
ORGANIZATION IS LINE SEQUENTIAL
FILE STATUS IS WS-B-ST.
SELECT OUT-F ASSIGN TO "merged.dat"
ORGANIZATION IS LINE SEQUENTIAL
FILE STATUS IS WS-OUT-ST.
DATA DIVISION.
FILE SECTION.
FD A-F.
01 A-REC.
05 A-KEY PIC X(3).
05 A-VAL PIC 9(3).
FD B-F.
01 B-REC.
05 B-KEY PIC X(3).
05 B-VAL PIC 9(3).
SD M-F.
01 M-REC.
05 M-KEY PIC X(3).
05 M-VAL PIC 9(3).
FD OUT-F.
01 OUT-REC.
05 OUT-KEY PIC X(3).
05 OUT-VAL PIC 9(3).
WORKING-STORAGE SECTION.
01 WS-MRG-ST PIC XX.
01 WS-A-ST PIC XX.
01 WS-B-ST PIC XX.
01 WS-OUT-ST PIC XX.
PROCEDURE DIVISION.
OPEN OUTPUT A-F.
MOVE "A10" TO A-KEY.
MOVE 10 TO A-VAL.
WRITE A-REC.
MOVE "C30" TO A-KEY.
MOVE 30 TO A-VAL.
WRITE A-REC.
MOVE "E50" TO A-KEY.
MOVE 50 TO A-VAL.
WRITE A-REC.
CLOSE A-F.
OPEN OUTPUT B-F.
MOVE "B20" TO B-KEY.
MOVE 20 TO B-VAL.
WRITE B-REC.
MOVE "D40" TO B-KEY.
MOVE 40 TO B-VAL.
WRITE B-REC.
CLOSE B-F.
MERGE M-F
ON ASCENDING KEY M-KEY
USING A-F B-F
GIVING OUT-F.
DISPLAY "merge " WS-MRG-ST.
OPEN INPUT OUT-F.
PERFORM UNTIL WS-OUT-ST = "10"
READ OUT-F
AT END MOVE "10" TO WS-OUT-ST
NOT AT END DISPLAY OUT-KEY " " OUT-VAL
END-READ
END-PERFORM.
CLOSE OUT-F.
STOP RUN.
cobc -x -free -Wall -o mg1 mg1.cob
./mg1
merge 00
A10 010
B20 020
C30 030
D40 040
E50 050
The two files were in key order before the merge, and the output interleaves them. The clause takes more than two files: the same program with a third input holding A15 printed A10, A15, B20, C30, D40, E50, which is what a job that merges a master with two days of transactions does.
MERGE keeps duplicates
A merge is not a match-merge and not a join: it does not compare the records of two files against each other beyond their keys, and it does not drop a key that appears twice. Both records are in the output:
IDENTIFICATION DIVISION.
PROGRAM-ID. MG2.
ENVIRONMENT DIVISION.
INPUT-OUTPUT SECTION.
FILE-CONTROL.
SELECT M-F ASSIGN TO "mergew2.dat"
FILE STATUS IS WS-MRG-ST.
SELECT A-F ASSIGN TO "a.dat"
ORGANIZATION IS LINE SEQUENTIAL
FILE STATUS IS WS-A-ST.
SELECT B-F ASSIGN TO "b.dat"
ORGANIZATION IS LINE SEQUENTIAL
FILE STATUS IS WS-B-ST.
SELECT OUT-F ASSIGN TO "merged2.dat"
ORGANIZATION IS LINE SEQUENTIAL
FILE STATUS IS WS-OUT-ST.
DATA DIVISION.
FILE SECTION.
FD A-F.
01 A-REC.
05 A-KEY PIC X(3).
05 A-VAL PIC 9(3).
FD B-F.
01 B-REC.
05 B-KEY PIC X(3).
05 B-VAL PIC 9(3).
SD M-F.
01 M-REC.
05 M-KEY PIC X(3).
05 M-VAL PIC 9(3).
FD OUT-F.
01 OUT-REC.
05 OUT-KEY PIC X(3).
05 OUT-VAL PIC 9(3).
WORKING-STORAGE SECTION.
01 WS-MRG-ST PIC XX.
01 WS-A-ST PIC XX.
01 WS-B-ST PIC XX.
01 WS-OUT-ST PIC XX.
PROCEDURE DIVISION.
OPEN OUTPUT A-F.
MOVE "A10" TO A-KEY.
MOVE 10 TO A-VAL.
WRITE A-REC.
MOVE "C30" TO A-KEY.
MOVE 30 TO A-VAL.
WRITE A-REC.
MOVE "E50" TO A-KEY.
MOVE 50 TO A-VAL.
WRITE A-REC.
CLOSE A-F.
OPEN OUTPUT B-F.
MOVE "A10" TO B-KEY.
MOVE 99 TO B-VAL.
WRITE B-REC.
MOVE "D40" TO B-KEY.
MOVE 40 TO B-VAL.
WRITE B-REC.
CLOSE B-F.
MERGE M-F
ON ASCENDING KEY M-KEY
USING A-F B-F
GIVING OUT-F.
DISPLAY "merge " WS-MRG-ST.
OPEN INPUT OUT-F.
PERFORM UNTIL WS-OUT-ST = "10"
READ OUT-F
AT END MOVE "10" TO WS-OUT-ST
NOT AT END DISPLAY OUT-KEY " " OUT-VAL
END-READ
END-PERFORM.
CLOSE OUT-F.
STOP RUN.
cobc -x -free -Wall -o mg2 mg2.cob
./mg2
merge 00
A10 010
A10 099
C30 030
D40 040
E50 050
Two records with key A10, both in the output, the one of the first file first. A job that must not have two records with one key does the merge and then a pass over the result with a break on the key, which is where the choice between the two records is made and logged. Expecting MERGE to do that is the mistake.
Unsorted input, and what this compiler does
The standard requires the input files of a merge to be in the order of the key. This compiler does not enforce it, and it does not report it either: a merge of two unsorted files came out fully sorted here.
IDENTIFICATION DIVISION.
PROGRAM-ID. MG3.
ENVIRONMENT DIVISION.
INPUT-OUTPUT SECTION.
FILE-CONTROL.
SELECT M-F ASSIGN TO "mergew3.dat"
FILE STATUS IS WS-MRG-ST.
SELECT A-F ASSIGN TO "u1.dat"
ORGANIZATION IS LINE SEQUENTIAL
FILE STATUS IS WS-A-ST.
SELECT B-F ASSIGN TO "u2.dat"
ORGANIZATION IS LINE SEQUENTIAL
FILE STATUS IS WS-B-ST.
SELECT OUT-F ASSIGN TO "merged3.dat"
ORGANIZATION IS LINE SEQUENTIAL
FILE STATUS IS WS-OUT-ST.
DATA DIVISION.
FILE SECTION.
FD A-F.
01 A-REC.
05 A-KEY PIC X(3).
FD B-F.
01 B-REC.
05 B-KEY PIC X(3).
SD M-F.
01 M-REC PIC X(3).
FD OUT-F.
01 OUT-REC PIC X(3).
WORKING-STORAGE SECTION.
01 WS-MRG-ST PIC XX.
01 WS-A-ST PIC XX.
01 WS-B-ST PIC XX.
01 WS-OUT-ST PIC XX.
PROCEDURE DIVISION.
OPEN OUTPUT A-F.
MOVE "Z90" TO A-KEY.
WRITE A-REC.
MOVE "A10" TO A-KEY.
WRITE A-REC.
CLOSE A-F.
OPEN OUTPUT B-F.
MOVE "M50" TO B-KEY.
WRITE B-REC.
MOVE "B20" TO B-KEY.
WRITE B-REC.
CLOSE B-F.
MERGE M-F
ON ASCENDING KEY M-REC
USING A-F B-F
GIVING OUT-F.
DISPLAY "merge " WS-MRG-ST.
OPEN INPUT OUT-F.
PERFORM UNTIL WS-OUT-ST = "10"
READ OUT-F
AT END MOVE "10" TO WS-OUT-ST
NOT AT END DISPLAY OUT-REC
END-READ
END-PERFORM.
CLOSE OUT-F.
STOP RUN.
cobc -x -free -Wall -o mg3 mg3.cob
./mg3
merge 00
A10
B20
M50
Z90
The first file was written as Z90 then A10 and the second as M50 then B20, and the output is A10, B20, M50, Z90: GnuCOBOL sorted the inputs instead of interleaving them, without a warning and with status 00. A program that relies on that is wrong on a compiler that merges as the standard says, and the fault appears as a report out of order on somebody else's machine. Sort before the merge, and let the merge do the one thing it promises.
Exercises
- Add a third input file to mg1 with keys F60 and G70. Where do the two records land in the output?
- Make the two files of mg1 overlap completely, with the same key in both. How many records does the output hold?
- Sort the inputs of mg3 and run it again. Which output changes, and which does not?
- Add a count of the records of each input file to mg1 before the merge, and the count of the output after it.
Questions
A new set of questions in a new order every time. Choose one answer for each, then check.
-
What does MERGE do?
Answer
It interleaves several files that are already in key order into one file
The output of the first program interleaved the two sorted files.
-
Which clause names the input files of a MERGE?
Answer
USING
The verb read USING A-F B-F and wrote GIVING OUT-F.
-
What does MERGE do with a key that appears in both input files?
Answer
It writes both records to the output
The duplicate program printed A10 twice.
-
What did this compiler do with unsorted input files?
Answer
It sorted them and produced a sorted output with status 00
The output of the third program is in order although both inputs were not.
-
What does this program print?
IDENTIFICATION DIVISION. PROGRAM-ID. Q5. DATA DIVISION. WORKING-STORAGE SECTION. 01 W-A PIC X(3) VALUE "A10". 01 W-B PIC X(3) VALUE "B20". 01 W-N PIC 9 VALUE 0. PROCEDURE DIVISION. IF W-A IS LESS THAN W-B ADD 1 TO W-N END-IF. IF W-B IS LESS THAN W-A ADD 1 TO W-N END-IF. DISPLAY W-N. STOP RUN.Answer
1
Only the first comparison is true, which is how a merge decides which file to take from.
-
What does this program print?
IDENTIFICATION DIVISION. PROGRAM-ID. Q6. DATA DIVISION. WORKING-STORAGE SECTION. 01 W-K PIC X(3) VALUE SPACES. 01 W-N PIC 9 VALUE 0. 01 W-I PIC 9 VALUE 0. PROCEDURE DIVISION. PERFORM VARYING W-I FROM 1 BY 1 UNTIL W-I IS GREATER THAN 3 MOVE "A10" TO W-K ADD 1 TO W-N END-PERFORM. DISPLAY W-K " " W-N. STOP RUN.Answer
A10 3
The key field is overwritten three times and the counter counts the passes.
-
What does this program print?
IDENTIFICATION DIVISION. PROGRAM-ID. Q7. DATA DIVISION. WORKING-STORAGE SECTION. 01 W-A PIC X(3) VALUE "C30". 01 W-B PIC X(3) VALUE "C30". PROCEDURE DIVISION. IF W-A = W-B DISPLAY "same key" ELSE DISPLAY "different key" END-IF. STOP RUN.Answer
same key
Two equal keys are what a merge sees when it keeps both records.
-
What does this program print?
IDENTIFICATION DIVISION. PROGRAM-ID. Q8. DATA DIVISION. WORKING-STORAGE SECTION. 01 W-N PIC 9(2) VALUE 0. 01 W-E PIC X VALUE "N". PROCEDURE DIVISION. PERFORM UNTIL W-E = "Y" ADD 1 TO W-N IF W-N = 2 MOVE "Y" TO W-E END-IF END-PERFORM. DISPLAY W-N. STOP RUN.Answer
2
The loop ends on its second pass, where the flag is set.
-
Why is a merge cheaper than a sort of the same data?
Answer
It reads each record once and needs no comparison sort, because the inputs are already in order
A merge makes one pass over each input; a sort has to order the records first.
-
A job merges a master file with a transaction file and expects one record per key in the output. What is wrong?
Answer
A merge keeps every record, so the job needs a pass over the result to combine the duplicates
The duplicate program showed both records of key A10 in the output.
The code on this page was built with cobc (GnuCOBOL) 3.2.0 and run on Alpine Linux 3.24.3 (x86_64) in a Docker container.
Article author: Arthur Isaev