Fixed-width records and files

A file of fixed-width records is the oldest interface in data processing and still the one most batch systems exchange. The record has no separators: every field is at a position, every record is the same length, and the file is read by programs written years apart. This page makes one in C++ and reads it in COBOL, then writes one in COBOL and reads it in C++, and shows the byte count of each step.

The 37-byte record

position length field type 1 6 id six characters, C00001 7 20 name twenty characters, space padded 27 2 region two characters, N1 29 9 amount nine digits, two implied decimals

Nothing in that table says who wrote the file. The bytes are the contract, and both languages have a way to say the same bytes: a C++ struct with character arrays, and a COBOL record of four fields.

C++ writes the file

The struct is packed, and the program prints two sizes: the packed record, and a struct of a character array and an int that the compiler is free to align. The second number is the one that ruins a fixed-width file when the attribute is forgotten:

#include <cstdio> #include <cstring> struct Customer { char id[6]; char name[20]; char region[2]; char amount[9]; } __attribute__((packed)); struct Aligned { char id[6]; int number; }; int main() { std::printf("packed record %zu bytes\n", sizeof(Customer)); std::printf("aligned struct %zu bytes\n", sizeof(Aligned)); const char* names[] = {"Ada Lovelace", "Grace Hopper", "Alan Turing", "Barbara Liskov"}; const char* ids[] = {"C00001", "C00002", "C00003", "C00004"}; const char* regions[] = {"N1", "N1", "S2", "E3"}; const int cents[] = {123456, 234567, 45678, 123456}; std::FILE* out = std::fopen("customers.dat", "w"); if (out == nullptr) { std::perror("customers.dat"); return 1; } for (int i = 0; i < 4; ++i) { Customer record{}; std::memset(&record, ' ', sizeof(record)); std::memcpy(record.id, ids[i], 6); std::memcpy(record.name, names[i], std::strlen(names[i])); std::memcpy(record.region, regions[i], 2); char amount[10]; std::snprintf(amount, sizeof(amount), "%09d", cents[i]); std::memcpy(record.amount, amount, 9); std::fwrite(&record, sizeof(record), 1, out); std::fputc('\n', out); } std::fclose(out); std::printf("wrote 4 records of %zu bytes\n", sizeof(Customer)); return 0; }

g++ -std=c++23 -Wall -Wextra -O2 -o write_records write_records.cpp
./write_records
wc -c customers.dat

packed record 37 bytes
aligned struct 12 bytes
wrote 4 records of 37 bytes
152 customers.dat

Four records of thirty-seven bytes and four newlines: a hundred and fifty-two bytes. The file itself, which is the input of every program below:

C00001Ada Lovelace N1000123456 C00002Grace Hopper N1000234567 C00003Alan Turing S2000045678 C00004Barbara Liskov E3000123456

COBOL reads it

The COBOL record describes the same four fields in the same order, and the FD is a line sequential file of that record. The program checks the status of the OPEN before it reads a byte, so a missing file is a message and status 8 and not a crash:

IDENTIFICATION DIVISION. PROGRAM-ID. READRECS. ENVIRONMENT DIVISION. INPUT-OUTPUT SECTION. FILE-CONTROL. SELECT CUS-F ASSIGN TO "customers.dat" ORGANIZATION IS LINE SEQUENTIAL FILE STATUS IS WS-STATUS. DATA DIVISION. FILE SECTION. FD CUS-F. 01 CUSTOMER-REC. 05 CUST-ID PIC X(6). 05 CUST-NAME PIC X(20). 05 CUST-REGION PIC X(2). 05 CUST-AMOUNT PIC 9(7)V99. WORKING-STORAGE SECTION. 01 WS-STATUS PIC XX. 01 WS-EOF PIC X VALUE "N". 01 WS-COUNT PIC 9(3) VALUE 0. 01 WS-TOTAL PIC 9(9)V99 VALUE 0. 01 WS-AMT-OUT PIC Z(7)9.99. 01 WS-TOT-OUT PIC Z(9)9.99. PROCEDURE DIVISION. OPEN INPUT CUS-F IF WS-STATUS NOT = "00" DISPLAY "customers.dat missing: run write_records first" UPON STDERR MOVE 8 TO RETURN-CODE STOP RUN END-IF PERFORM UNTIL WS-EOF = "Y" READ CUS-F AT END MOVE "Y" TO WS-EOF NOT AT END MOVE CUST-AMOUNT TO WS-AMT-OUT DISPLAY CUST-ID " " FUNCTION TRIM(CUST-NAME) " " FUNCTION TRIM(CUST-REGION) " " FUNCTION TRIM(WS-AMT-OUT) ADD 1 TO WS-COUNT ADD CUST-AMOUNT TO WS-TOTAL END-READ END-PERFORM CLOSE CUS-F MOVE WS-TOTAL TO WS-TOT-OUT DISPLAY "records " WS-COUNT " total " FUNCTION TRIM(WS-TOT-OUT) STOP RUN.

cobc -x -free -Wall -O2 -o read_records read_records.cob
./read_records
echo $?

C00001 Ada Lovelace N1 1234.56
C00002 Grace Hopper N1 2345.67
C00003 Alan Turing S2 456.78
C00004 Barbara Liskov E3 1234.56
records 004 total 5271.57
0

And the same program with no file beside it, which is the case a batch job meets when the previous step failed:

./read_records
echo $?

customers.dat missing: run write_records first
8

COBOL writes it back

The other direction: a COBOL program writes the same four records to customers2.dat from literals in working storage. Each line is one thirty-seven character literal, which is how a COBOL programmer sees a fixed-width record:

IDENTIFICATION DIVISION. PROGRAM-ID. WRITERECS. ENVIRONMENT DIVISION. INPUT-OUTPUT SECTION. FILE-CONTROL. SELECT OUT-F ASSIGN TO "customers2.dat" ORGANIZATION IS LINE SEQUENTIAL FILE STATUS IS WS-STATUS. DATA DIVISION. FILE SECTION. FD OUT-F. 01 CUSTOMER-REC. 05 CUST-ID PIC X(6). 05 CUST-NAME PIC X(20). 05 CUST-REGION PIC X(2). 05 CUST-AMOUNT PIC 9(7)V99. WORKING-STORAGE SECTION. 01 WS-STATUS PIC XX. 01 WS-I PIC 9(2). 01 WS-LINE PIC X(37). PROCEDURE DIVISION. OPEN OUTPUT OUT-F MOVE "C00001Ada Lovelace N1000123456" TO WS-LINE WRITE CUSTOMER-REC FROM WS-LINE MOVE "C00002Grace Hopper N1000234567" TO WS-LINE WRITE CUSTOMER-REC FROM WS-LINE MOVE "C00003Alan Turing S2000045678" TO WS-LINE WRITE CUSTOMER-REC FROM WS-LINE MOVE "C00004Barbara Liskov E3000123456" TO WS-LINE WRITE CUSTOMER-REC FROM WS-LINE CLOSE OUT-F DISPLAY "wrote 4 records of " LENGTH OF CUSTOMER-REC " bytes" STOP RUN.

cobc -x -free -Wall -O2 -o write_records_cob write_records.cob
./write_records_cob
wc -c customers2.dat

wrote 4 records of 37 bytes
152 customers2.dat

C++ reads that file

The C++ reader takes each line, checks that it is thirty-seven bytes, cuts the fields by offset and turns the nine digits into cents. It prints the same five lines as the COBOL reader, which is the only test that matters for an interface: both sides agree about the bytes.

#include <cstdio> #include <cstdlib> #include <fstream> #include <string> int main() { std::ifstream in("customers2.dat"); if (!in) { std::fprintf(stderr, "customers2.dat missing: run write_records_cob first\n"); return 8; } int count = 0; long long total = 0; std::string line; while (std::getline(in, line)) { if (line.size() != 37) { std::fprintf(stderr, "bad record length %zu\n", line.size()); return 4; } std::string id = line.substr(0, 6); std::string name = line.substr(6, 20); std::string region = line.substr(26, 2); long long cents = std::strtoll(line.substr(28, 9).c_str(), nullptr, 10); total += cents; while (!name.empty() && name.back() == ' ') { name.pop_back(); } std::printf("%s %s %s %lld.%02lld\n", id.c_str(), name.c_str(), region.c_str(), cents / 100, cents % 100); ++count; } std::printf("records %03d total %lld.%02lld\n", count, total / 100, total % 100); return 0; }

g++ -std=c++23 -Wall -Wextra -O2 -o read_records_cpp read_records.cpp
./read_records_cpp
echo $?

C00001 Ada Lovelace N1 1234.56
C00002 Grace Hopper N1 2345.67
C00003 Alan Turing S2 456.78
C00004 Barbara Liskov E3 1234.56
records 004 total 5271.57
0

Padding and the missing struct keyword

Three things about the two record descriptions are worth carrying away. The first is the twelve against thirty-seven: a struct of char[6] and int is twelve bytes because the compiler aligns the integer to four, and a COBOL record without SYNC is the sum of its fields. A COBOL record is packed by default; a C++ struct is not.

The second is that COBOL pads on a MOVE and never on a declaration. PIC X(20) holds twenty bytes whether the value is three characters or twenty, and the name field in the file is spaces after the last letter. The C++ writer had to fill the record with spaces before it copied the names, or the file would carry whatever the allocator left in the array.

The third is the assumption that the two sides share an encoding. Both programs here write ASCII, and the file is readable in an editor; a mainframe file is usually EBCDIC, and the encodings lesson is about the conversion step that then belongs in front of both readers. Nothing in the record layout says which encoding it holds, which is why every fixed-width interface document names it on the first page.

The COBOL side of this page is the material of SELECT and FD, line sequential files and the first capstone; the next page makes both languages do more work with the same file.

The COBOL was built with cobc (GnuCOBOL) 3.2.0 and the C++ with g++ (Alpine 15.2.0) 15.2.0, both with -O2, and run on Alpine Linux 3.24.3 (x86_64) in a Docker container with nproc 4.

Article author: Arthur Isaev