Capstone 3: reading a record file

The third capstone is the other half of a Fortran programmer's work: not a solver but a batch that reads a file somebody else wrote, turns every line into a record, sorts the records and writes a report. It is where most of the I/O of part 5 is used at once, and where the two decisions that decide whether a batch survives contact with real data are made: what to do with a line that does not parse, and how to grow an array whose size is not known until the file has been read.

The record file

The file is comma-separated with three fields: a date, a region and an amount. It has fourteen lines, of which twelve are records, one is a comment and one has a field that is not a number. That is a realistic mix: a real export from a spreadsheet has a title row, a blank line at the end and a row somebody typed by hand.

The file sales.csv, which the program writes before it reads it so that the page runs anywhere:

2026-03-02,east,3050.00
2026-01-11,south,980.00
2026-02-16,north,1890.00
2026-01-04,north,1200.50
2026-03-16,south,1425.60
2026-02-09,east,540.75
2026-01-25,north,760.00
2026-02-16,east,410.00
2026-01-18,east,1430.25
2026-03-09,north,675.25
2026-02-23,south,1120.40
2026-03-23,east,880.00
# a comment line the batch skips
2026-03-30,west,oops

A real batch takes the file from its command line or from a fixed name and never writes it. The WRITE_SAMPLE procedure exists so that this page and its program carry their own input, which is the same rule the data-file lessons of the course follow.

The twelve records arrive in the order the export wrote them, which is neither sorted by date nor grouped by region, and one date, 2026-02-16, appears twice with two different regions. That is deliberate: the next lesson sorts this file by date and then by region, and a file that is already in order would show nothing.

Reading with IOSTAT

The loop reads one line at a time with READ (u, '(a)', iostat=ios) line and stops on any non-zero status, which is how the end of the file arrives. IOSTAT rather than IOSTAT_END is enough here because every non-zero value ends the loop; a program that has to tell a read error from the end of the file compares the value with IOSTAT_END, as lesson 057 shows.

The open uses NEWUNIT, which gives a unit number that no other part of the program can be using, and STATUS='OLD', which fails if the file is not there instead of creating an empty one. The failure is a ERROR STOP with a message: a batch that cannot open its input has nothing to do.

One line at a time

The line is parsed with a list-directed internal read, the same trick as lesson 058: READ (line, *, iostat=ios) r%date, r%region, r%amount. List-directed input takes a comma as a separator, so the three fields arrive without a format, and the status tells the program whether the line had three fields of the right kinds. The comment line fails because its first field is not a date of ten characters, and the last line fails because oops is not a number.

Two conditions decide whether a parsed line is a record: ios == 0 .and. len_trim(r%date) == 10 .and. r%amount >= 0.0_dp. The length test rejects a short first field, which list-directed input would accept as a short string, and the amount test rejects a negative value that the batch does not expect. Every other line goes into the list of skipped lines with its text, so the report can print it. That is the decision this page is really about: a batch that stops at the first bad line cannot process a real file, and a batch that ignores bad lines silently loses records.

Growing the array

The number of records is not known until the file has been read, so the array starts at eight and doubles when it is full. MOVE_ALLOC hands the new storage over without copying it twice: the temporary bigger receives the old elements, and the call makes list point at the new block and deallocates the temporary, which is now empty. Reallocating the array on the left of an assignment would copy the elements twice and, in a loop, cost the square of the size.

The same procedure is written twice, once for the records and once for the skipped lines, because Fortran has no generic over two different types without an interface. A project that needed more would make a generic interface with two specific procedures, as lesson 066 shows.

The run

The file load.f90, the module and a program that writes the sample, loads it and prints what it found:

module records use, intrinsic :: iso_fortran_env, only: dp => real64 implicit none private public :: record, load, write_sample integer, parameter :: date_len = 10, region_len = 8 type :: record character(len=date_len) :: date = ' ' character(len=region_len) :: region = ' ' real(dp) :: amount = 0.0_dp end type record contains subroutine write_sample(path) character(len=*), intent(in) :: path integer :: u open (newunit=u, file=path, status='replace', action='write') write (u, '(a)') '2026-03-02,east,3050.00' write (u, '(a)') '2026-01-11,south,980.00' write (u, '(a)') '2026-02-16,north,1890.00' write (u, '(a)') '2026-01-04,north,1200.50' write (u, '(a)') '2026-03-16,south,1425.60' write (u, '(a)') '2026-02-09,east,540.75' write (u, '(a)') '2026-01-25,north,760.00' write (u, '(a)') '2026-02-16,east,410.00' write (u, '(a)') '2026-01-18,east,1430.25' write (u, '(a)') '2026-03-09,north,675.25' write (u, '(a)') '2026-02-23,south,1120.40' write (u, '(a)') '2026-03-23,east,880.00' write (u, '(a)') '# a comment line the batch skips' write (u, '(a)') '2026-03-30,west,oops' close (u) end subroutine write_sample subroutine load(path, recs, n, skipped, n_skipped) character(len=*), intent(in) :: path type(record), allocatable, intent(out) :: recs(:) integer, intent(out) :: n character(len=80), allocatable, intent(out) :: skipped(:) integer, intent(out) :: n_skipped integer :: u, ios, line_no character(len=120) :: line type(record) :: r allocate (recs(8), skipped(8)) n = 0 n_skipped = 0 open (newunit=u, file=path, status='old', action='read', iostat=ios) if (ios /= 0) error stop 'load: cannot open the file' line_no = 0 do read (u, '(a)', iostat=ios) line if (ios /= 0) exit line_no = line_no + 1 read (line, *, iostat=ios) r%date, r%region, r%amount if (ios == 0 .and. len_trim(r%date) == date_len .and. r%amount >= 0.0_dp) then call append_record(recs, n, r) else call append_line(skipped, n_skipped, line) end if end do close (u) print '(a,a,a,i0,a,i0,a,i0,a)', 'read ', path, ': ', n, ' records, ', n_skipped, ' lines skipped from ', line_no, ' lines' end subroutine load subroutine append_record(list, n, r) type(record), allocatable, intent(inout) :: list(:) integer, intent(inout) :: n type(record), intent(in) :: r type(record), allocatable :: bigger(:) if (n == size(list)) then allocate (bigger(max(8, 2 * n))) bigger(1:n) = list(1:n) call move_alloc(bigger, list) end if n = n + 1 list(n) = r end subroutine append_record subroutine append_line(list, n, line) character(len=*), allocatable, intent(inout) :: list(:) integer, intent(inout) :: n character(len=*), intent(in) :: line character(len=80), allocatable :: bigger(:) if (n == size(list)) then allocate (bigger(max(8, 2 * n))) bigger(1:n) = list(1:n) call move_alloc(bigger, list) end if n = n + 1 list(n) = line end subroutine append_line end module records program load_demo use records implicit none type(record), allocatable :: recs(:) character(len=80), allocatable :: skipped(:) integer :: n, n_skipped, i call write_sample('sales.csv') call load('sales.csv', recs, n, skipped, n_skipped) print '(a)', 'date region amount' do i = 1, n print '(a10,2x,a8,2x,f9.2)', recs(i)%date, recs(i)%region, recs(i)%amount end do print '(a)', 'skipped:' do i = 1, n_skipped print '(a,a)', ' ', trim(skipped(i)) end do end program load_demo

gfortran -Wall -Wextra -std=f2018 load.f90 -o load
./load

read sales.csv: 12 records, 2 lines skipped from 14 lines
date        region       amount
2026-03-02  east        3050.00
2026-01-11  south        980.00
2026-02-16  north       1890.00
2026-01-04  north       1200.50
2026-03-16  south       1425.60
2026-02-09  east         540.75
2026-01-25  north        760.00
2026-02-16  east         410.00
2026-01-18  east        1430.25
2026-03-09  north        675.25
2026-02-23  south       1120.40
2026-03-23  east         880.00
skipped:
  # a comment line the batch skips
  2026-03-30,west,oops

The lines that do not parse

Two lines of fourteen were skipped, and both are named. The count is printed with the number of lines read, so a reader can see that twelve plus two is fourteen and nothing vanished. A production batch writes the skipped lines to a file of their own and reports the path, so that the person who produced the export can fix it; this one prints them.

The next two lessons take the twelve records further: a sort by date and region, and a report grouped by month with a total per group and a grand total. The module of this page is the file both of them start from.

What comes next

The records are in file order, which is the order the export happened to write. A batch that groups by month needs them ordered by date, and a report that lists regions inside a month needs a second key. Fortran has no sort intrinsic, so lesson 133 writes one: a comparison function over the record type and an insertion sort, which is fast enough for a file of this size and easy to read.

Exercises

  1. Add a fourth field to the file and to the record, and parse it. Which two places have to change?
  2. Make a line with a valid date and a negative amount. Is it a record or a skipped line, and which test decides?
  3. Replace the doubling with a first pass that counts the lines and a second that fills the array. Which version is simpler to read, and which is faster?
  4. Read the file name from the command line. What does the program do when the argument is missing?

Questions

A new set of questions in a new order every time. Choose one answer for each, then check.

  1. How does the loop detect the end of the file?

    Answer

    READ returns a non-zero IOSTAT and the loop exits

    The same status that reports an error ends the loop.

  2. What parses one line into three fields?

    Answer

    A list-directed internal read of the line

    Commas separate the fields without a format.

  3. What does the program do with a line that does not parse?

    Answer

    It keeps the text in a list and prints it at the end

    A batch that stops cannot read a real file.

  4. How many records did the file give?

    Answer

    12 of 14 lines

    One comment and one bad amount were skipped.

  5. What does MOVE_ALLOC do here?

    Answer

    Hands the new storage to the array without copying it twice

    The temporary is emptied by the call.

  6. Why does the array double instead of growing by one?

    Answer

    Growing by one in a loop costs the square of the size

    Each reallocation copies the elements.

  7. Why use NEWUNIT?

    Answer

    It gives a unit number no other part of the program can be using

    No collision with unit 5 or 6.

  8. What does STATUS="OLD" do?

    Answer

    Fails if the file does not exist instead of creating it

    A batch cannot work without its input.

  9. Which test rejects the comment line?

    Answer

    The read itself fails, so IOSTAT is not zero

    A # is not a ten-character date.

  10. Why does the page write its own input file?

    Answer

    So the program carries its data and the page can be run anywhere

    A real batch takes the file from outside.

The programs on this page were built with GNU Fortran (Alpine 15.2.0) 15.2.0 and run on Alpine Linux 3.24.3 (x86_64) in a Docker container.

Article author: Arthur Isaev

Banner image