Capstone 3: reading a record file
The third capstone is the other half of a Fortran programmer's work: not a solver but a batch that reads a file somebody else wrote, turns every line into a record, sorts the records and writes a report. It is where most of the I/O of part 5 is used at once, and where the two decisions that decide whether a batch survives contact with real data are made: what to do with a line that does not parse, and how to grow an array whose size is not known until the file has been read.
The record file
The file is comma-separated with three fields: a date, a region and an amount. It has fourteen lines, of which twelve are records, one is a comment and one has a field that is not a number. That is a realistic mix: a real export from a spreadsheet has a title row, a blank line at the end and a row somebody typed by hand.
The file sales.csv, which the program writes before it reads it so that the page runs anywhere:
2026-03-02,east,3050.00
2026-01-11,south,980.00
2026-02-16,north,1890.00
2026-01-04,north,1200.50
2026-03-16,south,1425.60
2026-02-09,east,540.75
2026-01-25,north,760.00
2026-02-16,east,410.00
2026-01-18,east,1430.25
2026-03-09,north,675.25
2026-02-23,south,1120.40
2026-03-23,east,880.00
# a comment line the batch skips
2026-03-30,west,oops
A real batch takes the file from its command line or from a fixed name and never writes it. The WRITE_SAMPLE procedure exists so that this page and its program carry their own input, which is the same rule the data-file lessons of the course follow.
The twelve records arrive in the order the export wrote them, which is neither sorted by date nor grouped by region, and one date, 2026-02-16, appears twice with two different regions. That is deliberate: the next lesson sorts this file by date and then by region, and a file that is already in order would show nothing.
Reading with IOSTAT
The loop reads one line at a time with READ (u, '(a)', iostat=ios) line and stops on any non-zero status, which is how the end of the file arrives. IOSTAT rather than IOSTAT_END is enough here because every non-zero value ends the loop; a program that has to tell a read error from the end of the file compares the value with IOSTAT_END, as lesson 057 shows.
The open uses NEWUNIT, which gives a unit number that no other part of the program can be using, and STATUS='OLD', which fails if the file is not there instead of creating an empty one. The failure is a ERROR STOP with a message: a batch that cannot open its input has nothing to do.
One line at a time
The line is parsed with a list-directed internal read, the same trick as lesson 058: READ (line, *, iostat=ios) r%date, r%region, r%amount. List-directed input takes a comma as a separator, so the three fields arrive without a format, and the status tells the program whether the line had three fields of the right kinds. The comment line fails because its first field is not a date of ten characters, and the last line fails because oops is not a number.
Two conditions decide whether a parsed line is a record: ios == 0 .and. len_trim(r%date) == 10 .and. r%amount >= 0.0_dp. The length test rejects a short first field, which list-directed input would accept as a short string, and the amount test rejects a negative value that the batch does not expect. Every other line goes into the list of skipped lines with its text, so the report can print it. That is the decision this page is really about: a batch that stops at the first bad line cannot process a real file, and a batch that ignores bad lines silently loses records.
Growing the array
The number of records is not known until the file has been read, so the array starts at eight and doubles when it is full. MOVE_ALLOC hands the new storage over without copying it twice: the temporary bigger receives the old elements, and the call makes list point at the new block and deallocates the temporary, which is now empty. Reallocating the array on the left of an assignment would copy the elements twice and, in a loop, cost the square of the size.
The same procedure is written twice, once for the records and once for the skipped lines, because Fortran has no generic over two different types without an interface. A project that needed more would make a generic interface with two specific procedures, as lesson 066 shows.
The run
The file load.f90, the module and a program that writes the sample, loads it and prints what it found:
module records
use, intrinsic :: iso_fortran_env, only: dp => real64
implicit none
private
public :: record, load, write_sample
integer, parameter :: date_len = 10, region_len = 8
type :: record
character(len=date_len) :: date = ' '
character(len=region_len) :: region = ' '
real(dp) :: amount = 0.0_dp
end type record
contains
subroutine write_sample(path)
character(len=*), intent(in) :: path
integer :: u
open (newunit=u, file=path, status='replace', action='write')
write (u, '(a)') '2026-03-02,east,3050.00'
write (u, '(a)') '2026-01-11,south,980.00'
write (u, '(a)') '2026-02-16,north,1890.00'
write (u, '(a)') '2026-01-04,north,1200.50'
write (u, '(a)') '2026-03-16,south,1425.60'
write (u, '(a)') '2026-02-09,east,540.75'
write (u, '(a)') '2026-01-25,north,760.00'
write (u, '(a)') '2026-02-16,east,410.00'
write (u, '(a)') '2026-01-18,east,1430.25'
write (u, '(a)') '2026-03-09,north,675.25'
write (u, '(a)') '2026-02-23,south,1120.40'
write (u, '(a)') '2026-03-23,east,880.00'
write (u, '(a)') '# a comment line the batch skips'
write (u, '(a)') '2026-03-30,west,oops'
close (u)
end subroutine write_sample
subroutine load(path, recs, n, skipped, n_skipped)
character(len=*), intent(in) :: path
type(record), allocatable, intent(out) :: recs(:)
integer, intent(out) :: n
character(len=80), allocatable, intent(out) :: skipped(:)
integer, intent(out) :: n_skipped
integer :: u, ios, line_no
character(len=120) :: line
type(record) :: r
allocate (recs(8), skipped(8))
n = 0
n_skipped = 0
open (newunit=u, file=path, status='old', action='read', iostat=ios)
if (ios /= 0) error stop 'load: cannot open the file'
line_no = 0
do
read (u, '(a)', iostat=ios) line
if (ios /= 0) exit
line_no = line_no + 1
read (line, *, iostat=ios) r%date, r%region, r%amount
if (ios == 0 .and. len_trim(r%date) == date_len .and. r%amount >= 0.0_dp) then
call append_record(recs, n, r)
else
call append_line(skipped, n_skipped, line)
end if
end do
close (u)
print '(a,a,a,i0,a,i0,a,i0,a)', 'read ', path, ': ', n, ' records, ', n_skipped, ' lines skipped from ', line_no, ' lines'
end subroutine load
subroutine append_record(list, n, r)
type(record), allocatable, intent(inout) :: list(:)
integer, intent(inout) :: n
type(record), intent(in) :: r
type(record), allocatable :: bigger(:)
if (n == size(list)) then
allocate (bigger(max(8, 2 * n)))
bigger(1:n) = list(1:n)
call move_alloc(bigger, list)
end if
n = n + 1
list(n) = r
end subroutine append_record
subroutine append_line(list, n, line)
character(len=*), allocatable, intent(inout) :: list(:)
integer, intent(inout) :: n
character(len=*), intent(in) :: line
character(len=80), allocatable :: bigger(:)
if (n == size(list)) then
allocate (bigger(max(8, 2 * n)))
bigger(1:n) = list(1:n)
call move_alloc(bigger, list)
end if
n = n + 1
list(n) = line
end subroutine append_line
end module records
program load_demo
use records
implicit none
type(record), allocatable :: recs(:)
character(len=80), allocatable :: skipped(:)
integer :: n, n_skipped, i
call write_sample('sales.csv')
call load('sales.csv', recs, n, skipped, n_skipped)
print '(a)', 'date region amount'
do i = 1, n
print '(a10,2x,a8,2x,f9.2)', recs(i)%date, recs(i)%region, recs(i)%amount
end do
print '(a)', 'skipped:'
do i = 1, n_skipped
print '(a,a)', ' ', trim(skipped(i))
end do
end program load_demo
gfortran -Wall -Wextra -std=f2018 load.f90 -o load
./load
read sales.csv: 12 records, 2 lines skipped from 14 lines
date region amount
2026-03-02 east 3050.00
2026-01-11 south 980.00
2026-02-16 north 1890.00
2026-01-04 north 1200.50
2026-03-16 south 1425.60
2026-02-09 east 540.75
2026-01-25 north 760.00
2026-02-16 east 410.00
2026-01-18 east 1430.25
2026-03-09 north 675.25
2026-02-23 south 1120.40
2026-03-23 east 880.00
skipped:
# a comment line the batch skips
2026-03-30,west,oops
The lines that do not parse
Two lines of fourteen were skipped, and both are named. The count is printed with the number of lines read, so a reader can see that twelve plus two is fourteen and nothing vanished. A production batch writes the skipped lines to a file of their own and reports the path, so that the person who produced the export can fix it; this one prints them.
The next two lessons take the twelve records further: a sort by date and region, and a report grouped by month with a total per group and a grand total. The module of this page is the file both of them start from.
What comes next
The records are in file order, which is the order the export happened to write. A batch that groups by month needs them ordered by date, and a report that lists regions inside a month needs a second key. Fortran has no sort intrinsic, so lesson 133 writes one: a comparison function over the record type and an insertion sort, which is fast enough for a file of this size and easy to read.
Exercises
- Add a fourth field to the file and to the record, and parse it. Which two places have to change?
- Make a line with a valid date and a negative amount. Is it a record or a skipped line, and which test decides?
- Replace the doubling with a first pass that counts the lines and a second that fills the array. Which version is simpler to read, and which is faster?
- Read the file name from the command line. What does the program do when the argument is missing?
Questions
A new set of questions in a new order every time. Choose one answer for each, then check.
-
How does the loop detect the end of the file?
Answer
READ returns a non-zero IOSTAT and the loop exits
The same status that reports an error ends the loop.
-
What parses one line into three fields?
Answer
A list-directed internal read of the line
Commas separate the fields without a format.
-
What does the program do with a line that does not parse?
Answer
It keeps the text in a list and prints it at the end
A batch that stops cannot read a real file.
-
How many records did the file give?
Answer
12 of 14 lines
One comment and one bad amount were skipped.
-
What does MOVE_ALLOC do here?
Answer
Hands the new storage to the array without copying it twice
The temporary is emptied by the call.
-
Why does the array double instead of growing by one?
Answer
Growing by one in a loop costs the square of the size
Each reallocation copies the elements.
-
Why use NEWUNIT?
Answer
It gives a unit number no other part of the program can be using
No collision with unit 5 or 6.
-
What does STATUS="OLD" do?
Answer
Fails if the file does not exist instead of creating it
A batch cannot work without its input.
-
Which test rejects the comment line?
Answer
The read itself fails, so IOSTAT is not zero
A # is not a ten-character date.
-
Why does the page write its own input file?
Answer
So the program carries its data and the page can be run anywhere
A real batch takes the file from outside.
The programs on this page were built with GNU Fortran (Alpine 15.2.0) 15.2.0 and run on Alpine Linux 3.24.3 (x86_64) in a Docker container.
Article author: Arthur Isaev