pFUnit, test-drive and fpm: not run here

A Fortran team that grows past a module of assertions wants a framework, and the four names it meets are pFUnit, test-drive, fortran-stdlib and fpm test. None of them is installed in this container, so none of them is run on this page. What the page does show is what each one is, the real command that proves this image does not have it, and the route to install it on a machine where that is allowed. The pages before this one are the alternative: what a project can do with the compiler, gcov, valgrind and a shell.

The four projects

pFUnit is the oldest of them and the one a large project is most likely to have. A test is a Fortran procedure with a specification comment above it, a Python program reads the specification and generates the suite that calls the procedures, and a runner executes the suite and reports pass, fail and skip per test. It knows about MPI, about parameterised tests and about a fixture that runs before and after each test, and it is installed with CMake and a Fortran compiler.

test-drive is the opposite end: one module to drop into the project, an include file with a few macros and a check subroutine. It has no runner of its own — the program is the runner — and it gives the three things a hand-written module lacks: a message with the file and line, a summary that a shell can read, and a check that can be called with a condition, a message or both.

fortran-stdlib is a community standard library for Fortran: sorting, strings, statistics, a hash map and more, each with its own tests. A project that uses it gets the library and a model of how a Fortran test suite is laid out; it is not a test framework, and a project that only wants assertions does not need it.

fpm is the Fortran package manager, and fpm test is one of its commands: it builds every program under test/ against the project and runs it, which is the runner that the other three leave to the user. A project that already uses fpm to build gets a test command for free; a project with a Makefile does not.

The real commands

Two commands are enough to show the state of this image, and both are real output:

command -v fpm; echo $?

(no output)
1

command -v prints the path of a program and exits 1 when there is none, so an empty line and a status of 1 mean that fpm is not on the path. The second is the documented way to ask the compiler where a library is, the same one lesson 101 uses for OpenCoarrays:

gfortran -print-file-name=libpfunit.a
gfortran -print-file-name=libtest-drive.a

libpfunit.a
libtest-drive.a

A library that is present comes back as a full path. A bare name, echoed unchanged, means the library is not there. The same answer is given for both frameworks, and there is no fortran-stdlib anywhere on the system either.

The install route

This page cannot install them, and it does not pretend to. The route for each is short enough to describe, and the commands are the ones the projects document, not ones run here:

The four routes have the same shape: build with the compiler the project uses, and keep the version in the repository. A testing framework that is built with a different compiler from the code under test brings its own problems, which is the argument for the module of the first page of this group in a project that cannot install anything.

What this image has

The tools that are here cover everything the previous pages did: gfortran with its checks and traps, gcov for coverage, valgrind for memory, gdb for a session, make for the target, and a POSIX shell for the runner. The compiler directory also holds the small runtime archives that a Fortran program links against, including libgcov.a for coverage and libcaf_single.a for coarrays on one image, which is what lessons 098 to 101 build against.

That set is enough for the three questions a test suite asks: is the answer right (assertions with tolerances), did the tests run all the code (gcov), and is the program itself broken (the checked build and valgrind). A framework changes how the first question is spelled and how the answers are reported; it does not add a fourth question.

When to install one

Install a framework when the suite outgrows the module: when tests need fixtures that build and free a database, when a failure has to name a file and a line without the writer passing it, when the report has to be machine-readable for a dashboard, or when the team wants the tests written before the code and a specification language helps. Until then the module, the runner and the flags of this group are less work and have no dependency to keep current.

The honest summary of this page is one sentence: the frameworks are good, this image does not have them, and everything the group demonstrates was done without them.

Exercises

  1. Look at the header of test-drive and write the three lines a project needs to use it. Which file goes in the source directory, and which in the include path?
  2. Write the same three tests in your own module and in test-drive's check style. Which version prints the file and the line on failure?
  3. A project builds with fpm on a laptop and with make on a cluster. What would fpm test have to call to run the same suite in both places?
  4. Which of the four projects would you add to a code that must build with nothing but a Fortran compiler? Why?

The programs on this page were built with GNU Fortran (Alpine 15.2.0) 15.2.0 and run on Alpine Linux 3.24.3 (x86_64) in a Docker container. The commands that show the frameworks are absent were run in the same container; no output of pFUnit, test-drive, fortran-stdlib or fpm is shown anywhere on this page, because none of them was run.

Article author: Arthur Isaev