The Python wrapper with ctypes

The Python side is one module of eighty-four lines that loads the library and describes every function to ctypes. It uses nothing but the standard library: no numpy, no f2py, no compiler at import time. The description of the four functions is the whole of the interface, and it is the part that has to match the Fortran side or the program produces a wrong number instead of an error.

The wrapper

The file python/numerics.py:

"""A ctypes wrapper around the Fortran statistics core. The core is built from ``core/stats.f90`` into ``libstats.so`` beside this project's root. Every function takes a sequence of floats, copies it into a C array and calls the shared library; the Fortran side never holds a reference after the call returns. """ from __future__ import annotations import ctypes import pathlib from collections.abc import Sequence LIBRARY_PATH = pathlib.Path(__file__).resolve().parent.parent / "libstats.so" class CoreError(RuntimeError): """Raised when the Fortran core reports a failure through its status code.""" def _load() -> ctypes.CDLL: """Load the shared library and describe every function it exports.""" if not LIBRARY_PATH.is_file(): message = f"{LIBRARY_PATH} is missing: run build.sh first" raise FileNotFoundError(message) lib = ctypes.CDLL(str(LIBRARY_PATH)) lib.stats_mean.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.c_int] lib.stats_mean.restype = ctypes.c_double lib.stats_sd.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.c_int] lib.stats_sd.restype = ctypes.c_double lib.stats_median.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.c_int] lib.stats_median.restype = ctypes.c_double lib.stats_quantile.argtypes = [ ctypes.POINTER(ctypes.c_double), ctypes.c_int, ctypes.c_double, ctypes.POINTER(ctypes.c_double), ] lib.stats_quantile.restype = ctypes.c_int return lib _LIB = _load() def _as_array(values: Sequence[float]) -> ctypes.Array[ctypes.c_double]: """Copy a sequence into a C array of doubles, refusing an empty one.""" if len(values) == 0: message = "at least one value is required" raise ValueError(message) return (ctypes.c_double * len(values))(*values) def mean(values: Sequence[float]) -> float: """Return the arithmetic mean of the values.""" array = _as_array(values) return float(_LIB.stats_mean(array, len(values))) def sd(values: Sequence[float]) -> float: """Return the sample standard deviation, with n - 1 in the divisor.""" array = _as_array(values) return float(_LIB.stats_sd(array, len(values))) def median(values: Sequence[float]) -> float: """Return the median of the values.""" array = _as_array(values) return float(_LIB.stats_median(array, len(values))) def quantile(values: Sequence[float], q: float) -> float: """Return the quantile at ``q``, linearly interpolated between neighbours.""" if not 0.0 <= q <= 1.0: message = f"q must be between 0 and 1, got {q}" raise ValueError(message) array = _as_array(values) out = ctypes.c_double(0.0) status = _LIB.stats_quantile(array, len(values), q, ctypes.byref(out)) if status != 0: message = f"the core reported status {status} for q={q} and n={len(values)}" raise CoreError(message) return float(out.value)

ARGTYPES and RESTYPE

ctypes.CDLL loads the file and gives an object whose attributes are the exported symbols; the four names are the four BIND(C) names from the previous page, and nothing else is reachable. The load happens at import, once, and a missing library fails there with a FileNotFoundError that says to run the build, which is a better error than an OSError from the loader three functions later.

Each function then gets two declarations. argtypes is a list with one entry per C parameter, and it is what makes ctypes convert a Python list element to a double and pass the address of the array. restype is what the function returns: c_double for the three that compute, c_int for the one that reports a status. A missing restype makes ctypes assume an int and return the contents of an integer register, which is the trap the course lesson demonstrates.

The array is built by (ctypes.c_double * n)(*values) in _as_array, and the length goes in as a plain Python int because ctypes converts it to the c_int the Fortran side declared with VALUE. The output argument of stats_quantile is a ctypes.c_double() passed with ctypes.byref, which is the address the INTENT(OUT) dummy expects.

Two kinds of refusal

The wrapper refuses two things in Python before it calls anything. An empty sequence is a ValueError, because the Fortran side declares X(N) and a zero length would be an empty array it cannot average; and a quantile outside zero to one is a ValueError too, because that is a mistake in the calling code and a traceback with the value in it is the fastest fix.

The third refusal is the core's, and it is the interesting one: stats_quantile returns a status, and the wrapper turns a non-zero status into a CoreError. That is the C convention translated into the Python one at the boundary, and it means a caller in the service can catch (ValueError, CoreError) and answer 400 without knowing which side refused.

The unittest suite

Nine cases in three classes: the mean and its empty input, the spread (standard deviation and median), and the quantile with its two ends and its bad argument. Every expected value is written to twelve decimal places, which is the tolerance a double-precision result can hold.

The file python/test_numerics.py:

"""Tests of the ctypes wrapper, run with unittest.""" from __future__ import annotations import unittest import numerics VALUES = [2.0, 4.0, 4.0, 5.0] class MeanTests(unittest.TestCase): """The mean, including the empty input the core cannot take.""" def test_mean_of_four_values(self) -> None: """The mean of 2 4 4 5 is 3.75.""" self.assertAlmostEqual(numerics.mean(VALUES), 3.75, places=12) def test_mean_of_one_value_is_the_value(self) -> None: """A series of one value is its own mean.""" self.assertAlmostEqual(numerics.mean([7.5]), 7.5, places=12) def test_empty_input_is_refused(self) -> None: """An empty sequence fails in the wrapper, before the call.""" with self.assertRaises(ValueError): numerics.mean([]) class SpreadTests(unittest.TestCase): """The standard deviation and the median.""" def test_sd_of_four_values(self) -> None: """The sample standard deviation of 2 4 4 5 is 1.2583057392117916.""" self.assertAlmostEqual(numerics.sd(VALUES), 1.2583057392117916, places=12) def test_median_of_an_even_count(self) -> None: """The median of an even count is the mean of the middle two.""" self.assertAlmostEqual(numerics.median(VALUES), 4.0, places=12) def test_median_of_an_odd_count(self) -> None: """The median of 3 1 2 is 2, and the input order does not matter.""" self.assertAlmostEqual(numerics.median([3.0, 1.0, 2.0]), 2.0, places=12) class QuantileTests(unittest.TestCase): """The quantile and the status code the core returns.""" def test_median_quantile_is_the_median(self) -> None: """The 0.5 quantile is the median for this series.""" self.assertAlmostEqual(numerics.quantile(VALUES, 0.5), 4.0, places=12) def test_quantile_of_the_ends(self) -> None: """Quantile 0 is the minimum and quantile 1 the maximum.""" self.assertAlmostEqual(numerics.quantile(VALUES, 0.0), 2.0, places=12) self.assertAlmostEqual(numerics.quantile(VALUES, 1.0), 5.0, places=12) def test_quantile_outside_the_range_is_refused(self) -> None: """A q outside 0 to 1 is a ValueError in the wrapper.""" with self.assertRaises(ValueError): numerics.quantile(VALUES, 1.5) if __name__ == "__main__": unittest.main()

cd python
python3 -m unittest -v test_numerics

test_empty_input_is_refused (test_numerics.MeanTests.test_empty_input_is_refused)
An empty sequence fails in the wrapper, before the call. ... ok
test_mean_of_four_values (test_numerics.MeanTests.test_mean_of_four_values)
The mean of 2 4 4 5 is 3.75. ... ok
test_mean_of_one_value_is_the_value (test_numerics.MeanTests.test_mean_of_one_value_is_the_value)
A series of one value is its own mean. ... ok
test_median_quantile_is_the_median (test_numerics.QuantileTests.test_median_quantile_is_the_median)
The 0.5 quantile is the median for this series. ... ok
test_quantile_of_the_ends (test_numerics.QuantileTests.test_quantile_of_the_ends)
Quantile 0 is the minimum and quantile 1 the maximum. ... ok
test_quantile_outside_the_range_is_refused (test_numerics.QuantileTests.test_quantile_outside_the_range_is_refused)
A q outside 0 to 1 is a ValueError in the wrapper. ... ok
test_median_of_an_even_count (test_numerics.SpreadTests.test_median_of_an_even_count)
The median of an even count is the mean of the middle two. ... ok
test_median_of_an_odd_count (test_numerics.SpreadTests.test_median_of_an_odd_count)
The median of 3 1 2 is 2, and the input order does not matter. ... ok
test_sd_of_four_values (test_numerics.SpreadTests.test_sd_of_four_values)
The sample standard deviation of 2 4 4 5 is 1.2583057392117916. ... ok

----------------------------------------------------------------------
Ran 9 tests in 0.002s

OK

The expected numbers are the ones the Fortran test asserted, which is the point of testing both sides: the two suites agree on the same literals, so a change of rounding in one language would show up as a failing test in the other.

The numpy route

The usual alternative to this wrapper is f2py, which reads the Fortran source and generates the ctypes layer, and the usual way to write these tests with numpy is numpy.testing.assert_allclose. The file python/test_numpy_route.py is the second opinion: it computes the mean and the standard deviation with numpy and compares them with the wrapper's, so a mistake that the Fortran core and its test share still shows up as a difference.

import numpy as np from numpy.testing import assert_allclose import numerics def test_mean_against_numpy() -> None: values = [2.0, 4.0, 4.0, 5.0] assert_allclose(numerics.mean(values), np.mean(values), rtol=1e-12) assert_allclose(numerics.sd(values), np.std(values, ddof=1), rtol=1e-12)

The run was made in a second disposable container of the same image, with the same project directory, the library built by ./build.sh and network access for the one install. numpy 2.5.5 went into a virtual environment at /tmp/v, outside the project, so the project itself still needs only the standard library and none of its files changed:

python3 -m venv /tmp/v
/tmp/v/bin/pip install numpy

From the python directory, the interpreter of that environment printed the version it holds and then ran the test:

/tmp/v/bin/python -c "import numpy; print('numpy', numpy.__version__)"
/tmp/v/bin/python -c "
import test_numpy_route as t
t.test_mean_against_numpy(); print('numpy route: ok')"

numpy 2.5.5
numpy route: ok

The assertion passed with rtol=1e-12 on the mean and on the standard deviation with ddof=1, against the Fortran-backed wrapper. That is what the route is for: an independent implementation of the two formulas agrees with the library to twelve significant digits, which says more than the literal values of the unittest suite, whose expected numbers were written by the same person as the algorithm. What the run does not test is f2py: it was not run here, and the ctypes layer on this page is hand-written. f2py ships with numpy, so a project that wants to compare the generated layer with this one has it in the same virtual environment.

Exercises

  1. Wrap the same library in a class that copies the values into one array and reuses it. Which of the nine tests would have to change?
  2. Delete the restype of stats_mean and run the tests. What does mean return, and why is that not an exception?
  3. Add a quantile test with q = 0.25 and a five-value series. What is the interpolated answer, and does the Fortran side agree?
  4. Time the wrapper on a series of a million values and compare it with a Python loop. Which part of the time is the ctypes call, and which is the copy?

The programs on this page were built with GNU Fortran (Alpine 15.2.0) 15.2.0 and run with Python 3.14.10 on Alpine Linux 3.24.3 (x86_64) in a Docker container. The numpy run used numpy 2.5.5 with Python 3.14.10, installed with pip in a virtual environment outside the project, in a second container of the same image.

Article author: Arthur Isaev

Banner image