Real arithmetic, rounding and numeric inquiry
A Fortran real is a binary floating point number: a sign, an exponent and a mantissa of 24 or 53 bits. Almost everything surprising about real arithmetic follows from those two numbers, and the language gives a program the tools to see them. This lesson prints the value a real really holds, measures the gap between neighbouring values, compares two reals correctly, and shows an operation whose answer depends on the precision it is computed in.
Single and double
The program below names the two kinds through iso_fortran_env and prints the same decimal value stored at each precision. It then asks for the machine epsilon of both, the smallest value and the largest value of both, and the decimal digits and range each one guarantees.
The file realarith.f90:
program realarith
use iso_fortran_env, only: sp => real32, dp => real64
implicit none
real(sp) :: a
real(dp) :: b
a = 0.1_sp
b = 0.1_dp
print *, "single 0.1 :", a
print *, "double 0.1 :", b
print *, "0.1 + 0.2 == 0.3 :", 0.1_sp + 0.2_sp == 0.3_sp
print *, "double same test :", 0.1_dp + 0.2_dp == 0.3_dp
print *, "single eps :", epsilon(a)
print *, "double eps :", epsilon(b)
print *, "single tiny/huge :", tiny(a), huge(a)
print *, "double tiny/huge :", tiny(b), huge(b)
print *, "precision/range :", precision(a), range(a), precision(b), range(b)
end program realarith
gfortran -Wall -Wextra -std=f2018 realarith.f90 -o realarith
./realarith
single 0.1 : 0.100000001
double 0.1 : 0.10000000000000001
0.1 + 0.2 == 0.3 : T
double same test : F
single eps : 1.19209290E-07
double eps : 2.2204460492503131E-016
single tiny/huge : 1.17549435E-38 3.40282347E+38
double tiny/huge : 2.2250738585072014E-308 1.7976931348623157E+308
precision/range : 6 37 15 307
Read the first two lines together. One tenth has no exact binary representation, so the stored value is the nearest binary fraction, and the two precisions store different fractions: the single prints seven digits of its error, the double sixteen. The comparison in the third and fourth line then goes two ways, and that is the most useful surprise on the page: in single precision the two roundings happen to cancel, and in double they do not. 0.1 + 0.2 == 0.3 is therefore not a test of arithmetic but a test of a particular rounding, and a program that depends on its answer is a program with a bug waiting for a compiler upgrade.
Epsilon and spacing
Two intrinsics measure the gap between a value and its neighbours, and they are not the same number:
- epsilon(x) is the gap at 1.0. For a single it is about 1.19e-7, for a double about 2.22e-16, and both are fixed for the kind.
- spacing(x) is the gap at x itself. It grows with the exponent, because the mantissa has a fixed number of bits and the exponent moves the binary point.
A value of a million stored as a single has a spacing of one sixteenth, so a change smaller than that is lost the moment it is stored. This is the fact behind the next section, and spacing is the way to check it instead of guessing.
HUGE, TINY, PRECISION and RANGE
Four intrinsics describe a kind without printing a single value of it:
- huge(x) is the largest finite value, 3.4e38 for a single and 1.8e308 for a double.
- tiny(x) is the smallest positive normal value.
- precision(x) is the number of decimal digits that survive the round trip, 6 and 15 here.
- range(x) is the decimal exponent range, 37 and 307.
A program that prints these four at start-up documents its own numerics, and a results table with that header can be reproduced. The numbers are the same ones selected_real_kind takes as requirements, which is not a coincidence: the two functions answer the same question from the two directions.
Comparing reals
Equality between reals is what a reader writes first and what a reviewer removes. The program below computes 0.1 + 0.2 in double, compares it with 0.3, and then compares the difference with a tolerance instead.
The file compare.f90:
program compare
use iso_fortran_env, only: dp => real64
implicit none
real(dp) :: x, y
x = 0.1_dp + 0.2_dp
y = 0.3_dp
print *, "x == y :", x == y
print *, "x - y :", x - y
print *, "abs(x-y) small:", abs(x - y) < 1.0e-12_dp
print *, "spacing y :", spacing(y)
print *, "100 * epsilon :", 100.0_dp * epsilon(y)
end program compare
gfortran -Wall -Wextra -std=f2018 compare.f90 -o compare
compare.f90:8:29:
8 | print *, "x == y :", x == y
| 1
Warning: Equality comparison for REAL(8) at (1) [-Wcompare-reals]
The warning is worth knowing by name: -Wcompare-reals is part of -Wextra here, so a program built with the course flags is told about every equality test between reals. The build continues, and the run shows why the warning exists:
./compare
x == y : F
x - y : 5.5511151231257827E-017
abs(x-y) small: T
spacing y : 5.5511151231257827E-017
100 * epsilon : 2.2204460492503131E-014
The difference is exactly one spacing(y): the two computations landed on adjacent representable values. The tolerance test is true, and it is true for the right reason: it asks whether the two numbers are closer than a threshold a reader chose, which is a question about the program rather than about the last bit. A tolerance of 100 * epsilon is the usual starting point, and each operation in a longer calculation spends some of it.
Cancellation
The last surprise is not a rounding in the last digit: it is a whole digit lost because two large numbers cancel. Add one to a hundred million, subtract the hundred million, and the answer is the one that was added, if the precision held it on the way in.
The file cancel.f90:
program cancel
use iso_fortran_env, only: sp => real32, dp => real64
implicit none
real(sp) :: a, c
real(dp) :: d, e
a = 1.0e8_sp
c = (a + 1.0_sp) - a
d = 1.0e8_dp
e = (d + 1.0_dp) - d
print *, "single (a + 1) - a:", c
print *, "double (d + 1) - d:", e
end program cancel
gfortran -Wall -Wextra -std=f2018 cancel.f90 -o cancel
./cancel
single (a + 1) - a: 0.00000000
double (d + 1) - d: 1.0000000000000000
At a hundred million a single has a spacing of eight, so adding one changes nothing and the subtraction returns zero. The double has room for sixteen digits and returns the one. Both programs are correct Fortran and both answer the same mathematical question; only one of them had enough bits.
The habit that follows: choose the precision from the range of the data and the number of digits the answer needs, not from the size of the binary. A sum of a million small values in single precision loses the small ones once the running total is large; the same sum in double does not, and a Kahan summation in single can recover most of the loss, which the lesson on numerical stability covers.
Exercises
- Print spacing(1.0) and spacing(1.0e8) as a single. How many times larger is the second?
- Change the comparison in the first program to abs(0.1 + 0.2 - 0.3) < 1.0e-6 and rebuild. Does the warning appear?
- Compute the sum of 0.1 ten thousand times in single and in double. Which result is further from 1000?
- Print nearest(1.0, 2.0) - 1.0 and compare it with epsilon(1.0).
Questions
A new set of questions in a new order every time. Choose one answer for each, then check.
-
Why is 0.1 not stored exactly in a binary real?
Answer
One tenth has no finite binary expansion, so the nearest fraction is stored
The stored value is the nearest binary fraction to one tenth, and the error shows in the print.
-
What does
epsilon(1.0)measure?Answer
The gap between 1.0 and the next representable value
It is the machine epsilon, about 1.19e-7 for a single.
-
What does
spacing(x)give thatepsilondoes not?Answer
The gap between neighbouring values at x itself
The gap grows with the exponent, so it depends on where x is.
-
Which of these is true in double precision?
print *, 0.1_dp + 0.2_dp == 0.3_dpAnswer
False: the two sums land on different binary values
In double the roundings differ by one spacing; in single they happen to cancel.
-
What warning does gfortran give for an equality comparison of reals?
Answer
Warning: Equality comparison for REAL(8) [-Wcompare-reals]
The -Wcompare-reals check is part of the warning set the course uses.
-
What does this program print?
program p implicit none print *, huge(1.0) > 1.0e38 end program pAnswer
T
The largest single is 3.4e38, which is larger than 1.0e38.
-
What is
(1.0e8 + 1.0) - 1.0e8in single precision?Answer
0.0, because the spacing at 1e8 is larger than 1
A single has 24 bits of mantissa, so adding one to a hundred million changes nothing.
-
What does
precision(1.0)return on this compiler?Answer
6
It is the decimal digits of the single; 24 is digits, the binary count.
-
What does
tiny(1.0_dp)give?Answer
The smallest positive normal double, about 2.2e-308
tiny is the smallest normal value of the kind; epsilon is the gap at 1.
-
Why is a tolerance comparison better than equality for computed reals?
Answer
It asks whether the numbers are close enough for the program, not whether they match bit for bit
Equality of computed reals depends on rounding order; a tolerance states the requirement.
The programs on this page were built with GNU Fortran (Alpine 15.2.0) 15.2.0 and run on Alpine Linux 3.24.3 (x86_64) in a Docker container.
Article author: Arthur Isaev