Floating point and delta

A double does not hold a decimal fraction exactly, so a test that compares two calculated double values for equality is a test that fails at random. JUnit has a second form of assertEquals for this case: it takes a delta, the largest difference the test accepts. This lesson runs both forms on the same addition and reads what each one says.

The two tests

One test class, two methods, the same addition. The first method compares the result with 0.3 exactly; the second accepts a difference of at most 1e-9.

import static org.junit.jupiter.api.Assertions.assertEquals; import org.junit.jupiter.api.Test; public class FloatTest { @Test void addsTenthsExactly() { assertEquals(0.3, 0.1 + 0.2); } @Test void addsTenthsWithADelta() { assertEquals(0.3, 0.1 + 0.2, 1e-9); } }

The exact comparison

The three-argument call is the exact overload. It compares the two doubles bit for bit, so it asks whether 0.1 + 0.2 produced the same value as the literal 0.3. It did not: the sum is the nearest double to 0.30000000000000004, and the literal is the nearest double to 0.3, and those are two different doubles.

The report

javac -Xlint:all -cp /opt/java-tools/junit-platform-console-standalone.jar -d out FloatTest.java
junit execute --class-path out --scan-classpath --disable-ansi-colors

Thanks for using JUnit! Support its development at https://junit.org/sponsoring

╷
├─ JUnit Platform Suite ✔
├─ JUnit Jupiter ✔
│  └─ FloatTest ✔
│     ├─ addsTenthsWithADelta() ✔
│     └─ addsTenthsExactly() ✘ expected: <0.3> but was: <0.30000000000000004>
└─ JUnit Vintage ✔

Failures (1):
  JUnit Jupiter:FloatTest:addsTenthsExactly()
    MethodSource [className = 'FloatTest', methodName = 'addsTenthsExactly', methodParameterTypes = '']
    => org.opentest4j.AssertionFailedError: expected: <0.3> but was: <0.30000000000000004>
       org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
       org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
       org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
       org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:70)
       org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:65)
       org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:889)
       FloatTest.addsTenthsExactly(FloatTest.java:8)
       java.base/java.lang.reflect.Method.invoke(Method.java:580)
       java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
       java.base/java.util.ArrayList.forEach(ArrayList.java:1596)

Test run finished after 303 ms
[         4 containers found      ]
[         0 containers skipped    ]
[         4 containers started    ]
[         0 containers aborted    ]
[         4 containers successful ]
[         0 containers failed     ]
[         2 tests found           ]
[         0 tests skipped         ]
[         2 tests started         ]
[         0 tests aborted         ]
[         1 tests successful      ]
[         1 tests failed          ]

The delta method passed while the exact method failed, on the same expression. The report counts two tests: one successful, one failed, and the failure names the exact overload.

The delta overload

assertEquals(expected, actual, delta) passes when Math.abs(expected - actual) <= delta. The third argument is the tolerance, in the same unit as the values. 1e-9 means a billionth, which is far larger than the difference between 0.3 and 0.30000000000000004 (about 5.5e-17) and far smaller than any difference a test should catch.

The overload also exists for float, and the delta has to be a float there. A bare 0.1f + 0.2f compared with 0.3f has the same problem in a smaller number of bits.

Choosing a delta

A delta is a statement about the precision the result needs, not a way to hide a bug. For money in cents, compare integers; for a physical measurement, use the precision of the instrument; for a mathematical result, use the error the algorithm promises. A delta larger than the difference the test is meant to detect turns the test green for the wrong reason.

A special case is Double.NaN: it is not equal to itself, so assertEquals(Double.NaN, result) fails even when the result is NaN. assertEquals(Double.NaN, result, 0.0) passes, because the delta form treats two NaNs as equal. Use it when NaN is the expected answer.

Exercises

  1. Change the delta to 1e-20 and run. Does the delta test still pass? What is the smallest delta that passes?
  2. Print the difference with System.out.println(0.1 + 0.2 - 0.3) in a plain program and compare it with the delta.
  3. Write a test for 0.1f + 0.2f with a float delta and find a value that passes.
  4. Add a test that expects Double.NaN with and without a delta. Which one passes?

Questions

A new set of questions in a new order every time. Choose one answer for each, then check.

  1. Why does assertEquals(0.3, 0.1 + 0.2) fail?

    Answer

    The sum is the nearest double to 0.30000000000000004, not to 0.3

    Binary floating point cannot represent 0.1, 0.2 or 0.3 exactly, and the rounded sum differs from the rounded literal.

  2. What does the third argument of assertEquals mean?

    Answer

    The largest difference the test accepts

    The delta is a tolerance: the test passes when the absolute difference is at most the delta.

  3. Which delta is reasonable for a sum of doubles that should be 0.3?

    Answer

    1e-9

    A billionth is far above the rounding error and far below a real difference.

  4. What does this program print? public class Q1 { public static void main(String[] args) { System.out.println(0.1 + 0.2); } }

    Answer

    0.30000000000000004

    The nearest double to the sum is printed with its shortest exact representation.

  5. What does this program print? public class Q2 { public static void main(String[] args) { System.out.println(Math.abs(0.1 + 0.2 - 0.3)); } }

    Answer

    5.551115123125783E-17

    The difference is about 5.5 times 10 to the minus 17.

  6. What does this program print? public class Q3 { public static void main(String[] args) { System.out.println(1e-9 == 0.000000001); } }

    Answer

    true

    The two literals are the same number, so the comparison is true.

  7. What does this program print? public class Q4 { public static void main(String[] args) { double value = 2.5; System.out.println(value == 2.5); } }

    Answer

    true

    2.5 is exactly representable in binary, so the comparison is true.

  8. Why is Double.NaN special in an equality test?

    Answer

    NaN is not equal to itself, so the exact overload fails on it

    The IEEE rules make NaN unequal to everything, including another NaN; the delta form treats two NaNs as equal.

  9. What does a delta larger than the bug you want to catch do?

    Answer

    It hides the bug and the test passes

    The test only checks that the difference is within the tolerance, so a large tolerance accepts a wrong result.

  10. What does this program print? public class Q5 { public static void main(String[] args) { System.out.println(Math.abs(0.5 - 0.25)); } }

    Answer

    0.25

    The difference between 0.5 and 0.25 is 0.25, and both are exactly representable.

The code on this page was built with OpenJDK 21.0.13 (javac) and run with JUnit Platform Console Launcher 1.14.3 on Alpine Linux in a Docker container.

Article author: Arthur Isaev