Exceptions in tests
A method that throws is part of its contract, and a test has to check that it throws the right thing for the right input. A try and a catch inside a test would do it, but a test that passes when nothing is thrown is the failure this subject is famous for. JUnit gives the assertion a lambda, so the expected exception is stated once and the framework fails the test when the call returns normally.
The code that throws
Two methods with two kinds of failure. parse wraps the NumberFormatException of Integer.parseInt in one with a message that names the input, and divide refuses zero with an IllegalArgumentException. Both messages are part of what the tests check:
public class Parser {
static int parse(String text) {
try {
return Integer.parseInt(text.trim());
} catch (NumberFormatException e) {
throw new NumberFormatException("not a number: " + text);
}
}
static int divide(int a, int b) {
if (b == 0) {
throw new IllegalArgumentException("division by zero");
}
return a / b;
}
public static void main(String[] args) {
System.out.println("parse: " + parse(" 42 "));
System.out.println("divide: " + divide(10, 4));
try {
parse("x");
} catch (NumberFormatException e) {
System.out.println("caught: " + e.getMessage());
}
}
}
The main prints the message the code wrote for the bad input, which is the value the tests assert. A message that a person cannot act on is a message the test should not check; the one here names the text, so a failure of the library tells a reader which input broke it.
assertThrows
assertThrows takes the expected class and an executable, which is a lambda with no arguments whose body may throw. It runs the lambda, checks the class of what came out and answers the exception object, so the test can go on to check the message. The call that should not throw is not written inside a try at all:
The type matters. NumberFormatException is a subclass of IllegalArgumentException, and a test that expects the parent passes for the child as well. That is Java's rule for a catch block and the assertion follows it, so a test that wants exactly the child names the child.
The message and the cause
The exception the assertion answers is the one the code threw, so getMessage() is the sentence of the contract. getCause() is the exception that was wrapped, when the code wrapped one; a test that checks the cause is testing that the original failure is not lost on the way out.
assertDoesNotThrow
The opposite assertion exists because the interesting half of a contract is often the call that must work. assertDoesNotThrow(() -> Parser.parse("7")) fails when the lambda throws anything, and it answers the value when the executable is a ThrowingSupplier, which lets the test check it. The form with a Runnable answers nothing.
assertAll
Five assertions in two test methods would stop at the first failure, which hides the second. assertAll takes several executables, runs all of them and throws one exception that lists every failure, so a run of a broken class reports all the broken cases at once:
import static org.junit.jupiter.api.Assertions.assertAll;
import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;
import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;
@TestMethodOrder(MethodOrderer.MethodName.class)
public class ParserTest {
@Test
void parsesANumberWithSpaces() {
assertEquals(42, Parser.parse(" 42 "));
}
@Test
void refusesAText() {
NumberFormatException error = assertThrows(NumberFormatException.class, () -> Parser.parse("x"));
assertEquals("not a number: x", error.getMessage());
}
@Test
void refusesADivisionByZero() {
IllegalArgumentException error = assertThrows(IllegalArgumentException.class, () -> Parser.divide(1, 0));
assertEquals("division by zero", error.getMessage());
}
@Test
void acceptsAValidNumber() {
assertDoesNotThrow(() -> Parser.parse("7"));
}
@Test
void checksBothMessages() {
assertAll(
() -> assertEquals("not a number: y", assertThrows(NumberFormatException.class, () -> Parser.parse("y")).getMessage()),
() -> assertEquals("division by zero", assertThrows(IllegalArgumentException.class, () -> Parser.divide(5, 0)).getMessage()));
}
}
The parallel form is assertAll("messages", streamOfExecutables), which runs the executables in parallel and collects the failures the same way. The names matter: an Executable may throw a checked exception, so the lambda of a checked call needs no catch, which is the point of the type.
The green run
The program first, then the five tests of ParserTest, all of which hold:
javac -Xlint:all -cp /opt/java-tools/junit-platform-console-standalone.jar -d out Parser.java ParserTest.java
java -cp out Parser
parse: 42
divide: 2
caught: not a number: x
junit execute --class-path out --select-class ParserTest --disable-ansi-colors
Thanks for using JUnit! Support its development at https://junit.org/sponsoring
╷
├─ JUnit Platform Suite ✔
├─ JUnit Jupiter ✔
│ └─ ParserTest ✔
│ ├─ acceptsAValidNumber() ✔
│ ├─ checksBothMessages() ✔
│ ├─ parsesANumberWithSpaces() ✔
│ ├─ refusesADivisionByZero() ✔
│ └─ refusesAText() ✔
└─ JUnit Vintage ✔
Test run finished after 400 ms
[ 4 containers found ]
[ 0 containers skipped ]
[ 4 containers started ]
[ 0 containers aborted ]
[ 4 containers successful ]
[ 0 containers failed ]
[ 5 tests found ]
[ 0 tests skipped ]
[ 5 tests started ]
[ 0 tests aborted ]
[ 5 tests successful ]
[ 0 tests failed ]
The exception that never comes
A test of a failure must fail when the failure does not happen, and the clearest way to see that is to write the wrong test on purpose. QuietTest expects a NumberFormatException from the parse of a good number, so the assertion has nothing to catch and the framework reports it:
import static org.junit.jupiter.api.Assertions.assertThrows;
import org.junit.jupiter.api.Test;
public class QuietTest {
@Test
void expectsAnExceptionThatNeverComes() {
assertThrows(NumberFormatException.class, () -> Parser.parse("42"), "42 is a number");
}
}
javac -Xlint:all -cp /opt/java-tools/junit-platform-console-standalone.jar:out -d out QuietTest.java
junit execute --class-path out --select-class QuietTest --disable-ansi-colors
echo "status: $?"
Thanks for using JUnit! Support its development at https://junit.org/sponsoring
╷
├─ JUnit Platform Suite ✔
├─ JUnit Jupiter ✔
│ └─ QuietTest ✔
│ └─ expectsAnExceptionThatNeverComes() ✘ 42 is a number ==> Expected java.lang.NumberFormatException to be thrown, but nothing was thrown.
└─ JUnit Vintage ✔
Failures (1):
JUnit Jupiter:QuietTest:expectsAnExceptionThatNeverComes()
MethodSource [className = 'QuietTest', methodName = 'expectsAnExceptionThatNeverComes', methodParameterTypes = '']
=> org.opentest4j.AssertionFailedError: 42 is a number ==> Expected java.lang.NumberFormatException to be thrown, but nothing was thrown.
org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:152)
org.junit.jupiter.api.AssertThrows.assertThrows(AssertThrows.java:73)
org.junit.jupiter.api.AssertThrows.assertThrows(AssertThrows.java:39)
org.junit.jupiter.api.Assertions.assertThrows(Assertions.java:3153)
QuietTest.expectsAnExceptionThatNeverComes(QuietTest.java:8)
java.base/java.lang.reflect.Method.invoke(Method.java:580)
java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
Test run finished after 424 ms
[ 4 containers found ]
[ 0 containers skipped ]
[ 4 containers started ]
[ 0 containers aborted ]
[ 4 containers successful ]
[ 0 containers failed ]
[ 1 tests found ]
[ 0 tests skipped ]
[ 1 tests started ]
[ 0 tests aborted ]
[ 0 tests successful ]
[ 1 tests failed ]
status: 1
The sentence Expected java.lang.NumberFormatException to be thrown, but nothing was thrown is the whole value of the assertion. A hand-written try with a fail() after it says the same thing only if the author remembers the fail(), and the version in the framework cannot be forgotten.
What a test of an exception is
Three questions, one assertion each: the class of the exception, the message, and the cause when there is one. The class is the contract, the message is what a person reads, and the cause is what a program that logs the failure will need. A test that checks only the class still passes when the message becomes useless, which is why the pages of this group check the two together.
An exception the code throws for a case the program should have prevented is a bug, and a test that expects it is a test that locks the bug in. The distinction is the subject of the exception design lesson, and it is the reason the messages here name the input: a test asserts what the caller can act on.
What comes next
The next page is about files: a test that writes one needs a folder it may write to, and @TempDir gives it a folder of its own and deletes it afterwards. The page after that puts the sources and the tests in the layout a build tool expects and compiles them with two commands.
The Java course covers the exceptions themselves from lesson 096, and the message of an assertion is part of lesson 127.
The code on this page was compiled with OpenJDK 21.0.13 (javac) and run with the JUnit Platform Console Launcher 1.14.3 on Alpine Linux in a Docker container. The time in each report is that machine's and changes between runs, and the line numbers of the framework frames belong to the version of the jar.
Article author: Arthur Isaev