Reports and exit status
A person reads the console report; a machine reads a file. The launcher writes both: the tree and the table on standard output, and one XML file per engine in a folder named with --reports-dir. This lesson writes the files and reads the status a script uses to stop a build.
The test class
One class with one test, so the report is short and the files are easy to name.
import static org.junit.jupiter.api.Assertions.assertEquals;
import org.junit.jupiter.api.Test;
public class CalcTest {
@Test
void adds() {
assertEquals(4, 2 + 2);
}
}
--reports-dir
The option names a folder. The launcher creates it if needed and writes a file per engine that ran. The console report is unchanged, which is why the same command serves a person at the terminal and a build server that reads the folder.
javac -Xlint:all -cp /opt/java-tools/junit-platform-console-standalone.jar -d out CalcTest.java
junit execute --class-path out --scan-classpath --reports-dir reports --disable-ansi-colors
Thanks for using JUnit! Support its development at https://junit.org/sponsoring
╷
├─ JUnit Platform Suite ✔
├─ JUnit Jupiter ✔
│ └─ CalcTest ✔
│ └─ adds() ✔
└─ JUnit Vintage ✔
Test run finished after 388 ms
[ 4 containers found ]
[ 0 containers skipped ]
[ 4 containers started ]
[ 0 containers aborted ]
[ 4 containers successful ]
[ 0 containers failed ]
[ 1 tests found ]
[ 0 tests skipped ]
[ 1 tests started ]
[ 0 tests aborted ]
[ 1 tests successful ]
[ 0 tests failed ]
The XML files
The folder holds three files for this run: TEST-junit-jupiter.xml, TEST-junit-platform-suite.xml and TEST-junit-vintage.xml. Each is the XML form of one engine's result, with a test suite element per class, a test case element per test, the time, and a failure element with the message and the stack when there is one. A build server reads the file and draws its own table; the console report is for the person at the terminal.
The names begin with TEST- because that is the convention JUnit uses, and a CI job is usually configured to collect TEST-*.xml. The folder is a build artifact: it is deleted with the output, not committed.
The exit status
A green run returns 0; a run with at least one failed test returns 1. Skipped and aborted tests do not change that: the run is green when nothing failed. A script writes the launcher command, checks $?, and only packages when it is 0.
The status is the coarse answer, the report is the detail. A build that only reads the status knows whether to stop, not what to fix; the XML file is what a dashboard reads to show the failing class.
In a script
The shape of a job is small: compile, run with a reports folder, publish the folder, fail the job with the status. The image of this course has no build server, so the commands are run by hand here; a Maven project gets the same two steps from Surefire, which writes the same XML names. The loop without a build tool shows the status of a red run in the middle of a fix.
Exercises
- Delete the reports folder, run again, and list the files that come back.
- Make the test fail and run with the same command. Which status does the shell see, and what does the XML failure element contain?
- Disable the test and run. Which status does the shell see now?
- Run with --reports-dir reports --details=tree and compare the console output. Does the XML change?
Questions
A new set of questions in a new order every time. Choose one answer for each, then check.
-
Which option writes the XML reports?
Answer
--reports-dir
The reports folder is named with --reports-dir, and the launcher writes one file per engine.
-
What do the file names begin with?
Answer
TEST-
The JUnit convention prefixes the engine file with TEST-, as CI collectors expect.
-
What status does a run with one skipped test and no failure return?
Answer
0
A skipped test did not fail, so the run is green.
-
What does this program print?
public class Q1 { public static void main(String[] args) { System.out.println("TEST-" + "junit-jupiter" + ".xml"); } }Answer
TEST-junit-jupiter.xml
The three strings are joined in order, and the dot is part of the last one.
-
What does this program print?
public class Q2 { public static void main(String[] args) { System.out.println(4 == 2 + 2); } }Answer
true
The sum on the right is 4, so the equality is true.
-
What does this program print?
public class Q3 { public static void main(String[] args) { System.out.println("reports".endsWith("s")); } }Answer
true
The word ends with s, so the check is true.
-
What does this program print?
public class Q4 { public static void main(String[] args) { System.out.println("xml".toUpperCase()); } }Answer
XML
toUpperCase changes every letter to upper case.
-
Why is the reports folder not committed?
Answer
It is a build artifact that changes on every run
The folder is regenerated by the build and belongs with the output, not the source.
-
What does a build server usually read?
Answer
The XML files, and the status of the command
The machine reads TEST-*.xml for detail and the exit status for the pass or fail decision.
-
What does this program print?
public class Q5 { public static void main(String[] args) { System.out.println(2 + 2); } }Answer
4
The two integers are added, and println prints 4.
The code on this page was built with OpenJDK 21.0.13 (javac) and run with JUnit Platform Console Launcher 1.14.3 on Alpine Linux in a Docker container.
Article author: Arthur Isaev