Disabled tests and display names

A test method name is for the compiler, but the report is for a person. JUnit has @DisplayName for the sentence a reader should see, and @Disabled for a test that is known to be wrong and must not fail the build. Both change the tree, and the second changes the counts.

@DisplayName

The annotation takes a string and replaces the method name in the tree. It does not change the method name in the failure list, which still needs the real name, and it does not change the class name. A display name is a sentence, with spaces and lower case if the sentence reads better that way.

The display name can contain spaces, punctuation and Unicode; it cannot contain a newline unless the string does. It is not a replacement for a good method name: both are read, one by the compiler and one by the person.

@Disabled

A method marked @Disabled is discovered and shown in the tree, but it is not started. The optional string is the reason, and the report prints it next to the method with a small arrow. A test disabled without a reason is a debt with no note; the reason is what lets the next person decide whether to fix or delete it.

The annotation also works on a class, which disables every test in it. That is useful while a whole fixture is being rewritten, and dangerous when it stays for a month: the tests disappear from the counts and nobody notices.

import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.fail; import org.junit.jupiter.api.Disabled; import org.junit.jupiter.api.DisplayName; import org.junit.jupiter.api.Test; public class ReportTest { @Test @DisplayName("the sum of two and three") void adds() { assertEquals(5, 2 + 3); } @Test @Disabled("waiting for the new parser") void parsesNewSyntax() { fail("not implemented yet"); } }

The report

javac -Xlint:all -cp /opt/java-tools/junit-platform-console-standalone.jar -d out ReportTest.java
junit execute --class-path out --scan-classpath --disable-ansi-colors

Thanks for using JUnit! Support its development at https://junit.org/sponsoring

╷
├─ JUnit Platform Suite ✔
├─ JUnit Jupiter ✔
│  └─ ReportTest ✔
│     ├─ the sum of two and three ✔
│     └─ parsesNewSyntax() ↷ waiting for the new parser
└─ JUnit Vintage ✔

Test run finished after 310 ms
[         4 containers found      ]
[         0 containers skipped    ]
[         4 containers started    ]
[         0 containers aborted    ]
[         4 containers successful ]
[         0 containers failed     ]
[         2 tests found           ]
[         1 tests skipped         ]
[         1 tests started         ]
[         0 tests aborted         ]
[         1 tests successful      ]
[         0 tests failed          ]

Skipped against failed

The table counts the disabled test as found and skipped, not as started. A skipped test does not fail the run, so the status is 0. A failed test is found, started and failed, and the status is 1. The two words are the difference between not run on purpose and run and wrong, and a report that mixes them hides the second.

The display name appears only for the enabled test. The disabled line keeps the method name, because the reason is attached to the method the reader will find in the source.

When to disable a test

A disabled test is a promise to come back. Use it when the code is being rewritten and the test describes the new behaviour before it exists, or when an external service the test needs is gone. Do not use it to make a red build green: the red test is the information, and disabling it throws the information away. A tag, a separate suite or a shorter test is usually the better answer, and the next parts cover them.

Exercises

  1. Remove @Disabled and run. What happens to the counts and the status?
  2. Disable the whole class with the annotation on the class line and read the tree. How many tests are skipped?
  3. Change the display name to a sentence with a comma and a number. Does the tree print it exactly?
  4. Add a second enabled test and a second disabled test. Predict the found, skipped and successful counts before you run.

Questions

A new set of questions in a new order every time. Choose one answer for each, then check.

  1. How does a disabled test appear in the counts table?

    Answer

    Found and skipped, not started

    The test is discovered but not started, so it adds to found and skipped only.

  2. What does @DisplayName change?

    Answer

    The name in the tree

    The display name is for the reader of the report; the method keeps its real name.

  3. Why give @Disabled a reason?

    Answer

    It tells the next person why the test is off and when to turn it back on

    The reason is the note that turns a silent skip into a decision.

  4. What does this program print? public class Q1 { public static void main(String[] args) { System.out.println(2 + 3); } }

    Answer

    5

    The two integers are added, so println prints 5.

  5. What does this program print? public class Q2 { public static void main(String[] args) { String text = "JUnit"; System.out.println(text.toUpperCase()); } }

    Answer

    JUNIT

    toUpperCase changes every letter to upper case.

  6. What does this program print? public class Q3 { public static void main(String[] args) { int count = 0; for (int n = 1; n != 4; n++) { count = count + n; } System.out.println(count); } }

    Answer

    6

    The loop adds 1, 2 and 3, and stops when n is 4.

  7. What does this program print? public class Q4 { public static void main(String[] args) { System.out.println("a" + "b" + "c"); } }

    Answer

    abc

    The three strings are joined in order without separators.

  8. A test is disabled with a reason. What status does the run return?

    Answer

    0, because a skipped test is not a failure

    Skipping is a deliberate non-run, and the launcher returns 0 when nothing failed.

  9. When is disabling a test the wrong choice?

    Answer

    When it is done only to hide a red build

    Disabling hides information; a red test that is simply switched off loses its value.

  10. What does this program print? public class Q5 { public static void main(String[] args) { String name = "Ada"; System.out.println(name.length() + 2); } }

    Answer

    5

    The name has three characters, and 3 + 2 is 5.

The code on this page was built with OpenJDK 21.0.13 (javac) and run with JUnit Platform Console Launcher 1.14.3 on Alpine Linux in a Docker container.

Article author: Arthur Isaev