Coverage with coverlet

Coverage answers one question: which lines did the tests execute? The answer is a number per line, and the number is useful and easy to misread. This page runs the coverage collector of the xUnit template, reads the file it writes with a program of its own, and shows three test states in which the line percentage stays at a hundred while the branch percentage does not.

The collector

The test project of the previous page already references coverlet.collector, so the collector is one option of dotnet test and not an install. The file it writes is attached to the run, and the last lines of the output name it:

dotnet test --collect:"XPlat Code Coverage"
echo "status: $?"

  Determining projects to restore...
  All projects are up-to-date for restore.
  Calc -> /tmp/r10/xt/src/Calc/bin/Debug/net10.0/Calc.dll
  tests -> /tmp/r10/xt/tests/bin/Debug/net10.0/tests.dll
Test run for /tmp/r10/xt/tests/bin/Debug/net10.0/tests.dll (.NETCoreApp,Version=v10.0)
A total of 1 test files matched the specified pattern.

Passed!  - Failed:     0, Passed:     5, Skipped:     0, Total:     5, Duration: 101 ms - tests.dll (net10.0)

Attachments:
  /tmp/r10/xt/tests/TestResults/71f7d774-b599-431c-81ad-5bc9b371ee4a/coverage.cobertura.xml

The name of the folder under TestResults is a fresh identifier per run, so a script finds the file with find rather than by writing the path. The format is cobertura XML, which the other .NET tools and the build servers read.

The cobertura file

The root element of the file holds the summary of the whole run, and one attribute of it is the number people quote:

<?xml version="1.0" encoding="utf-8"?> <coverage line-rate="1" branch-rate="1" version="1.9" timestamp="1795258337" lines-covered="4" lines-valid="4" branches-covered="2" branches-valid="2"> <sources>

A reader for it

The file is XML, and the program below turns it into five lines by reading four attributes. It is a console project of its own; the only part worth knowing is XDocument, which is System.Xml.Linq from the base class library:

using System.Xml.Linq; var path = args.Length > 0 ? args[0] : "coverage.cobertura.xml"; if (!File.Exists(path)) { Console.WriteLine("usage: coverage <path of coverage.cobertura.xml>"); return 0; } var root = XDocument.Load(path).Root!; var lines = (int)root.Attribute("lines-valid")!; var covered = (int)root.Attribute("lines-covered")!; var branches = (int)root.Attribute("branches-valid")!; var branchCovered = (int)root.Attribute("branches-covered")!; Console.WriteLine($"file {Path.GetFileName(path)}"); Console.WriteLine($"lines {covered} of {lines}"); Console.WriteLine($"branches {branchCovered} of {branches}"); Console.WriteLine($"line rate {(double)covered / lines:P0}"); Console.WriteLine($"branch rate {(double)branchCovered / Math.Max(branches, 1):P0}"); return 0;

coverage=$(find tests/TestResults -name coverage.cobertura.xml | head -1)
dotnet run -- "$coverage"
echo "status: $?"

file coverage.cobertura.xml
lines 4 of 4
branches 2 of 2
line rate 100 %
branch rate 100 %
status: 0

The percentage is formatted with :P0, and the invariant culture of this container writes it with a space before the sign, which is why the output reads 100 % and not 100%. The division guards the branch count with Math.Max(branches, 1), because a method without a branch has zero valid branches and the rate would be a division by zero. The first check prints a usage line when the file is not there, so the program is also readable when it is started from the wrong folder:

dotnet run
echo "status: $?"

usage: coverage <path of coverage.cobertura.xml>
status: 0

Three states

The library has two methods: Add is one line and Divide is a ternary with two branches. Three versions of the test class, each run with the same collector and the same reader:

TestsLinesBranchesLine rateBranch rate
the addition test only1 of 40 of 225 %0 %
addition and the exception4 of 41 of 2100 %50 %
all five tests4 of 42 of 2100 %100 %

The middle row is the lesson. Every line ran, so a reader who quotes the line rate sees a fully covered library; the true half of the ternary, the division that returns a value, never ran, and only the branch rate says so. A test of the exception proves the error path and says nothing about the path that returns.

The first row is the other direction: a single test that covers a quarter of the lines and none of the branches. A percentage without the two counts behind it cannot be compared between projects, because a library of one method and a library of a thousand reach the same number from very different places.

A green status with no tests

The console test project of the previous page is a program for dotnet run and invisible to dotnet test. Running the test command on it does not fail: it restores, builds nothing that it can run, and reports success:

dotnet test tests/Calc.Tests/Calc.Tests.csproj
echo "status: $?"

  Determining projects to restore...
  All projects are up-to-date for restore.
status: 0

A status of zero with no tests is worse than a red run, because a build server and a person both read it as a passing suite. The test SDK is what makes a project a test project: it adds the entry point that lists the tests in the assembly, and a project without it has no tests to list.

The last page of the group is the reference for the frameworks and libraries around these tests: NUnit, MSTest, Moq, FluentAssertions and BenchmarkDotNet, each installed and run here, with the version that was restored.

Exercises

  1. Add a test that covers the true branch of the division and check the two rates of the middle row.
  2. Delete the exception test and describe the row the table would show.
  3. Print the per-class rates of the cobertura file, which are nested one level below the root.
  4. Run the collector on a project with no tests and explain the file it does or does not write.

Questions

A new set of questions in a new order every time. Choose one answer for each, then check.

  1. What does coverlet.collector write?

    Answer

    A cobertura XML file under TestResults

    The measured run names the coverage.cobertura.xml it attached.

  2. What does a branch rate of 50 % mean?

    Answer

    Half of the branches were taken, for example one side of a ternary

    The middle row of the table has every line and one of two branches.

  3. What does this program print? var covered = 1; var valid = 4; Console.WriteLine($"{(double)covered / valid:P0}");

    Answer

    25 %

    The invariant culture of the container writes a space before the sign.

  4. Why is the line rate 100 % in the middle row while a branch is not taken?

    Answer

    Every line ran, and one branch of the ternary did not

    The page names this as the lesson of the middle row.

  5. What does this program print? var lines = 4; var covered = 4; Console.WriteLine($"{covered} of {lines}");

    Answer

    4 of 4

    That is the line count of the last two rows of the table.

  6. What does dotnet test do on a console project without the test SDK?

    Answer

    It restores, finds no tests and returns zero

    The measured run printed the restore lines and status 0.

  7. Why is a status of zero without tests dangerous?

    Answer

    It looks like a passing suite to a build server and to a person

    The page calls it worse than a red run.

  8. What does this program print? var branches = 0; Console.WriteLine($"{(double)0 / Math.Max(branches, 1):P0}");

    Answer

    0 %

    The guard keeps the division out of zero.

  9. Which attribute holds the number of lines that could run?

    Answer

    lines-valid

    The covered count is the other attribute of the pair.

  10. Which test state has the best branch rate?

    Answer

    The five tests, with 2 of 2 branches

    The last row of the table is the only one at 100 % on both rates.

The code on this page was built with the .NET SDK 10.0.303 and run on Alpine Linux 3.24.3 (x86_64) in a Docker container.

Article author: Arthur Isaev