A test driver and its assertions
An assertion is a comparison with a name. When it holds, the program says so and moves on; when it does not, the program says what was expected and what it found. A driver is a program that runs a list of assertions and counts the failures, and its exit status is the result of the suite. This page writes both by hand, which is the shortest way to see what a test framework does for you and what it does not.
The helper
The helper is one small class with two counters and three methods. Equal compares with EqualityComparer<T>.Default, so it works for numbers, strings and records without a method per type; True is Equal with the expectation fixed; and Throws<TException> passes only when the action throws the named type. The whole program is one listing:
Console.WriteLine("assertions 1.0");
var check = new Check();
check.Equal(4, 2 + 2, "two and two");
check.Equal("AB", "ab".ToUpperInvariant(), "upper case");
check.True(3 > 2, "three is greater than two");
check.Throws<DivideByZeroException>(
() =>
{
var zero = 0;
_ = 1 / zero;
},
"an integer division by zero throws");
Console.WriteLine($"checks {check.Passed} passed {check.Failed} failed");
return check.Failed == 0 ? 0 : 1;
sealed class Check
{
public int Passed { get; private set; }
public int Failed { get; private set; }
public void Equal<T>(T expected, T actual, string name)
{
if (EqualityComparer<T>.Default.Equals(expected, actual))
{
Passed++;
Console.WriteLine($"ok {name}");
return;
}
Failed++;
Console.WriteLine($"FAIL {name}: expected {expected}, got {actual}");
}
public void True(bool value, string name) => Equal(true, value, name);
public void Throws<TException>(Action action, string name) where TException : Exception
{
try
{
action();
}
catch (TException)
{
Passed++;
Console.WriteLine($"ok {name}");
return;
}
Failed++;
Console.WriteLine($"FAIL {name}: nothing of type {typeof(TException).Name} was thrown");
}
}
The program is a top-level program whose return becomes the exit status, which is the same shape as the driver of the log capstone (lesson 119). The helper lives in the same file because a test file of a few hundred lines does not need a library; the next page gives it a project.
A green run
Four assertions of three kinds run, and the summary counts them:
dotnet run
echo "status: $?"
assertions 1.0
ok two and two
ok upper case
ok three is greater than two
ok an integer division by zero throws
checks 4 passed 0 failed
status: 0
The last line before the status is the one a script reads: three of the four assertions are comparisons and one is an exception, and the driver counts all four the same way. The status is zero, so && in a shell would continue.
A red run
The second program is the same file with one expectation changed: it expects five from two plus two. The failing line is the whole point of an assertion helper, because it holds both values:
Console.WriteLine("assertions 1.0");
var check = new Check();
check.Equal(5, 2 + 2, "two and two");
check.Equal("AB", "ab".ToUpperInvariant(), "upper case");
check.True(3 > 2, "three is greater than two");
check.Throws<DivideByZeroException>(
() =>
{
var zero = 0;
_ = 1 / zero;
},
"an integer division by zero throws");
Console.WriteLine($"checks {check.Passed} passed {check.Failed} failed");
return check.Failed == 0 ? 0 : 1;
sealed class Check
{
public int Passed { get; private set; }
public int Failed { get; private set; }
public void Equal<T>(T expected, T actual, string name)
{
if (EqualityComparer<T>.Default.Equals(expected, actual))
{
Passed++;
Console.WriteLine($"ok {name}");
return;
}
Failed++;
Console.WriteLine($"FAIL {name}: expected {expected}, got {actual}");
}
public void True(bool value, string name) => Equal(true, value, name);
public void Throws<TException>(Action action, string name) where TException : Exception
{
try
{
action();
}
catch (TException)
{
Passed++;
Console.WriteLine($"ok {name}");
return;
}
Failed++;
Console.WriteLine($"FAIL {name}: nothing of type {typeof(TException).Name} was thrown");
}
}
dotnet run
echo "status: $?"
assertions 1.0
FAIL two and two: expected 5, got 4
ok upper case
ok three is greater than two
ok an integer division by zero throws
checks 3 passed 1 failed
status: 1
The run does not stop at the failure: the other three assertions still run, which is what a suite wants. The summary says three passed and one failed, and the status is one. A driver that stopped at the first failure would report the first defect only, and a reader would fix it and run again to see the next.
What a framework adds
A framework such as xUnit brings no new kind of assertion. What it brings is around the driver:
- Discovery: a method marked [Fact] is found by reflection, so a suite is a set of methods and not a list in Main.
- Isolation: each test gets a new instance of its class, so one test cannot leave state for the next.
- Reporting: the runner prints the failed test, its message and its stack trace in a form that a continuous integration server reads.
- A filter, a parallel run and a data-driven case, which the xUnit page shows with a real dotnet test.
None of that is needed to test a parser or a calculator, and all of it is wanted in a repository with a thousand tests. The rest of this group shows the hand-written version first and the framework second, so the second has something to be compared with.
Exercises
- Add a NotEqual method and a test that uses it.
- Make Throws print the type of the exception that was thrown when it is the wrong one.
- Add a counter of assertions per kind and print the three totals.
- Change the driver to stop at the first failure and explain what a reader loses.
Questions
A new set of questions in a new order every time. Choose one answer for each, then check.
-
What are the three parts of a hand-written test?
Answer
An assertion, a driver that counts failures, and an exit status
The page builds the three by hand before any framework appears.
-
What does the status of the red run mean?
Answer
One assertion failed, and the driver returned 1
The summary says three passed and one failed, and the status is one.
-
What does this program print?
var expected = 5; var actual = 2 + 2; Console.WriteLine(actual == expected ? "ok" : $"expected {expected}, got {actual}");Answer
expected 5, got 4
That is the message of the failing assertion of the page.
-
Why does Equal compare with EqualityComparer<T>.Default?
Answer
One method then works for numbers, strings and records
The three assertions of the page use the same method on int, string and bool.
-
What does this program print?
var zero = 0; try { _ = 1 / zero; Console.WriteLine("no exception"); } catch (DivideByZeroException) { Console.WriteLine("threw"); }Answer
threw
An integer division by zero throws, unlike the double division of lesson 123.
-
Why does the driver keep going after a failed assertion?
Answer
So one run reports every failure instead of the first one
A suite that stops at the first defect hides the rest.
-
What does this program print?
Console.WriteLine(3 > 2);Answer
True
A bool prints with a capital letter, which the green run shows.
-
Which of these does a framework add around the assertions?
Answer
Discovery, isolation, reporting and a filter
The assertion itself is the same comparison a helper writes.
-
What does this program print?
var passed = 3; var failed = 1; Console.WriteLine($"{passed} passed {failed} failed"); return failed == 0 ? 0 : 1;Answer
3 passed 1 failed
That is the summary of the red run, and the return value is 1.
-
What does Throws<TException> check?
Answer
That the action throws that type, or a type derived from it
A catch of a base type also takes a derived exception, which is why the helper catches TException.
The code on this page was built with the .NET SDK 10.0.303 and run on Alpine Linux 3.24.3 (x86_64) in a Docker container.
Article author: Arthur Isaev