Capstone A: testing the parser
The parser of this capstone has one job and many inputs: a line with a level it knows, a line cut short, a level nobody defined, a time in the wrong shape. A table of such lines with the expected answer is a test suite, and a loop over the table is the runner. No package is needed, which matters here: xUnit, NUnit and MSTest are NuGet packages, and this container has no network.
A table of cases
Each row of the table is a line and the answer the parser must give for it. The driver numbers the rows, compares the answer with the expectation and counts the rows that differ. The whole program is one listing:
var cases = new (string Line, bool Parses)[]
{
("2024-03-10T09:30:00Z INFO started", true),
("2024-03-10T09:30:01Z WARN slow request ms=120", true),
("not a log line", false),
("2024-03-10T09:30:02Z TRACE noisy", false),
("2024-03-10T09:30:02Z ERROR failed id=7", true),
};
var failed = 0;
var number = 0;
foreach (var (line, expected) in cases)
{
number++;
var actual = TryParse(line, out var entry);
var ok = actual == expected;
if (!ok)
{
failed++;
}
Console.WriteLine($"{(ok ? "ok " : "FAIL")} {number} {expected,-5} {actual,-5} [{line}]");
}
Console.WriteLine($"cases {cases.Length} failed {failed}");
return failed == 0 ? 0 : 1;
static bool TryParse(string line, out LogLine entry)
{
entry = default!;
var parts = line.Split(' ', 3, StringSplitOptions.RemoveEmptyEntries);
if (parts.Length < 3)
{
return false;
}
if (!DateTime.TryParseExact(parts[0], "yyyy-MM-ddTHH:mm:ssZ", null, System.Globalization.DateTimeStyles.AdjustToUniversal, out _))
{
return false;
}
if (parts[1] is not ("INFO" or "WARN" or "ERROR"))
{
return false;
}
entry = new LogLine(parts[0], parts[1], parts[2]);
return true;
}
record LogLine(string At, string Level, string Message);
The loop prints one line per case with the expectation and the answer next to each other, so a failure is readable without a debugger. The last line is the summary, and the status of the process is the summary in a form a script can read:
dotnet run
echo "status: $?"
ok 1 True True [2024-03-10T09:30:00Z INFO started]
ok 2 True True [2024-03-10T09:30:01Z WARN slow request ms=120]
ok 3 False False [not a log line]
ok 4 False False [2024-03-10T09:30:02Z TRACE noisy]
ok 5 True True [2024-03-10T09:30:02Z ERROR failed id=7]
cases 5 failed 0
status: 0
The driver
The parser is the TryParse of lesson 114, unchanged in shape: a static method with an out parameter that answers false for a line it cannot read. return failed == 0 ? 0 : 1; is the whole policy of the driver: zero when every case passed, one when any case failed. A shell reads the same number, and so does a continuous integration server.
- The table is a tuple array, so a case is one line of source and the compiler checks the types of both members.
- The fourth case is the one that matters most: a well-formed line with a level outside the three known ones is a line the parser must refuse.
- The count of cases and the count of failures are both printed; a driver that prints only failures cannot tell a suite that passed from a suite that ran no cases at all.
A case that fails
A test suite is only trusted after it has been seen to fail. The second program is the same driver with one expectation deliberately changed: it expects the line with the unknown level to parse. The parser refuses it, so the fourth case fails, the summary counts one failure and the status is one:
var cases = new (string Line, bool Parses)[]
{
("2024-03-10T09:30:00Z INFO started", true),
("2024-03-10T09:30:01Z WARN slow request ms=120", true),
("not a log line", false),
("2024-03-10T09:30:02Z TRACE noisy", true),
("2024-03-10T09:30:02Z ERROR failed id=7", true),
};
var failed = 0;
var number = 0;
foreach (var (line, expected) in cases)
{
number++;
var actual = TryParse(line, out var entry);
var ok = actual == expected;
if (!ok)
{
failed++;
}
Console.WriteLine($"{(ok ? "ok " : "FAIL")} {number} {expected,-5} {actual,-5} [{line}]");
}
Console.WriteLine($"cases {cases.Length} failed {failed}");
return failed == 0 ? 0 : 1;
static bool TryParse(string line, out LogLine entry)
{
entry = default!;
var parts = line.Split(' ', 3, StringSplitOptions.RemoveEmptyEntries);
if (parts.Length < 3)
{
return false;
}
if (!DateTime.TryParseExact(parts[0], "yyyy-MM-ddTHH:mm:ssZ", null, System.Globalization.DateTimeStyles.AdjustToUniversal, out _))
{
return false;
}
if (parts[1] is not ("INFO" or "WARN" or "ERROR"))
{
return false;
}
entry = new LogLine(parts[0], parts[1], parts[2]);
return true;
}
record LogLine(string At, string Level, string Message);
dotnet run
echo "status: $?"
ok 1 True True [2024-03-10T09:30:00Z INFO started]
ok 2 True True [2024-03-10T09:30:01Z WARN slow request ms=120]
ok 3 False False [not a log line]
FAIL 4 True False [2024-03-10T09:30:02Z TRACE noisy]
ok 5 True True [2024-03-10T09:30:02Z ERROR failed id=7]
cases 5 failed 1
status: 1
The red run is the proof that the driver can fail. A suite whose every row passes on the first run may be a suite of rows that ask nothing.
A later group of pages on this site takes testing further: an assertion helper, a property-style check with a seeded generator, a golden file and what dotnet test does when its packages cannot be restored. The next part is the second capstone, the expression parser, and its last lesson is a table like this one.
Exercises
- Add a case for a line with a correct level and no message, and decide what the parser should answer before you run it.
- Print the total number of cases that were expected to fail and the number that did, so the summary reads 5 cases, 2 expected failures, 0 failed.
- Make the driver take the table from a text file, one case per line, and report the file line number of a failure.
- Turn the status into one, two or three: one for a failed case, two for a missing input, three for an unreadable one.
Questions
A new set of questions in a new order every time. Choose one answer for each, then check.
-
Why is a test driver a plain program in this course?
Answer
xUnit, NUnit and MSTest are NuGet packages, and the container has no network
A table and a loop need nothing that is not in the SDK.
-
What does the driver return when every case passes?
Answer
0
The measured green run ends with status 0.
-
What does this program print?
var cases = new[] { 1, 2, 3 }; var failed = cases.Count(x => x > 2); Console.WriteLine(failed); return failed == 0 ? 0 : 1;Answer
1
One value is above two, so the driver would fail.
-
Which case fails in the red run?
Answer
The fourth, a line with the unknown level TRACE, which the parser refuses
The expectation says True and the parser answers False.
-
What does this program print?
var ok = false; Console.WriteLine($"{(ok ? "ok " : "FAIL")} case");Answer
FAIL case
The condition is false, so the second branch is printed.
-
Why print the count of cases next to the count of failures?
Answer
A suite that ran no cases would otherwise look green
Five cases and no failure is different from no cases and no failure.
-
What does this program print?
string[] parts = "2024 INFO started".Split(' ', 3, StringSplitOptions.RemoveEmptyEntries); Console.WriteLine(parts.Length);Answer
3
The limit of three keeps the message whole, and all three parts are filled.
-
What does a failing case prove about the driver?
Answer
That it can tell a wrong answer from a right one
A suite never seen to fail has not been shown to test anything.
-
What does this program print?
var expected = true; var actual = false; Console.WriteLine(expected == actual);Answer
False
The two values differ, so the comparison is false.
-
Where does the driver get its exit status from?
Answer
A return from the top-level statements
The return value of the entry point becomes the status.
The code on this page was built with the .NET SDK 10.0.303 and run on Alpine Linux 3.24.3 (x86_64) in a Docker container.
Article author: Arthur Isaev