Capstone C: measuring the pool
A pool of workers is worth its complexity only if the work it does comes back faster, and the only way to know is to measure the same jobs with one worker and with four. The benchmark of this lesson runs eight jobs of twenty milliseconds five times with each pool size, and it prints the minimum and the median rather than the average, because one slow run would move an average and a reader cannot tell how much.
The measurement
Stopwatch around one complete run of the pool, OrderBy over the five times for the minimum and the third element for the median, and the same checksum after every run so a fast run that lost a job is visible. The whole program is one listing:
using System.Diagnostics;
using System.Threading.Channels;
Console.WriteLine("benchmark 1.0");
var jobs = Enumerable.Range(1, 8).Select(id => new Job(id, 20)).ToArray();
foreach (var workers in new[] { 1, 4 })
{
var times = new List<double>();
var total = 0L;
for (var run = 1; run <= 5; run++)
{
var watch = Stopwatch.StartNew();
total = await Run(workers, jobs);
watch.Stop();
times.Add(watch.Elapsed.TotalMilliseconds);
Console.WriteLine($"workers {workers} run {run} {watch.Elapsed.TotalMilliseconds:F2} ms");
}
var sorted = times.OrderBy(time => time).ToList();
Console.WriteLine($"workers {workers} min {sorted[0]:F2} median {sorted[2]:F2} total {total}");
}
static async Task<long> Run(int workers, Job[] jobs)
{
var queue = Channel.CreateBounded<Job>(8);
var total = 0L;
var producer = Task.Run(async () =>
{
foreach (var job in jobs)
{
await queue.Writer.WriteAsync(job);
}
queue.Writer.Complete();
});
var pool = Enumerable.Range(1, workers).Select(_ => Task.Run(async () =>
{
await foreach (var job in queue.Reader.ReadAllAsync())
{
await Task.Delay(job.DelayMs);
Interlocked.Add(ref total, job.Id);
}
})).ToArray();
await producer;
await Task.WhenAll(pool);
return total;
}
record Job(int Id, int DelayMs);
dotnet run
echo "status: $?"
benchmark 1.0
workers 1 run 1 209.18 ms
workers 1 run 2 178.87 ms
workers 1 run 3 165.04 ms
workers 1 run 4 167.47 ms
workers 1 run 5 162.03 ms
workers 1 min 162.03 median 167.47 total 36
workers 4 run 1 44.83 ms
workers 4 run 2 40.65 ms
workers 4 run 3 40.73 ms
workers 4 run 4 43.44 ms
workers 4 run 5 40.66 ms
workers 4 min 40.65 median 40.73 total 36
status: 0
The numbers
The measured table of this container, one run of the program above:
| Workers | Runs (ms) | Min | Median | Checksum |
|---|---|---|---|---|
| 1 | 209.18, 178.87, 165.04, 167.47, 162.03 | 162.03 | 167.47 | 36 |
| 4 | 44.83, 40.65, 40.73, 43.44, 40.66 | 40.65 | 40.73 | 36 |
The machine, measured with nproc and /etc/os-release:
| Item | Value |
|---|---|
| CPU | 4 cores, x86_64 |
| System | Alpine Linux 3.24.3 in a Docker container |
| Runtime | .NET SDK 10.0.303, runtime 10.0.15 |
| Configuration | Debug build, dotnet run, no arguments |
Four workers took about a quarter of the time of one, and the first run of each pool size was the slowest of its five: 209.18 against 162.03 for one worker, 44.83 against 40.65 for four. The first run pays for the JIT, the type initialisers and the first task starts, which is the warm-up lesson 102 describes; that is why a benchmark runs the code more than once and reports the minimum next to the median.
The floor of eight jobs
Eight jobs of twenty milliseconds are a hundred and sixty milliseconds of waiting. One worker cannot do better than that, and its minimum of 162.03 ms is as close to the floor as a measurement gets. Four workers divide the eight jobs into two each, so the floor is forty milliseconds, and the measured 40.65 ms is again the floor plus the cost of starting and ending the tasks.
- The jobs of this capstone wait with Task.Delay and do no arithmetic, so the pool scales with the number of jobs and not with the number of cores. Eight workers would finish in about twenty milliseconds.
- The checksum is 36 in both rows, because every run adds the ids one to eight. A faster run that printed another number would have lost a job, which is the failure mode of a pool and the reason the total is printed next to the time.
- A median is the third of five. With an even number of runs the median is the average of the two middle ones, and the code would need that line.
When the work is CPU
A pool of Task objects over a channel of waiting jobs is not the same as a pool of threads over arithmetic. A job that computes without awaiting holds its thread for the whole computation, and four workers on four cores are the most the machine can run at once; a fifth worker waits for a core and adds switching, not throughput. The next group of measurements in the performance part of this course, lessons 102 to 108, is the other half of the same question.
The capstone ends here: a model, a queue, a pool, a result, a shutdown, a logger and a measurement. The next lesson is the last part of the course, the one that says where to go from C#, and what this Linux container cannot run.
Exercises
- Run the benchmark with eight workers and with two, and add both rows to the table.
- Give the jobs different delays and check whether the floor of the run is the longest delay or the sum divided by the workers.
- Replace the waiting with a loop that adds a million numbers and run the benchmark again; describe why four workers are now close to one.
- Print the average next to the median and say which of the two the first run moves more.
Questions
A new set of questions in a new order every time. Choose one answer for each, then check.
-
Why does the benchmark print the minimum and the median and not the average?
Answer
One slow run moves an average, and the minimum and median show the usual cost
The first run of each size was the slowest of its five.
-
What is the floor of one worker over eight jobs of 20 ms?
Answer
160 ms
The measured minimum is 162.03 ms.
-
What does this program print?
var times = new[] { 209.18, 178.87, 165.04, 167.47, 162.03 }; Console.WriteLine(times.Min());Answer
162.03
The minimum of the five runs of one worker.
-
Why is the first run of a pool size slower?
Answer
It pays for the JIT, the type initialisers and the first task starts
The measured first run is the slowest of each five.
-
What does this program print?
var times = new[] { 3.0, 1.0, 2.0 }.OrderBy(time => time).ToList(); Console.WriteLine(times[times.Count / 2]);Answer
2
The sorted middle of three values is the median.
-
Why would eight workers not take a quarter of the time of four?
Answer
Four cores run four workers at once, and the jobs are two per worker
The floor with four workers is already the longest delay of a round.
-
What does this program print?
var watch = System.Diagnostics.Stopwatch.StartNew(); await Task.Delay(20); watch.Stop(); Console.WriteLine(watch.ElapsedMilliseconds >= 19);Answer
True
A delay of twenty milliseconds does not end earlier than that.
-
What is the checksum of the benchmark?
Answer
The sum of the job ids, 36, in every run
Both rows of the measured table show 36.
-
What does this program print?
var sum = 0; for (var id = 1; id <= 8; id++) { sum += id; } Console.WriteLine(sum);Answer
36
One to eight adds to thirty-six.
-
Why is a CPU job different from a waiting job in a pool?
Answer
A CPU job holds its thread, so the cores limit the pool
The pool of this capstone waits, and waiting scales past the cores.
The code on this page was built with the .NET SDK 10.0.303 and run on Alpine Linux 3.24.3 (x86_64) in a Docker container.
Article author: Arthur Isaev