Capstone C: measuring the pool

A pool of workers is worth its complexity only if the work it does comes back faster, and the only way to know is to measure the same jobs with one worker and with four. The benchmark of this lesson runs eight jobs of twenty milliseconds five times with each pool size, and it prints the minimum and the median rather than the average, because one slow run would move an average and a reader cannot tell how much.

The measurement

Stopwatch around one complete run of the pool, OrderBy over the five times for the minimum and the third element for the median, and the same checksum after every run so a fast run that lost a job is visible. The whole program is one listing:

using System.Diagnostics; using System.Threading.Channels; Console.WriteLine("benchmark 1.0"); var jobs = Enumerable.Range(1, 8).Select(id => new Job(id, 20)).ToArray(); foreach (var workers in new[] { 1, 4 }) { var times = new List<double>(); var total = 0L; for (var run = 1; run <= 5; run++) { var watch = Stopwatch.StartNew(); total = await Run(workers, jobs); watch.Stop(); times.Add(watch.Elapsed.TotalMilliseconds); Console.WriteLine($"workers {workers} run {run} {watch.Elapsed.TotalMilliseconds:F2} ms"); } var sorted = times.OrderBy(time => time).ToList(); Console.WriteLine($"workers {workers} min {sorted[0]:F2} median {sorted[2]:F2} total {total}"); } static async Task<long> Run(int workers, Job[] jobs) { var queue = Channel.CreateBounded<Job>(8); var total = 0L; var producer = Task.Run(async () => { foreach (var job in jobs) { await queue.Writer.WriteAsync(job); } queue.Writer.Complete(); }); var pool = Enumerable.Range(1, workers).Select(_ => Task.Run(async () => { await foreach (var job in queue.Reader.ReadAllAsync()) { await Task.Delay(job.DelayMs); Interlocked.Add(ref total, job.Id); } })).ToArray(); await producer; await Task.WhenAll(pool); return total; } record Job(int Id, int DelayMs);

dotnet run
echo "status: $?"

benchmark 1.0
workers 1 run 1 209.18 ms
workers 1 run 2 178.87 ms
workers 1 run 3 165.04 ms
workers 1 run 4 167.47 ms
workers 1 run 5 162.03 ms
workers 1 min 162.03 median 167.47 total 36
workers 4 run 1 44.83 ms
workers 4 run 2 40.65 ms
workers 4 run 3 40.73 ms
workers 4 run 4 43.44 ms
workers 4 run 5 40.66 ms
workers 4 min 40.65 median 40.73 total 36
status: 0

The numbers

The measured table of this container, one run of the program above:

WorkersRuns (ms)MinMedianChecksum
1209.18, 178.87, 165.04, 167.47, 162.03162.03167.4736
444.83, 40.65, 40.73, 43.44, 40.6640.6540.7336

The machine, measured with nproc and /etc/os-release:

ItemValue
CPU4 cores, x86_64
SystemAlpine Linux 3.24.3 in a Docker container
Runtime.NET SDK 10.0.303, runtime 10.0.15
ConfigurationDebug build, dotnet run, no arguments

Four workers took about a quarter of the time of one, and the first run of each pool size was the slowest of its five: 209.18 against 162.03 for one worker, 44.83 against 40.65 for four. The first run pays for the JIT, the type initialisers and the first task starts, which is the warm-up lesson 102 describes; that is why a benchmark runs the code more than once and reports the minimum next to the median.

The floor of eight jobs

Eight jobs of twenty milliseconds are a hundred and sixty milliseconds of waiting. One worker cannot do better than that, and its minimum of 162.03 ms is as close to the floor as a measurement gets. Four workers divide the eight jobs into two each, so the floor is forty milliseconds, and the measured 40.65 ms is again the floor plus the cost of starting and ending the tasks.

When the work is CPU

A pool of Task objects over a channel of waiting jobs is not the same as a pool of threads over arithmetic. A job that computes without awaiting holds its thread for the whole computation, and four workers on four cores are the most the machine can run at once; a fifth worker waits for a core and adds switching, not throughput. The next group of measurements in the performance part of this course, lessons 102 to 108, is the other half of the same question.

The capstone ends here: a model, a queue, a pool, a result, a shutdown, a logger and a measurement. The next lesson is the last part of the course, the one that says where to go from C#, and what this Linux container cannot run.

Exercises

  1. Run the benchmark with eight workers and with two, and add both rows to the table.
  2. Give the jobs different delays and check whether the floor of the run is the longest delay or the sum divided by the workers.
  3. Replace the waiting with a loop that adds a million numbers and run the benchmark again; describe why four workers are now close to one.
  4. Print the average next to the median and say which of the two the first run moves more.

Questions

A new set of questions in a new order every time. Choose one answer for each, then check.

  1. Why does the benchmark print the minimum and the median and not the average?

    Answer

    One slow run moves an average, and the minimum and median show the usual cost

    The first run of each size was the slowest of its five.

  2. What is the floor of one worker over eight jobs of 20 ms?

    Answer

    160 ms

    The measured minimum is 162.03 ms.

  3. What does this program print? var times = new[] { 209.18, 178.87, 165.04, 167.47, 162.03 }; Console.WriteLine(times.Min());

    Answer

    162.03

    The minimum of the five runs of one worker.

  4. Why is the first run of a pool size slower?

    Answer

    It pays for the JIT, the type initialisers and the first task starts

    The measured first run is the slowest of each five.

  5. What does this program print? var times = new[] { 3.0, 1.0, 2.0 }.OrderBy(time => time).ToList(); Console.WriteLine(times[times.Count / 2]);

    Answer

    2

    The sorted middle of three values is the median.

  6. Why would eight workers not take a quarter of the time of four?

    Answer

    Four cores run four workers at once, and the jobs are two per worker

    The floor with four workers is already the longest delay of a round.

  7. What does this program print? var watch = System.Diagnostics.Stopwatch.StartNew(); await Task.Delay(20); watch.Stop(); Console.WriteLine(watch.ElapsedMilliseconds >= 19);

    Answer

    True

    A delay of twenty milliseconds does not end earlier than that.

  8. What is the checksum of the benchmark?

    Answer

    The sum of the job ids, 36, in every run

    Both rows of the measured table show 36.

  9. What does this program print? var sum = 0; for (var id = 1; id <= 8; id++) { sum += id; } Console.WriteLine(sum);

    Answer

    36

    One to eight adds to thirty-six.

  10. Why is a CPU job different from a waiting job in a pool?

    Answer

    A CPU job holds its thread, so the cores limit the pool

    The pool of this capstone waits, and waiting scales past the cores.

The code on this page was built with the .NET SDK 10.0.303 and run on Alpine Linux 3.24.3 (x86_64) in a Docker container.

Article author: Arthur Isaev