Tiered compilation

The runtime does not compile a method once and for ever. It starts with a quick compilation that produces correct but unoptimized code, called tier 0, and when a method has run enough times it compiles it again with the optimizer, tier 1. The first calls are fast to start and slow to run; the later ones are the opposite. The switch DOTNET_TieredCompilation turns the two tiers off or on.

Two tiers of code

The measured program

The loop calls a method that mixes bits, so the optimizer has something to do. It runs twenty million times, five times, and the table keeps every run:

using System.Diagnostics; var runs = new List<double>(); long total = 0; for (var run = 0; run < 5; run++) { var watch = Stopwatch.StartNew(); total = 0; for (var i = 0; i < 20_000_000; i++) { total += Mix(i); } watch.Stop(); runs.Add(watch.Elapsed.TotalMilliseconds); Console.WriteLine($"run {run + 1} {runs[^1]:F2} ms"); } Console.WriteLine($"min {runs.Min():F2} median {runs.Order().ElementAt(2):F2}"); Console.WriteLine($"total {total}"); static int Mix(int value) { var x = value * 31; x ^= x >> 7; return x & 1023; }

dotnet run

run 1 292.68 ms
run 2 223.67 ms
run 3 204.18 ms
run 4 199.99 ms
run 5 207.00 ms
min 199.99 median 207.00
total 10229995130

DOTNET_TieredCompilation=0 dotnet run

run 1 314.83 ms
run 2 339.90 ms
run 3 303.94 ms
run 4 268.62 ms
run 5 367.24 ms
min 268.62 median 314.83
total 10229995130

DOTNET_TieredPGO=1 dotnet run

run 1 310.83 ms
run 2 319.89 ms
run 3 226.86 ms
run 4 215.36 ms
run 5 213.48 ms
min 213.48 median 226.86
total 10229995130

Three runs of the same loop

SettingRun 1MedianTotal
default (tiered)292.68207.0010 229 995 130
TieredCompilation=0314.83314.8310 229 995 130
TieredPGO=1310.83226.8610 229 995 130

Every setting printed the same total. The default run started at 292.68 milliseconds and improved to about 200 as the hot method was recompiled, which is the tier change visible in the output. With tiering off the runs stayed between 268 and 367 milliseconds: there was no early slow tier and no later fast one, and the median was worse than the default. PGO was between the two on this short run.

Reading the numbers

DOTNET_TieredCompilation and DOTNET_TieredPGO are environment variables of the runtime, like the GC settings of an earlier lesson. A framework-dependent app can also set them in its runtime configuration through the project property TieredCompilation; a published app carries the value, and an environment variable wins over it.

Exercises

  1. Print the environment variables from the program and see which of them the shell set.
  2. Run with DOTNET_TieredCompilation=0 and DOTNET_TC_QuickJitForLoops=0 and compare.
  3. Make the loop ten times longer and see whether the first run is still the slowest.
  4. Set DOTNET_ReadyToRun=0 and measure the startup of a published app.

Questions

A new set of questions in a new order every time. Choose one answer for each, then check.

  1. What is tiered compilation?

    Answer

    A quick first compile and an optimized recompile of hot methods

    Tier 0 starts fast and tier 1 replaces it when the method is hot.

  2. What does DOTNET_TieredCompilation=0 do?

    Answer

    Compiles each method once, fully optimized

    The measured runs with it on stayed at one speed from the first run.

  3. What does this program print? Console.WriteLine(Environment.GetEnvironmentVariable("DOTNET_TieredCompilation") ?? "unset");

    Answer

    unset

    The variable is only set when the command sets it, and this run did not.

  4. Which run of the default program was the slowest?

    Answer

    The first one

    The measured first run was 292.68 ms and the later ones about 200.

  5. What does the first run of a hot loop pay for?

    Answer

    Execution in the unoptimized tier 0 code

    The optimized version arrives after the method is called enough times.

  6. What does PGO use to optimize the tier-1 code?

    Answer

    The branches the tier-0 run actually took

    Profile-guided optimization instruments the first tier and recompiles with the profile.

  7. What does this program print? Console.WriteLine(Environment.GetEnvironmentVariable("DOTNET_TieredPGO") is null);

    Answer

    True

    The command did not set the variable, so it is absent.

  8. Why is tiering on by default?

    Answer

    Startup latency matters more than peak speed for most programs

    A short program would otherwise pay the optimizer before its first line.

  9. Does the tier of a method change its result?

    Answer

    No, every tier computes the same answer

    All three measured settings printed the same total.

  10. What does this program print? var x = 0; for (var i = 0; i < 1000; i++) x += i; Console.WriteLine(x > 0);

    Answer

    True

    The sum of the first thousand numbers is positive.

The code on this page was built with the .NET SDK 10.0.303 and run on Alpine Linux 3.24.3 (x86_64) in a Docker container.

Article author: Arthur Isaev