Tiered compilation
The runtime does not compile a method once and for ever. It starts with a quick compilation that produces correct but unoptimized code, called tier 0, and when a method has run enough times it compiles it again with the optimizer, tier 1. The first calls are fast to start and slow to run; the later ones are the opposite. The switch DOTNET_TieredCompilation turns the two tiers off or on.
Two tiers of code
- Tier 0 compiles fast and optimizes little, so a program reaches its first line sooner.
- A counter on the method notices that it is hot, and the background compiler produces tier 1 while tier 0 keeps running.
- Profile-guided optimization, DOTNET_TieredPGO=1, is on by default in recent runtimes: the tier-1 compile uses what the tier-0 run learned about the branches it takes.
- DOTNET_TieredCompilation=0 compiles each method once, fully optimized, which helps a short CPU-bound run and costs startup time.
The measured program
The loop calls a method that mixes bits, so the optimizer has something to do. It runs twenty million times, five times, and the table keeps every run:
using System.Diagnostics;
var runs = new List<double>();
long total = 0;
for (var run = 0; run < 5; run++)
{
var watch = Stopwatch.StartNew();
total = 0;
for (var i = 0; i < 20_000_000; i++)
{
total += Mix(i);
}
watch.Stop();
runs.Add(watch.Elapsed.TotalMilliseconds);
Console.WriteLine($"run {run + 1} {runs[^1]:F2} ms");
}
Console.WriteLine($"min {runs.Min():F2} median {runs.Order().ElementAt(2):F2}");
Console.WriteLine($"total {total}");
static int Mix(int value)
{
var x = value * 31;
x ^= x >> 7;
return x & 1023;
}
dotnet run
run 1 292.68 ms
run 2 223.67 ms
run 3 204.18 ms
run 4 199.99 ms
run 5 207.00 ms
min 199.99 median 207.00
total 10229995130
DOTNET_TieredCompilation=0 dotnet run
run 1 314.83 ms
run 2 339.90 ms
run 3 303.94 ms
run 4 268.62 ms
run 5 367.24 ms
min 268.62 median 314.83
total 10229995130
DOTNET_TieredPGO=1 dotnet run
run 1 310.83 ms
run 2 319.89 ms
run 3 226.86 ms
run 4 215.36 ms
run 5 213.48 ms
min 213.48 median 226.86
total 10229995130
Three runs of the same loop
| Setting | Run 1 | Median | Total |
|---|---|---|---|
| default (tiered) | 292.68 | 207.00 | 10 229 995 130 |
| TieredCompilation=0 | 314.83 | 314.83 | 10 229 995 130 |
| TieredPGO=1 | 310.83 | 226.86 | 10 229 995 130 |
Every setting printed the same total. The default run started at 292.68 milliseconds and improved to about 200 as the hot method was recompiled, which is the tier change visible in the output. With tiering off the runs stayed between 268 and 367 milliseconds: there was no early slow tier and no later fast one, and the median was worse than the default. PGO was between the two on this short run.
Reading the numbers
- A single run of a JIT-compiled method measures the tier the program happened to be in, not the speed of the code.
- A server that runs for days reaches tier 1 quickly and keeps it; a command that runs for a second spends most of its life in tier 0.
- That is why tiering is on by default: startup latency matters more for most programs than the peak speed of a method called a few times.
- A benchmark that must measure peak speed warms the method first, as the previous lessons did, or turns tiering off for the duration.
DOTNET_TieredCompilation and DOTNET_TieredPGO are environment variables of the runtime, like the GC settings of an earlier lesson. A framework-dependent app can also set them in its runtime configuration through the project property TieredCompilation; a published app carries the value, and an environment variable wins over it.
Exercises
- Print the environment variables from the program and see which of them the shell set.
- Run with DOTNET_TieredCompilation=0 and DOTNET_TC_QuickJitForLoops=0 and compare.
- Make the loop ten times longer and see whether the first run is still the slowest.
- Set DOTNET_ReadyToRun=0 and measure the startup of a published app.
Questions
A new set of questions in a new order every time. Choose one answer for each, then check.
-
What is tiered compilation?
Answer
A quick first compile and an optimized recompile of hot methods
Tier 0 starts fast and tier 1 replaces it when the method is hot.
-
What does DOTNET_TieredCompilation=0 do?
Answer
Compiles each method once, fully optimized
The measured runs with it on stayed at one speed from the first run.
-
What does this program print?
Console.WriteLine(Environment.GetEnvironmentVariable("DOTNET_TieredCompilation") ?? "unset");Answer
unset
The variable is only set when the command sets it, and this run did not.
-
Which run of the default program was the slowest?
Answer
The first one
The measured first run was 292.68 ms and the later ones about 200.
-
What does the first run of a hot loop pay for?
Answer
Execution in the unoptimized tier 0 code
The optimized version arrives after the method is called enough times.
-
What does PGO use to optimize the tier-1 code?
Answer
The branches the tier-0 run actually took
Profile-guided optimization instruments the first tier and recompiles with the profile.
-
What does this program print?
Console.WriteLine(Environment.GetEnvironmentVariable("DOTNET_TieredPGO") is null);Answer
True
The command did not set the variable, so it is absent.
-
Why is tiering on by default?
Answer
Startup latency matters more than peak speed for most programs
A short program would otherwise pay the optimizer before its first line.
-
Does the tier of a method change its result?
Answer
No, every tier computes the same answer
All three measured settings printed the same total.
-
What does this program print?
var x = 0; for (var i = 0; i < 1000; i++) x += i; Console.WriteLine(x > 0);Answer
True
The sum of the first thousand numbers is positive.
The code on this page was built with the .NET SDK 10.0.303 and run on Alpine Linux 3.24.3 (x86_64) in a Docker container.
Article author: Arthur Isaev