Parallel and PLINQ
Data parallelism splits one loop over the cores. Parallel.For and Parallel.ForEach run the body of a loop on several threads and wait for all of them before they return; PLINQ does the same for a LINQ query. Both are for work that is already split into independent pieces, and neither makes a slow single operation fast.
Parallel.For
A shared total needs an atomic update, and an array written by index does not, because each iteration writes its own slot:
var total = 0;
Parallel.For(0, 1000, i => Interlocked.Add(ref total, i));
Console.WriteLine($"sum {total}");
var sizes = new[] { 1, 2, 3, 4 };
var doubled = new int[sizes.Length];
Parallel.For(0, sizes.Length, i => doubled[i] = sizes[i] * 2);
Console.WriteLine($"doubled {string.Join(",", doubled)}");
dotnet run
sum 499500
doubled 2,4,6,8
The sum of zero to nine hundred and ninety-nine is 499 500, exact because the addition is atomic. The array kept its order because iteration i writes doubled[i]: the iterations ran in some order, and the result does not depend on which. Parallel.For returned after the last iteration, so the next line was safe.
Parallel.ForEach and options
ParallelOptions.MaxDegreeOfParallelism caps how many iterations run at once, which a program that calls a database or an API needs:
var options = new ParallelOptions { MaxDegreeOfParallelism = 2 };
var count = 0;
Parallel.ForEach(Enumerable.Range(1, 10), options, _ => Interlocked.Increment(ref count));
Console.WriteLine($"count {count}");
dotnet run
count 10
All ten iterations ran, at most two at a time. A cap of one makes the loop sequential, which is a way to test whether the parallelism itself is the problem. An exception thrown in an iteration is collected and thrown by the call as an AggregateException after the other iterations have been stopped, which is different from the await of a task, where the first exception is rethrown directly.
PLINQ
AsParallel turns a query into a parallel one. The aggregate operators do not care about the order, so they are the natural fit; AsOrdered restores the order when the output has to keep it:
var numbers = Enumerable.Range(1, 1000);
var even = numbers.AsParallel().Where(n => n % 2 == 0).Count();
Console.WriteLine($"even {even}");
var first = numbers.AsParallel().Where(n => n % 7 == 0).OrderBy(n => n).Take(3).ToArray();
Console.WriteLine($"first {string.Join(",", first)}");
var sum = numbers.AsParallel().Sum();
Console.WriteLine($"sum {sum}");
dotnet run
even 500
first 7,14,21
sum 500500
Half of the thousand numbers are even, the first three multiples of seven are 7, 14 and 21, and the total is 500 500. The OrderBy in the second query is a full sort after the filter, so it costs more than the filter itself; a query that only needs Take(3) of an unordered set would not sort. AsOrdered is cheaper than a sort when the source order is what the program wants.
When it pays
- The body must be heavy enough: a loop of a few arithmetic operations gets slower when the scheduling costs more than the work.
- The iterations must be independent; a shared counter needs Interlocked or a lock, which removes part of the gain.
- An I/O call inside the body wants Task.WhenAll, not a parallel loop: the loop would keep a thread per call while it waits.
- The lesson on performance measures the same loop with and without parallelism, which is the only way to know.
Exercises
- Write the sum with a lock instead of Interlocked and run both versions.
- Throw in every tenth iteration and read the AggregateException.
- Set MaxDegreeOfParallelism to one and compare the output.
- Replace the PLINQ count with Parallel.For and compare the code.
Questions
A new set of questions in a new order every time. Choose one answer for each, then check.
-
What does Parallel.For do?
Answer
Runs the iterations on several threads and waits for all of them
The call returns after the last iteration, so the next line is safe.
-
Why does the sum of the first program use Interlocked?
Answer
The iterations share one variable and update it at the same time
A plain addition would lose updates, as the lesson on locks showed.
-
What does this program print?
var total = 0; Parallel.For(0, 1000, i => Interlocked.Add(ref total, i)); Console.WriteLine(total);Answer
499500
The sum of zero to 999 is 499500, as the measured run printed.
-
What does MaxDegreeOfParallelism do?
Answer
Caps how many iterations run at the same time
A cap of two ran ten iterations with at most two inside.
-
What does this program print?
var count = 0; var options = new ParallelOptions { MaxDegreeOfParallelism = 2 }; Parallel.ForEach(Enumerable.Range(1, 10), options, _ => Interlocked.Increment(ref count)); Console.WriteLine(count);Answer
10
Every iteration runs; the option limits concurrency, not the count.
-
What does AsParallel do?
Answer
Turns a LINQ query into a parallel one
The query then runs on several threads, which PLINQ schedules.
-
What does this program print?
var even = Enumerable.Range(1, 1000).AsParallel().Where(n => n % 2 == 0).Count(); Console.WriteLine(even);Answer
500
Half of the thousand numbers are even.
-
Does a PLINQ query keep the order of its source by default?
Answer
No, AsOrdered or a sort is needed
Parallel execution changes the order, and AsOrdered restores it at a cost.
-
What does this program print?
var sum = Enumerable.Range(1, 1000).AsParallel().Sum(); Console.WriteLine(sum);Answer
500500
The sum of one to a thousand is 500500.
-
When does Parallel.For make a loop slower?
Answer
When the work per iteration is smaller than the scheduling cost
The overhead of splitting and joining must be paid by the work.
The code on this page was built with the .NET SDK 10.0.303 and run on Alpine Linux 3.24.3 (x86_64) in a Docker container.
Article author: Arthur Isaev