Throughput: the same loop in Haskell and C++
The one benchmark worth reading is the one you can check. This page runs the same twenty-million-iteration loop in both languages, twice: once with a division by a constant and once with a bit mask, and then reports five runs of each build with the minimum, the median, the exact flags and the runtime settings. Every number below came from this machine in this container, and the programs are short enough to copy.
The loop is deliberately arithmetic and nothing else. It allocates almost nothing in either language, so what it measures is the code the compiler emits for an integer add, a multiply and a division, which is the part of the two languages that people argue about. The second half of the page is why the two columns differ, and it is a compiler story more than a language one.
The loop in C++
Two functions with the same shape: a for loop over twenty million values, a strict accumulator, and one operation that keeps the optimizer from turning the sum into a formula. The first adds (i * i) % 1000003, the second (i * i) & 1048575. The result is printed before the time, so nothing the compiler considers unused can be removed.
#include <chrono>
#include <cstdint>
#include <cstdio>
constexpr std::int64_t limit = 20000000;
constexpr std::int64_t modulus = 1000003;
constexpr std::int64_t mask = 1048575;
std::int64_t total_mod(std::int64_t count) {
std::int64_t acc = 0;
for (std::int64_t i = 1; i <= count; ++i) {
acc += (i * i) % modulus;
}
return acc;
}
std::int64_t total_mask(std::int64_t count) {
std::int64_t acc = 0;
for (std::int64_t i = 1; i <= count; ++i) {
acc += (i * i) & mask;
}
return acc;
}
void report(const char* name, std::int64_t sum, double seconds) {
std::printf("%-5s %lld %.3f s\n", name, static_cast<long long>(sum), seconds);
}
int main() {
auto start = std::chrono::steady_clock::now();
const std::int64_t with_mod = total_mod(limit);
auto end = std::chrono::steady_clock::now();
report("mod", with_mod, std::chrono::duration<double>(end - start).count());
start = std::chrono::steady_clock::now();
const std::int64_t with_mask = total_mask(limit);
end = std::chrono::steady_clock::now();
report("mask", with_mask, std::chrono::duration<double>(end - start).count());
}
The build is clean under -Wall -Wextra and the run prints the sum and the seconds for both workloads:
g++ -std=c++23 -O2 -Wall -Wextra loop.cpp -o loop-o2
./loop-o2; echo "status $?"
mod 9997949923550 0.043 s
mask 10454938290048 0.020 s
status 0
The same loop in Haskell
The accumulator is strict by construction: go takes the next accumulator as an argument and the recursive call is what returns it, so there is no thunk chain to build. Int64 is a machine word, mod is the same division and .&. the same mask as in C++.
The two sums are forced with evaluate from Control.Exception, not bound with let: a lazy binding would move the work inside the printf and the timer would report the printing. That is the mistake the previous laziness page shows from the other side.
{-# LANGUAGE NumericUnderscores #-}
module Main (main) where
import Control.Exception (evaluate)
import Data.Bits ((.&.))
import Data.Int (Int64)
import Data.Time.Clock (diffUTCTime, getCurrentTime)
import Text.Printf (printf)
limit :: Int64
limit = 20_000_000
modulus :: Int64
modulus = 1_000_003
mask :: Int64
mask = 1_048_575
totalMod :: Int64 -> Int64
totalMod count = go 1 0
where
go i acc
| i > count = acc
| otherwise = go (i + 1) (acc + (i * i) `mod` modulus)
totalMask :: Int64 -> Int64
totalMask count = go 1 0
where
go i acc
| i > count = acc
| otherwise = go (i + 1) (acc + ((i * i) .&. mask))
main :: IO ()
main = do
start <- getCurrentTime
withMod <- evaluate (totalMod limit)
middle <- getCurrentTime
withMask <- evaluate (totalMask limit)
end <- getCurrentTime
printf "%-5s %d %.3f s\n" "mod" withMod (realToFrac (diffUTCTime middle start) :: Double)
printf "%-5s %d %.3f s\n" "mask" withMask (realToFrac (diffUTCTime end middle) :: Double)
ghc -Wall -O1 -o loop-hs1 Loop.hs
ghc -Wall -O2 -o loop-hs2 Loop.hs
./loop-hs1; ./loop-hs2; echo "status $?"
[1 of 2] Compiling Main ( Loop.hs, Loop.o )
[2 of 2] Linking loop-hs1
[1 of 2] Compiling Main ( Loop.hs, Loop.o ) [Flags changed]
[2 of 2] Linking loop-hs2
mod 9997949923550 0.389 s
mask 10454938290048 0.032 s
mod 9997949923550 0.383 s
mask 10454938290048 0.056 s
status 0
The numbers on this machine
Five runs of each build, one after another, with nothing else started in the container. The times are what the programs printed themselves, measured with steady_clock and getCurrentTime around the loop only. min is the fastest run, which is the number with the least interference, and median is the middle of the five, which is the number to quote.
| Build | Workload | The five runs, s | Min | Median |
|---|---|---|---|---|
| g++ -O0 | mod | 0.076, 0.076, 0.100, 0.078, 0.076 | 0.076 | 0.076 |
| g++ -O0 | mask | 0.081, 0.081, 0.080, 0.080, 0.080 | 0.080 | 0.080 |
| g++ -O2 | mod | 0.042, 0.042, 0.042, 0.042, 0.055 | 0.042 | 0.042 |
| g++ -O2 | mask | 0.020, 0.020, 0.020, 0.026, 0.020 | 0.020 | 0.020 |
| ghc -O1 | mod | 0.510, 0.427, 0.354, 0.365, 0.370 | 0.354 | 0.370 |
| ghc -O1 | mask | 0.056, 0.030, 0.033, 0.030, 0.030 | 0.030 | 0.030 |
| ghc -O2 | mod | 0.362, 0.352, 0.353, 0.353, 0.357 | 0.352 | 0.353 |
| ghc -O2 | mask | 0.030, 0.031, 0.030, 0.030, 0.030 | 0.030 | 0.030 |
The machine and the settings behind the table:
- 4 CPUs (nproc 4), Alpine Linux 3.24.3, x86_64, in a Docker container.
- g++ 15.2.0 (Alpine 15.2.0), flags -std=c++23 -Wall -Wextra and -O0 or -O2.
- GHC 9.10.3, flags -Wall and -O1 or -O2, no -threaded and no +RTS flags: the default runtime, one capability and the default GC settings.
- The first run of a build was not thrown away: those are runs one to five, and the first run of the Haskell -O1 binary is the 0.510 s line.
The GC settings deserve a separate check for this workload, because they are the first thing to suspect in a Haskell number. The same binary with +RTS -s reports the allocation and the GC time:
./loop-hs2 +RTS -s -RTS
mod 9997949923550 0.563 s
mask 10454938290048 0.048 s
164,824 bytes allocated in the heap
3,272 bytes copied during GC
44,328 bytes maximum residency (1 sample(s))
25,304 bytes maximum slop
6 MiB total memory in use (0 MiB lost due to fragmentation)
Tot time (elapsed) Avg pause Max pause
Gen 0 0 colls, 0 par 0.000s 0.000s 0.0000s 0.0000s
Gen 1 1 colls, 0 par 0.000s 0.000s 0.0002s 0.0002s
INIT time 0.001s ( 0.001s elapsed)
MUT time 0.608s ( 0.611s elapsed)
GC time 0.000s ( 0.000s elapsed)
EXIT time 0.000s ( 0.000s elapsed)
Total time 0.609s ( 0.612s elapsed)
%GC time 0.0% (0.0% elapsed)
Alloc rate 270,914 bytes per MUT second
Productivity 99.8% of total user, 99.8% of total elapsed
status 0
164,824 bytes allocated for twenty million iterations is nothing: the loop runs in the mutator and the collector never wakes up, so -A, -H and the nursery size cannot change these numbers. A page that tuned the RTS for this loop would be measuring noise.
Why the two columns differ
The mask workload is the fair comparison: 0.020 s against 0.030 s at the best settings, a factor of 1.5. Both compilers turn multiply, add and mask into a few instructions per iteration, and the loop is bound by those instructions. One nanosecond per iteration is a few cycles on a modern core, which is what a multiply and an add cost when they depend on each other.
The mod workload is where the gap opens: 0.042 s against 0.352 s, a factor of eight. The reason is the division. g++ sees % 1000003 with a compile-time constant and rewrites it into a multiply and a shift, a trick every optimising C compiler knows. GHC emits a call to the primitive that does the 64-bit division, and the processor takes tens of cycles for it, so the loop becomes bound by the divider rather than by the add.
That is why -O2 does not help the Haskell mod row: 0.353 s against 0.370 s is the same loop with a different instruction schedule around the division. The mask row shows what the same Haskell code does when the expensive operation goes away, and there the two languages are within a factor of two.
None of this says that one language is faster. It says that a benchmark row is a measurement of one expression on one compiler, and that a factor of eight can come from a strength reduction the other compiler did not do. A real program is a mix of arithmetic, allocation and IO, and the memory page measures the allocation half.
How these numbers were taken
Five runs, min and median, the same machine for both languages, the flags written down, and the workload printed so that no compiler can remove it. Those five rules are enough to make a number checkable, and they are the rules the course lesson on measuring performance teaches with /usr/bin/time and +RTS -s.
Three things this page does not do, and a reader should know before quoting it. The container is shared, so a run can be disturbed by another process; the median is there for that. The programs are microbenchmarks with no IO and no allocation worth the name, so they say nothing about a program that reads files. And the versions are the ones in this image: g++ 15.2.0 and GHC 9.10.3, both current, and both of which will produce different code in a year.
The Haskell lesson behind the flags is GHC flags, and the next page in this set is Memory: +RTS -s against /usr/bin/time.
The C++ programs were built with g++ 15.2.0 (Alpine 15.2.0) using -std=c++23 -Wall -Wextra with -O0 or -O2 and the Haskell programs with GHC 9.10.3 using -Wall with -O1 or -O2, all run on a 4-CPU Alpine Linux 3.24.3 (x86_64) in a Docker container.
Article author: Arthur Isaev