Throughput: the same loop in Haskell and C++

The one benchmark worth reading is the one you can check. This page runs the same twenty-million-iteration loop in both languages, twice: once with a division by a constant and once with a bit mask, and then reports five runs of each build with the minimum, the median, the exact flags and the runtime settings. Every number below came from this machine in this container, and the programs are short enough to copy.

The loop is deliberately arithmetic and nothing else. It allocates almost nothing in either language, so what it measures is the code the compiler emits for an integer add, a multiply and a division, which is the part of the two languages that people argue about. The second half of the page is why the two columns differ, and it is a compiler story more than a language one.

The loop in C++

Two functions with the same shape: a for loop over twenty million values, a strict accumulator, and one operation that keeps the optimizer from turning the sum into a formula. The first adds (i * i) % 1000003, the second (i * i) & 1048575. The result is printed before the time, so nothing the compiler considers unused can be removed.

#include <chrono> #include <cstdint> #include <cstdio> constexpr std::int64_t limit = 20000000; constexpr std::int64_t modulus = 1000003; constexpr std::int64_t mask = 1048575; std::int64_t total_mod(std::int64_t count) { std::int64_t acc = 0; for (std::int64_t i = 1; i <= count; ++i) { acc += (i * i) % modulus; } return acc; } std::int64_t total_mask(std::int64_t count) { std::int64_t acc = 0; for (std::int64_t i = 1; i <= count; ++i) { acc += (i * i) & mask; } return acc; } void report(const char* name, std::int64_t sum, double seconds) { std::printf("%-5s %lld %.3f s\n", name, static_cast<long long>(sum), seconds); } int main() { auto start = std::chrono::steady_clock::now(); const std::int64_t with_mod = total_mod(limit); auto end = std::chrono::steady_clock::now(); report("mod", with_mod, std::chrono::duration<double>(end - start).count()); start = std::chrono::steady_clock::now(); const std::int64_t with_mask = total_mask(limit); end = std::chrono::steady_clock::now(); report("mask", with_mask, std::chrono::duration<double>(end - start).count()); }

The build is clean under -Wall -Wextra and the run prints the sum and the seconds for both workloads:

g++ -std=c++23 -O2 -Wall -Wextra loop.cpp -o loop-o2
./loop-o2; echo "status $?"

mod   9997949923550 0.043 s
mask  10454938290048 0.020 s
status 0

The same loop in Haskell

The accumulator is strict by construction: go takes the next accumulator as an argument and the recursive call is what returns it, so there is no thunk chain to build. Int64 is a machine word, mod is the same division and .&. the same mask as in C++.

The two sums are forced with evaluate from Control.Exception, not bound with let: a lazy binding would move the work inside the printf and the timer would report the printing. That is the mistake the previous laziness page shows from the other side.

{-# LANGUAGE NumericUnderscores #-} module Main (main) where import Control.Exception (evaluate) import Data.Bits ((.&.)) import Data.Int (Int64) import Data.Time.Clock (diffUTCTime, getCurrentTime) import Text.Printf (printf) limit :: Int64 limit = 20_000_000 modulus :: Int64 modulus = 1_000_003 mask :: Int64 mask = 1_048_575 totalMod :: Int64 -> Int64 totalMod count = go 1 0 where go i acc | i > count = acc | otherwise = go (i + 1) (acc + (i * i) `mod` modulus) totalMask :: Int64 -> Int64 totalMask count = go 1 0 where go i acc | i > count = acc | otherwise = go (i + 1) (acc + ((i * i) .&. mask)) main :: IO () main = do start <- getCurrentTime withMod <- evaluate (totalMod limit) middle <- getCurrentTime withMask <- evaluate (totalMask limit) end <- getCurrentTime printf "%-5s %d %.3f s\n" "mod" withMod (realToFrac (diffUTCTime middle start) :: Double) printf "%-5s %d %.3f s\n" "mask" withMask (realToFrac (diffUTCTime end middle) :: Double)

ghc -Wall -O1 -o loop-hs1 Loop.hs
ghc -Wall -O2 -o loop-hs2 Loop.hs
./loop-hs1; ./loop-hs2; echo "status $?"

[1 of 2] Compiling Main             ( Loop.hs, Loop.o )
[2 of 2] Linking loop-hs1
[1 of 2] Compiling Main             ( Loop.hs, Loop.o ) [Flags changed]
[2 of 2] Linking loop-hs2
mod   9997949923550 0.389 s
mask  10454938290048 0.032 s
mod   9997949923550 0.383 s
mask  10454938290048 0.056 s
status 0

The numbers on this machine

Five runs of each build, one after another, with nothing else started in the container. The times are what the programs printed themselves, measured with steady_clock and getCurrentTime around the loop only. min is the fastest run, which is the number with the least interference, and median is the middle of the five, which is the number to quote.

Build Workload The five runs, s Min Median
g++ -O0 mod 0.076, 0.076, 0.100, 0.078, 0.076 0.076 0.076
g++ -O0 mask 0.081, 0.081, 0.080, 0.080, 0.080 0.080 0.080
g++ -O2 mod 0.042, 0.042, 0.042, 0.042, 0.055 0.042 0.042
g++ -O2 mask 0.020, 0.020, 0.020, 0.026, 0.020 0.020 0.020
ghc -O1 mod 0.510, 0.427, 0.354, 0.365, 0.370 0.354 0.370
ghc -O1 mask 0.056, 0.030, 0.033, 0.030, 0.030 0.030 0.030
ghc -O2 mod 0.362, 0.352, 0.353, 0.353, 0.357 0.352 0.353
ghc -O2 mask 0.030, 0.031, 0.030, 0.030, 0.030 0.030 0.030

The machine and the settings behind the table:

The GC settings deserve a separate check for this workload, because they are the first thing to suspect in a Haskell number. The same binary with +RTS -s reports the allocation and the GC time:

./loop-hs2 +RTS -s -RTS

mod   9997949923550 0.563 s
mask  10454938290048 0.048 s
         164,824 bytes allocated in the heap
           3,272 bytes copied during GC
          44,328 bytes maximum residency (1 sample(s))
          25,304 bytes maximum slop
               6 MiB total memory in use (0 MiB lost due to fragmentation)

                                     Tot time (elapsed)  Avg pause  Max pause
  Gen  0         0 colls,     0 par    0.000s   0.000s     0.0000s    0.0000s
  Gen  1         1 colls,     0 par    0.000s   0.000s     0.0002s    0.0002s

  INIT    time    0.001s  (  0.001s elapsed)
  MUT     time    0.608s  (  0.611s elapsed)
  GC      time    0.000s  (  0.000s elapsed)
  EXIT    time    0.000s  (  0.000s elapsed)
  Total   time    0.609s  (  0.612s elapsed)

  %GC     time       0.0%  (0.0% elapsed)

  Alloc rate    270,914 bytes per MUT second

  Productivity  99.8% of total user, 99.8% of total elapsed

status 0

164,824 bytes allocated for twenty million iterations is nothing: the loop runs in the mutator and the collector never wakes up, so -A, -H and the nursery size cannot change these numbers. A page that tuned the RTS for this loop would be measuring noise.

Why the two columns differ

The mask workload is the fair comparison: 0.020 s against 0.030 s at the best settings, a factor of 1.5. Both compilers turn multiply, add and mask into a few instructions per iteration, and the loop is bound by those instructions. One nanosecond per iteration is a few cycles on a modern core, which is what a multiply and an add cost when they depend on each other.

The mod workload is where the gap opens: 0.042 s against 0.352 s, a factor of eight. The reason is the division. g++ sees % 1000003 with a compile-time constant and rewrites it into a multiply and a shift, a trick every optimising C compiler knows. GHC emits a call to the primitive that does the 64-bit division, and the processor takes tens of cycles for it, so the loop becomes bound by the divider rather than by the add.

That is why -O2 does not help the Haskell mod row: 0.353 s against 0.370 s is the same loop with a different instruction schedule around the division. The mask row shows what the same Haskell code does when the expensive operation goes away, and there the two languages are within a factor of two.

None of this says that one language is faster. It says that a benchmark row is a measurement of one expression on one compiler, and that a factor of eight can come from a strength reduction the other compiler did not do. A real program is a mix of arithmetic, allocation and IO, and the memory page measures the allocation half.

How these numbers were taken

Five runs, min and median, the same machine for both languages, the flags written down, and the workload printed so that no compiler can remove it. Those five rules are enough to make a number checkable, and they are the rules the course lesson on measuring performance teaches with /usr/bin/time and +RTS -s.

Three things this page does not do, and a reader should know before quoting it. The container is shared, so a run can be disturbed by another process; the median is there for that. The programs are microbenchmarks with no IO and no allocation worth the name, so they say nothing about a program that reads files. And the versions are the ones in this image: g++ 15.2.0 and GHC 9.10.3, both current, and both of which will produce different code in a year.

The Haskell lesson behind the flags is GHC flags, and the next page in this set is Memory: +RTS -s against /usr/bin/time.

The C++ programs were built with g++ 15.2.0 (Alpine 15.2.0) using -std=c++23 -Wall -Wextra with -O0 or -O2 and the Haskell programs with GHC 9.10.3 using -Wall with -O1 or -O2, all run on a 4-CPU Alpine Linux 3.24.3 (x86_64) in a Docker container.

Article author: Arthur Isaev

Banner image