OpenMP in both
OpenMP is not part of either language. It is a set of compiler directives that both compilers implement from the same specification, so this is the one comparison where the two programs can be almost the same text: a comment in Fortran, a pragma in C++, and the same clause. This page measures whether the same directive buys the same speedup from both compilers on this machine.
The loop
Sixteen million doubles, each a value one to seven, summed with a square root of each. The sum is a reduction: every iteration writes to the same variable, which is what the REDUCTION(+:S) clause exists to make correct. The timing uses omp_get_wtime, which both languages call with the same name.
The Fortran program
The file par.f90:
program par_f
use, intrinsic :: iso_fortran_env, only: dp => real64
use omp_lib
implicit none
integer :: n, i
real(dp), allocatable :: a(:)
real(dp) :: s, t0, t1
character(len=8) :: size_arg
call get_command_argument(1, size_arg)
read (size_arg, *) n
allocate(a(n))
a = 1.0_dp + real([(mod(i, 7), i = 1, n)], dp)
t0 = omp_get_wtime()
s = 0.0_dp
!$omp parallel do reduction(+:s)
do i = 1, n
s = s + sqrt(a(i))
end do
!$omp end parallel do
t1 = omp_get_wtime()
print '(a,i0,a,i0,a,es20.12,a,es10.3,a)', 'fortran threads=', omp_get_max_threads(), ' n=', n, ' sum=', s, &
' time=', t1 - t0, ' s'
end program par_f
gfortran -Wall -Wextra -std=f2018 -O2 -fopenmp par.f90 -o par_f
OMP_NUM_THREADS=1 ./par_f 16000000
The C++ program
The file par.cpp:
#include <omp.h>
#include <cmath>
#include <cstdio>
#include <cstdlib>
#include <vector>
int main(int argc, char** argv) {
const int n = argc > 1 ? std::atoi(argv[1]) : 16000000;
std::vector<double> a(n);
for (int i = 0; i < n; ++i) {
a[i] = 1.0 + static_cast<double>((i + 1) % 7);
}
const double t0 = omp_get_wtime();
double s = 0.0;
#pragma omp parallel for reduction(+ : s)
for (int i = 0; i < n; ++i) {
s += std::sqrt(a[i]);
}
const double t1 = omp_get_wtime();
std::printf("c++ threads=%d n=%d sum=%20.12e time=%10.3e s\n", omp_get_max_threads(), n, s, t1 - t0);
return 0;
}
g++ -std=c++23 -O2 -Wall -Wextra -fopenmp par.cpp -o par_cpp
OMP_NUM_THREADS=1 ./par_cpp 16000000
The timings
Five runs of each binary at each thread count, the loop timed inside the program:
| program | threads | min | median | speedup of the minimum |
|---|---|---|---|---|
| Fortran, gfortran -O2 -fopenmp | 1 | 46.1 ms | 46.2 ms | 1.0 |
| Fortran, gfortran -O2 -fopenmp | 4 | 23.3 ms | 32.8 ms | 2.0 |
| C++, g++ -O2 -fopenmp | 1 | 46.2 ms | 46.2 ms | 1.0 |
| C++, g++ -O2 -fopenmp | 4 | 23.3 ms | 23.3 ms | 2.0 |
GNU Fortran (Alpine 15.2.0) 15.2.0 and G++ (Alpine 15.2.0) 15.2.0, -O2 -fopenmp, on an Intel Core i3-6006U at 2.00 GHz: 4 logical CPUs over 2 physical cores, 3 MB cache, in a Docker container that shares a host. Five runs per row, min and median.
The Fortran row at four threads is the honest one to read twice. Its five runs were 23.3, 23.3, 32.8, 32.9 and 37.0 ms, so the median is 32.8 while the minimum is 23.3; the C++ binary's were 23.3, 23.3, 23.3, 23.5 and 31.8. A shared host schedules containers against each other, and a four-thread run is where that shows. The minimum is the number to compare across languages, because it is the run that got its four threads at once: both land at 23.3 ms.
The digits that move
The two languages print the same sum, and OpenMP changes it. On one thread both print 3.080588135214E+07; on four threads both print 3.080588135563E+07. The loop adds the same sixteen million values in both cases, and the difference is the order: a reduction combines one partial sum per thread, so four threads add four blocks of four million and then combine them, and floating-point addition is not associative.
This is the same lesson as the tolerance page in a different dress: the number a parallel program produces depends on the thread count, so a test that compares against a reference has to allow for it. A program that must have the same answer at any thread count either fixes the combination order by hand or accumulates in a way that does not depend on it, and both cost some of the speedup.
Four CPUs, two cores
The speedup is two, not four, and the reason is the machine rather than the language: nproc reports four logical CPUs and /proc/cpuinfo reports cpu cores: 2 with siblings: 4, which is one physical core per two hardware threads. Two threads share the arithmetic units of one core, so a compute-bound loop gains about the number of cores, and this loop is also memory-bound, which caps it further.
The measurement to repeat on another machine is three lines: nproc, the cpu cores line of /proc/cpuinfo, and the two-thread and four-thread runs of the same binary. A speedup reported without the first two is a number without a context.
Exercises
- Run both binaries at OMP_NUM_THREADS=2. How close to two times is each, and does the Fortran spread shrink?
- Remove the reduction clause from both and run at four threads. What do the sums become, and are they the same? Build with -fcheck=all in Fortran and say what it reports.
- Add schedule(static) and then schedule(dynamic) to both. Which changes the time on this loop, and why is the loop a poor test of scheduling?
- Compile the Fortran program without -fopenmp and run it. What does omp_get_max_threads() return without the flag, and does the program still build?
The programs on this page were built with GNU Fortran (Alpine 15.2.0) 15.2.0 and G++ (Alpine 15.2.0) 15.2.0 and run on Alpine Linux 3.24.3 (x86_64) in a Docker container of 4 logical CPUs (2 cores), on an Intel Core i3-6006U at 2.00 GHz with a 3 MB cache. Every timing is five runs of the same binary, with min and median stated.
Article author: Arthur Isaev