SYNC ALL and SYNC IMAGES

A coarray program is one program running several times, and the only thing that ties the copies together is the synchronisation. Fortran gives three statements and a handful of atomic subroutines: sync all is a barrier for every image, sync images is a barrier for a named set, and sync memory is the memory ordering on its own. This lesson writes all three, follows one image through the program, and reads the error of an image number that does not exist.

SYNC ALL

sync all waits until every image has reached the same statement. Everything an image wrote before it is visible to every image after it, and nothing after it can run until the slowest image arrives. It is the barrier of the language, and it is the one to write when the program has a phase boundary: read the data, then compute, then write.

The file sync.f90:

program syncdemo implicit none integer :: me, n real :: shared[*] me = this_image() n = num_images() shared = 0.0 print '(a,i0,a,i0,a)', 'image ', me, ' of ', n, ': before sync all' sync all print '(a,i0,a)', 'image ', me, ': after sync all' shared = real(me) sync all print '(a,i0,a,f6.1)', 'image ', me, ': shared on image 1 = ', shared[1] sync images(1) print '(a,i0,a)', 'image ', me, ': after sync images(1)' end program syncdemo

gfortran -Wall -Wextra -std=f2018 -fcoarray=single sync.f90 -o sync
./sync

image 1 of 1: before sync all
image 1: after sync all
image 1: shared on image 1 = 1.0
image 1: after sync images(1)

One image, so the barrier has nothing to wait for and the output is the program in order. What the run does show is the shape of the code: every image executes the same four prints, and the values differ only through this_image() and the cosubscripts. With four images the same source would print four blocks, and the first block of every image would appear before the second block of any of them, because of the barrier.

SYNC IMAGES

sync images(1) waits for image one only. The argument may be a list, sync images([1, 3]), or the number of an image that has to reach the same statement, and two images that name each other pair up. That is what a program uses when two images exchange a boundary row and the rest do not care: a full barrier would stop every other image for no reason.

The rule that catches writers is the symmetry: sync images(3) on image one needs a sync images(1) on image three, or the two deadlock. The standard calls this a pairing, and the compiler cannot check it, because which image runs which branch is a run-time decision.

SYNC MEMORY

sync memory orders memory and waits for nobody. It exists for the atomic subroutines, whose operations already carry a memory ordering, and for the rare case where a program has ordered its images by other means, a file or a signal, and only needs the writes to be visible. A program that uses it without an atomic or a sync is a program that has invented its own synchronisation, and the language offers nothing to check it.

What the order buys

The standard divides an image's execution into segments separated by synchronisation statements. A statement before a sync all on image one happens before the statements after it on image two, and the memory model guarantees that the writes are visible. Without a segment boundary there is no such promise: two statements in two images are concurrent, whatever order the prints came out in.

That is the whole of the reasoning a coarray program needs. Write down which writes must be seen by which image, put a synchronisation between them, and check that every image executes the same sequence of barriers. The remaining bugs are the ordinary sequential ones, which is the point of the model.

The check for a wrong image

An image number that does not exist is caught only when the program is built with the runtime checks:

program badsync implicit none sync images(2) print *, 'never reached on one image' end program badsync

gfortran -Wall -Wextra -std=f2018 -fcoarray=single -fcheck=all badsync.f90 -o badsync

GFORTRAN_ERROR_BACKTRACE=0 ./badsync

At line 3 of file badsync.f90
Fortran runtime error: Invalid image number 2 in SYNC IMAGES

Without -fcheck=all the same program compiles and runs, and the statement does something with an image that is not there: on one image the barrier has nothing to wait for and the program continues. The checked run stops with exit status 2. That is the general shape of a coarray bug, and the reason a test run in this course is built with the checks on.

The subroutine form

sync images is a statement, and the language also provides the subroutine sync_images so that an image list and a stat= argument can be computed. The subroutine needs an explicit interface, and a program that names an argument without one is refused:

program badcall implicit none integer :: stat call sync_images([2], stat=stat) print '(a,i0)', 'stat = ', stat end program badcall

gfortran -Wall -Wextra -std=f2018 -fcoarray=single badcall.f90 -o badcall

badcall.f90:4:29:

4 | call sync_images([2], stat=stat)
| 1
Error: Keyword argument requires explicit interface for procedure 'sync_images' at (1)

The fix is the statement form, which takes neither a list variable nor a status and cannot be called with a wrong keyword; or, for the rare program that needs the status, an interface block that declares the subroutine. The course uses the statement, and the error message is the reason.

One image again

Everything on this page was compiled and run with -fcoarray=single, so the barriers had nothing to order and the output is one image's sequence. What the page cannot show is the interesting half: two images that would print different values, or a deadlock from an unpaired sync images. Those need the OpenCoarrays runtime, which is not in this image; lesson 101 shows the real link error and explains what the library adds.

Exercises

  1. Add a second shared = real(me * 10) after the first barrier and print shared[1] without a barrier. What would go wrong with two images?
  2. Move the sync all after the first print to before it. Which lines of the output change on one image?
  3. Change sync images(1) to sync images([1, 2]) and build with -fcheck=all. What happens with one image?
  4. Write the interface block that would let call sync_images([2], stat=stat) compile, and say which library supplies the code.

Questions

A new set of questions in a new order every time. Choose one answer for each, then check.

  1. What does sync all do?

    Answer

    Waits for every image and orders the memory before it against the memory after it

    It is the barrier of the language.

  2. What does sync images(3) require of image three?

    Answer

    A matching sync images(1), or the two images deadlock

    The pairing is symmetric and the compiler cannot check it.

  3. Which statement waits for a named set of images only?

    Answer

    sync images

    sync images takes a list; sync all waits for every image.

  4. What does sync memory do?

    Answer

    Orders memory without waiting for any image

    It is the ordering alone, for the atomic route.

  5. What is a segment?

    Answer

    The part of an image's execution between two synchronisation statements

    Ordering is defined between segments, not within them.

  6. Why does sync images(2) on one image do nothing without -fcheck=all?

    Answer

    The check of the image number is a runtime option, not a compile-time one

    With -fcheck=all the run stops with "Invalid image number".

  7. What does gfortran report for call sync_images([2], stat=stat)?

    Answer

    Error: keyword argument requires explicit interface

    The subroutine form needs an interface block.

  8. Which statement is the barrier for a phase boundary?

    Answer

    sync all

    Every image arrives before any continues.

  9. What can a coarray program not rely on?

    Answer

    The order of statements in two images without a synchronisation

    Without a segment boundary the statements are concurrent.

  10. What does the OpenCoarrays runtime add?

    Answer

    The communication behind remote references, so more than one image can run

    It is what -fcoarray=lib links against; lesson 101.

The programs on this page were built with GNU Fortran (Alpine 15.2.0) 15.2.0 and run on Alpine Linux 3.24.3 (x86_64) in a Docker container.

Article author: Arthur Isaev