Capstone C: testing concurrency

A deterministic test

A concurrency test is useful when it is deterministic: one worker, a queue of two jobs and a result list that must equal a known value. The worker is started, the main thread waits for the finished counter, and the test checks the sorted results against the expected list.

runOnce :: IO Bool runOnce = do ... pure (sortOn fst values == [(1, 4), (2, 6)])

One worker is the choice that makes the order irrelevant and the result exact; the hundred repetitions then show that the test is not flaky, because a bug that depends on a race would fail in at least one of them.

The program

module Main (main) where import Control.Concurrent (forkIO) import Control.Concurrent.STM import Control.Monad (replicateM, replicateM_, when) import Data.List (sortOn) data Job = Job { jobId :: Int , jobCost :: Int } readJob :: TVar [Job] -> STM (Maybe Job) readJob queue = do jobs <- readTVar queue case jobs of [] -> pure Nothing job : rest -> writeTVar queue rest >> pure (Just job) worker :: TVar [Job] -> TVar [(Int, Int)] -> TVar Int -> IO () worker queue results finished = loop where loop = do next <- atomically (readJob queue) case next of Nothing -> atomically (modifyTVar' finished (+ 1)) Just job -> do atomically (modifyTVar' results ((jobId job, jobCost job * 2) :)) loop runOnce :: IO Bool runOnce = do queue <- newTVarIO [Job 1 2, Job 2 3] results <- newTVarIO [] finished <- newTVarIO 0 replicateM_ 1 (forkIO (worker queue results finished)) atomically $ do done <- readTVar finished when (done < 1) retry values <- readTVarIO results pure (sortOn fst values == [(1, 4), (2, 6)]) main :: IO () main = do outcomes <- replicateM 100 runOnce let passed = length (filter id outcomes) putStrLn (show passed ++ " of " ++ show (length outcomes) ++ " runs passed")

ghc -Wall -O1 -threaded -o tests tests.hs
./tests

100 of 100 runs passed

Reading the result

The next lesson measures the same pool with one worker and with four.

Questions

A new set of questions in a new order every time. Choose one answer for each, then check.

  1. Why does the test use one worker?

    Answer

    The result order is fixed, so the expected value is exact

    A single worker makes the completion order the job order.

  2. Why repeat the test a hundred times?

    Answer

    A bug that depends on a race fails in at least one repetition

    Repetition is the cheapest way to expose flakiness.

  3. What does runOnce return?

    Answer

    A Bool that says whether the results were the expected ones

    The Bool is what the repetition counts.

  4. What does this program print? module Main (main) where import Data.List (sortOn) main :: IO () main = print (sortOn fst [(2, 6), (1, 4)] :: [(Int, Int)])

    Answer

    [(1,4),(2,6)]

    sortOn fst orders the pairs by their first component.

  5. What does the finished counter guarantee?

    Answer

    That the results are complete before they are read

    The counter is the join of the worker.

  6. What does this program print? module Main (main) where main :: IO () main = print (length [1, 2, 3 :: Int])

    Answer

    3

    The list has three elements.

  7. What would a test with three workers have to do?

    Answer

    Sort the results before comparing them

    The completion order varies, so the comparison sorts.

  8. What does this program print? module Main (main) where main :: IO () main = print (1 + 1 :: Int)

    Answer

    2

    The sum is two.

  9. What is a flaky test?

    Answer

    One that passes and fails on the same code because of timing

    The result depends on a schedule the test does not control.

  10. What does this program print? module Main (main) where main :: IO () main = print (null [] :: Bool)

    Answer

    True

    The empty list is null.

The code on this page was built with GHC 9.10.3 and run on Alpine Linux 3.24.3 (x86_64) in a Docker container.

Article author: Arthur Isaev

Banner image