Capstone C: testing concurrency
A deterministic test
A concurrency test is useful when it is deterministic: one worker, a queue of two jobs and a result list that must equal a known value. The worker is started, the main thread waits for the finished counter, and the test checks the sorted results against the expected list.
runOnce :: IO Bool
runOnce = do
...
pure (sortOn fst values == [(1, 4), (2, 6)])
One worker is the choice that makes the order irrelevant and the result exact; the hundred repetitions then show that the test is not flaky, because a bug that depends on a race would fail in at least one of them.
The program
module Main (main) where
import Control.Concurrent (forkIO)
import Control.Concurrent.STM
import Control.Monad (replicateM, replicateM_, when)
import Data.List (sortOn)
data Job = Job
{ jobId :: Int
, jobCost :: Int
}
readJob :: TVar [Job] -> STM (Maybe Job)
readJob queue = do
jobs <- readTVar queue
case jobs of
[] -> pure Nothing
job : rest -> writeTVar queue rest >> pure (Just job)
worker :: TVar [Job] -> TVar [(Int, Int)] -> TVar Int -> IO ()
worker queue results finished = loop
where
loop = do
next <- atomically (readJob queue)
case next of
Nothing -> atomically (modifyTVar' finished (+ 1))
Just job -> do
atomically (modifyTVar' results ((jobId job, jobCost job * 2) :))
loop
runOnce :: IO Bool
runOnce = do
queue <- newTVarIO [Job 1 2, Job 2 3]
results <- newTVarIO []
finished <- newTVarIO 0
replicateM_ 1 (forkIO (worker queue results finished))
atomically $ do
done <- readTVar finished
when (done < 1) retry
values <- readTVarIO results
pure (sortOn fst values == [(1, 4), (2, 6)])
main :: IO ()
main = do
outcomes <- replicateM 100 runOnce
let passed = length (filter id outcomes)
putStrLn (show passed ++ " of " ++ show (length outcomes) ++ " runs passed")
ghc -Wall -O1 -threaded -o tests tests.hs
./tests
100 of 100 runs passed
Reading the result
- One run proves the result; a hundred runs prove that no scheduling made it differ.
- The line names the pass count and the total, so a failure would be visible without reading the code.
- A test that must be repeated to pass is a race, and the count is what makes it visible.
The next lesson measures the same pool with one worker and with four.
Questions
A new set of questions in a new order every time. Choose one answer for each, then check.
-
Why does the test use one worker?
Answer
The result order is fixed, so the expected value is exact
A single worker makes the completion order the job order.
-
Why repeat the test a hundred times?
Answer
A bug that depends on a race fails in at least one repetition
Repetition is the cheapest way to expose flakiness.
-
What does
runOncereturn?Answer
A Bool that says whether the results were the expected ones
The Bool is what the repetition counts.
-
What does this program print?
module Main (main) where import Data.List (sortOn) main :: IO () main = print (sortOn fst [(2, 6), (1, 4)] :: [(Int, Int)])Answer
[(1,4),(2,6)]
sortOn fst orders the pairs by their first component.
-
What does the finished counter guarantee?
Answer
That the results are complete before they are read
The counter is the join of the worker.
-
What does this program print?
module Main (main) where main :: IO () main = print (length [1, 2, 3 :: Int])Answer
3
The list has three elements.
-
What would a test with three workers have to do?
Answer
Sort the results before comparing them
The completion order varies, so the comparison sorts.
-
What does this program print?
module Main (main) where main :: IO () main = print (1 + 1 :: Int)Answer
2
The sum is two.
-
What is a flaky test?
Answer
One that passes and fails on the same code because of timing
The result depends on a schedule the test does not control.
-
What does this program print?
module Main (main) where main :: IO () main = print (null [] :: Bool)Answer
True
The empty list is null.
The code on this page was built with GHC 9.10.3 and run on Alpine Linux 3.24.3 (x86_64) in a Docker container.
Article author: Arthur Isaev