A test suite that Cabal runs

The driver of the first page is a program, and a program that a build tool knows how to build and run is a test suite. Cabal has a component for exactly that: a test-suite stanza of type exitcode-stdio-1.0, whose program is built against the library of the package and whose exit status is the result of the run. Cabal prints PASS for 0 and FAIL for anything else.

The package here has a library with the code, a suite that tests it and no executable, because the page is about the suite. It depends on base and on itself, so it builds with --offline on a machine with no package index and no network, which is what this container is.

A suite is a component

Three files make the package: the manifest, the module with the code and the program that checks it. The suite lives in test/ and is not part of the library, so a user of the library never builds it, and the library never imports it.

price/ ├── price.cabal ├── src/Shop/Price.hs └── test/Spec.hs

The working copy is /tmp/price, which is where Cabal writes dist-newstyle/. The command is cabal test --offline from the package folder; it builds the library, then the suite, then runs the suite and reads its status.

The library

The module under test formats an amount of cents as euros, parses the same text back and sums a list of amounts. The three functions are small on purpose: what the page shows is the suite around them, not the arithmetic.

module Shop.Price ( money , parseMoney , total ) where import Data.Char (isDigit) -- | Format a number of cents as euros and cents: 1234 becomes "12.34". money :: Int -> String money cents = whole ++ "." ++ fraction where whole = show (cents `div` 100) rest = show (cents `mod` 100) fraction = replicate (2 - length rest) '0' ++ rest -- | Parse "12.34" into 1234 cents; anything else is Nothing. parseMoney :: String -> Maybe Int parseMoney text = case break (== '.') text of (whole, '.':fraction) | not (null whole) , all isDigit whole , length fraction == 2 , all isDigit fraction -> Just (read whole * 100 + read fraction) _ -> Nothing -- | The sum of a list of amounts in cents. total :: [Int] -> Int total = sum

money pads the cents to two digits with replicate (2 - length rest) '0', which handles 7 as "0.07" and leaves 1234 alone. parseMoney splits on the dot, rejects an empty whole part, a fraction that is not two characters and a character that is not a digit, and returns Nothing for all of them.

The .cabal file

One library and one suite, both with an explicit default-language. The suite declares the library of the same package in its build-depends, which is how it sees Shop.Price without a path:

cabal-version: 3.0 name: price version: 0.1.0.0 build-type: Simple synopsis: A price library with an exitcode-stdio test suite library exposed-modules: Shop.Price hs-source-dirs: src build-depends: base >=4.19 && <5 default-language: GHC2021 test-suite spec type: exitcode-stdio-1.0 main-is: Spec.hs hs-source-dirs: test build-depends: base, price default-language: GHC2021

type: exitcode-stdio-1.0 is the whole contract of a suite: Cabal runs main-is as a program and reads its exit status. The other suite type Cabal knows, detailed-0.9, loads the tests into a runner of its own and needs the Cabal library in the suite; the exit-code type is the one every framework uses underneath, and the one that needs nothing.

The suite program

The same shape as the first page, split into its own file: a record for a check, one helper that compares and builds the message, a list of checks and a driver that prints them and exits with the count of failures. Six checks cover the two formatting cases, the parse, two rejects and the sum.

module Main (main) where import Control.Monad (forM_) import Shop.Price (money, parseMoney, total) import System.Exit (exitFailure, exitSuccess) import Text.Printf (printf) data Check = Check String Bool String expectEqual :: (Eq a, Show a) => String -> a -> a -> Check expectEqual name expected actual | expected == actual = Check name True "" | otherwise = Check name False (printf "expected %s, got %s" (show expected) (show actual)) checks :: [Check] checks = [ expectEqual "format a round amount" "12.00" (money 1200) , expectEqual "format the cents" "0.07" (money 7) , expectEqual "parse an amount" (Just 1234) (parseMoney "12.34") , expectEqual "reject a word" Nothing (parseMoney "twelve") , expectEqual "reject one decimal" Nothing (parseMoney "12.3") , expectEqual "add the amounts" 2050 (total [1200, 700, 150]) ] main :: IO () main = do forM_ checks $ \(Check name passed detail) -> if passed then printf "ok - %s\n" name else printf "not ok - %s: %s\n" name detail let failed = length [() | Check _ False _ <- checks] printf "%d checks, %d failed\n" (length checks) failed if failed == 0 then exitSuccess else exitFailure

The suite imports Shop.Price and nothing else from the package, which is what a test suite should do: it tests the public interface, so a refactor inside the module cannot break the suite for the wrong reason.

build and test

cabal build --offline builds the library and says that the package list is missing, because there is no hackage.haskell.org index here. The build does not need it: both components depend on base, which is part of GHC.

cd /tmp/price
cabal build --offline

Warning: The package list for 'hackage.haskell.org' does not exist. Run 'cabal
update' to download it.
Resolving dependencies...
Build profile: -w ghc-9.10.3 -O1
In order, the following will be built (use -v for more details):
 - price-0.1.0.0 (lib) (first run)
Configuring library for price-0.1.0.0...
Preprocessing library for price-0.1.0.0...
Building library for price-0.1.0.0...
[1 of 1] Compiling Shop.Price ( src/Shop/Price.hs, dist-newstyle/build/x86_64-linux/ghc-9.10.3/price-0.1.0.0/build/Shop/Price.o, dist-newstyle/build/x86_64-linux/ghc-9.10.3/price-0.1.0.0/build/Shop/Price.dyn_o )

cabal test --offline builds the suite against that library, runs it and prints the status. The log path below the status is the one Cabal wrote, build folder and all.

cabal test --offline

Build profile: -w ghc-9.10.3 -O1
In order, the following will be built (use -v for more details):
 - price-0.1.0.0 (test:spec) (first run)
Configuring test suite 'spec' for price-0.1.0.0...
Preprocessing test suite 'spec' for price-0.1.0.0...
Building test suite 'spec' for price-0.1.0.0...
[1 of 1] Compiling Main ( test/Spec.hs, dist-newstyle/build/x86_64-linux/ghc-9.10.3/price-0.1.0.0/t/spec/build/spec/spec-tmp/Main.o )
[2 of 2] Linking dist-newstyle/build/x86_64-linux/ghc-9.10.3/price-0.1.0.0/t/spec/build/spec/spec
Running 1 test suites...
Test suite spec: RUNNING...
ok - format a round amount
ok - format the cents
ok - parse an amount
ok - reject a word
ok - reject one decimal
ok - add the amounts
6 checks, 0 failed
Test suite spec: PASS
Test suite logged to:
/tmp/price/./dist-newstyle/build/x86_64-linux/ghc-9.10.3/price-0.1.0.0/t/spec/test/price-0.1.0.0-spec.log
1 of 1 test suites (1 of 1 test cases) passed.

The status of the command is 0, and PASS is Cabal repeating what the program said. Run it again with nothing changed and it prints the same lines without compiling anything, which makes the test cheap enough to run after every edit.

When a check fails

One expectation is changed on purpose: the cents of seven are expected to print as "0.7" instead of "0.07". Only the suite is recompiled, and the run reports the failure, then the package fails:

cabal test --offline; echo "status $?"

Build profile: -w ghc-9.10.3 -O1
In order, the following will be built (use -v for more details):
 - price-0.1.0.0 (test:spec) (file test/Spec.hs changed)
Preprocessing test suite 'spec' for price-0.1.0.0...
Building test suite 'spec' for price-0.1.0.0...
[1 of 1] Compiling Main ( test/Spec.hs, dist-newstyle/build/x86_64-linux/ghc-9.10.3/price-0.1.0.0/t/spec/build/spec/spec-tmp/Main.o ) [Source file changed]
[2 of 2] Linking dist-newstyle/build/x86_64-linux/ghc-9.10.3/price-0.1.0.0/t/spec/build/spec/spec [Objects changed]
Running 1 test suites...
Test suite spec: RUNNING...
ok - format a round amount
not ok - format the cents: expected "0.7", got "0.07"
ok - parse an amount
ok - reject a word
ok - reject one decimal
ok - add the amounts
6 checks, 1 failed
Test suite spec: FAIL
Test suite logged to:
/tmp/price/./dist-newstyle/build/x86_64-linux/ghc-9.10.3/price-0.1.0.0/t/spec/test/price-0.1.0.0-spec.log
0 of 1 test suites (0 of 1 test cases) passed.
Error: [Cabal-7125]
Tests failed for test:spec from price-0.1.0.0.
status 1

The last two lines are Cabal's, not the suite's: the suite exited 1 because one check failed, and Cabal turns that into a failed test run with its own error code. A build server reads the status of cabal test and needs nothing else, which is why the exit-code suite type is the one to pick.

What the run tells you

The log file under dist-newstyle holds the output of every run, so a failure on a machine you do not have can be read later. The suite name in PASS is the name of the stanza, which is why test-suite spec prints Test suite spec.

The lesson A library, an executable and a test suite builds a package with all three components, and the lesson on running the suite breaks a check and reads the status. The next page here tests a program that is not a library at all, by comparing its whole output with a file.

The package was built with GHC 9.10.3 and cabal-install 3.16.0.0, and run on Alpine Linux 3.24.3 (x86_64) in a Docker container.

Article author: Arthur Isaev

Banner image