Build automation and CI

A test suite is useful when one command runs it and its status is the answer. The command is a make target, the status is the process exit code, and a continuous integration service is only a machine that runs the command after every push. Everything in this lesson runs locally; the CI file itself does not.

A check target

The target check depends on the test binary, so make builds it when needed and then runs it. The recipe is one line, and make prints both the compile and the run. A zero exit status from the binary is a passing check.

#include <stdio.h> static int add(int a, int b) { return a + b; } int main(void) { int failures = 0; if (add(2, 2) != 4) { printf("failed: add(2, 2)\n"); failures++; } if (add(1, 1) != 2) { printf("failed: add(1, 1)\n"); failures++; } printf("checks: 2, failures: %d\n", failures); return failures == 0 ? 0 : 1; }

vi Makefile

CC = gcc CFLAGS = -Wall -Wextra .PHONY: check clean check: tests ./tests tests: test.c $(CC) $(CFLAGS) test.c -o tests clean: rm -f tests

make check; echo "status: $?"

gcc -Wall -Wextra test.c -o tests
./tests
checks: 2, failures: 0
status: 0

The .PHONY line from lesson 20 keeps check running even if a file of that name appears. The target name is a convention; a project may also use test or all, but the CI script and the developers must agree on one.

A failing check

The same target with one wrong expectation fails, and the failure travels through two layers: the test binary returns 1, and make reports the recipe as an error and exits with status 2. The CI service only has to look at that number, not at the text.

#include <stdio.h> static int add(int a, int b) { return a + b; } int main(void) { int failures = 0; if (add(2, 2) != 4) { printf("failed: add(2, 2)\n"); failures++; } if (add(1, 1) != 3) { printf("failed: add(1, 1)\n"); failures++; } printf("checks: 2, failures: %d\n", failures); return failures == 0 ? 0 : 1; }

make check; echo "status: $?"

gcc -Wall -Wextra test.c -o tests
./tests
failed: add(1, 1)
checks: 2, failures: 1
make: *** [Makefile:7: check] Error 1
status: 2

The first number in the make message is the line of the failing recipe, and Error 1 is the exit status of ./tests. Make exits with 2 for any failed recipe, so the two codes mean different things: the test says what failed, make says that the build step did.

A test script

#include <stdio.h> static int add(int a, int b) { return a + b; } int main(void) { int failures = 0; if (add(2, 2) != 4) { printf("failed: add(2, 2)\n"); failures++; } if (add(1, 1) != 2) { printf("failed: add(1, 1)\n"); failures++; } printf("checks: 2, failures: %d\n", failures); return failures == 0 ? 0 : 1; }

A shell script wraps the command with the steps that must precede it, such as creating a directory or choosing a compiler. The important line is set -e: without it the script continues after a failed command and returns the status of its last line, which is zero. The same script then lies about the tests.

vi bad.sh

#!/bin/sh make check echo "script done"

vi good.sh

#!/bin/sh set -e make check echo "script done"

./bad.sh; echo "status: $?"
./good.sh; echo "status: $?"

./tests
failed: add(1, 1)
checks: 2, failures: 1
make: *** [Makefile:7: check] Error 1
script done
status: 0
./tests
failed: add(1, 1)
checks: 2, failures: 1
make: *** [Makefile:7: check] Error 1
status: 2

The bad script printed its final message and returned 0, which a CI system would treat as success. The good script stopped at the failed make and returned its status. The missing set -e is the most common reason a red build looks green.

The C program that both scripts run is the test printed above; the directory used for this run contains the failing variant from the previous section, so the output shows its failure. The script itself contains no C, which is why its text sits in a shell block rather than a code block.

CI in one script

A CI service runs the same commands on a fresh machine: install the compiler, build, test, report. The script below is that sequence, and it runs here. The first step is a small program that prints the compiler and the language level, so the log records what the build actually used.

#include <stdio.h> int main(void) { printf("compiler %s\n", __VERSION__); printf("standard %ld\n", __STDC_VERSION__); return 0; }

vi ci.sh

#!/bin/sh set -e gcc -Wall -Wextra probe.c -o probe ./probe make check echo "ci ok"

./ci.sh; echo "status: $?"

compiler 15.2.0
standard 202311
gcc -Wall -Wextra test.c -o tests
./tests
checks: 2, failures: 0
ci ok
status: 0

On a hosted service the same steps live in a YAML file, for example a job that checks out the repository, installs build-base and runs ./ci.sh. That file is not shown as executed here: this container has no CI runner, and the page only claims what the local script did. The script is the part that can be tested on the developer machine.

The other failure a CI job meets early is a misspelled target: make has no way to know that check-all was meant to be check, so it reports that no rule exists and exits with status 2 before running anything.

make check-all; echo "status: $?"

make: *** No rule to make target 'check-all'. Stop.
status: 2

Exercises

1. Add a clean step to the CI script and check that the next make check rebuilds the test binary.
2. Change the script to write its output to ci.log with tee and check the exit status.
3. Remove set -e from the good script, run it in the failing directory, and explain why the status changes.

The code on this page was run with gcc (Alpine 15.2.0) 15.2.0 and GNU Make 4.4.1 on Alpine Linux 3.24.3 (x86_64) in a Docker container.

Article author: Arthur Isaev