Benchmarks, the race detector and coverage

Three tools of the same command answer three different questions. A benchmark says how long a function takes and what it allocates; a profile says where the time went inside it; the race detector says whether two goroutines touched the same memory without a lock; coverage says which statements no test ran. All four run under go test, and the numbers below are from this machine.

Three benchmarks

The package joins twenty strings of a thousand bytes. The naive version builds the result with +=, the builder version appends to a strings.Builder that grows as it needs, and the third knows the total length and calls Grow once. The benchmarks use b.Loop, which runs the body as many times as the benchmark needs and keeps the compiler from removing a call whose result is unused.

package text import "strings" // JoinSlow concatenates parts with the + operator. func JoinSlow(parts []string) string { out := "" for _, p := range parts { out += p } return out } // JoinGrow concatenates parts with a builder whose size is known in advance. func JoinGrow(parts []string) string { n := 0 for _, p := range parts { n += len(p) } var b strings.Builder b.Grow(n) for _, p := range parts { b.WriteString(p) } return b.String() } // JoinFast concatenates parts with a strings.Builder. func JoinFast(parts []string) string { var b strings.Builder for _, p := range parts { b.WriteString(p) } return b.String() } // Checksum returns the FNV-1a hash of data. func Checksum(data []byte) uint64 { h := uint64(14695981039346656037) for _, b := range data { h ^= uint64(b) h *= 1099511628211 } return h }

package text import ( "strings" "testing" ) func TestJoinFast(t *testing.T) { if got, want := JoinFast([]string{"a", "b", "c"}), "abc"; got != want { t.Errorf("JoinFast = %q, want %q", got, want) } } var parts = makeParts(20, 1000) // makeParts returns n strings of size bytes each. func makeParts(n, size int) []string { out := make([]string, n) for i := range out { out[i] = strings.Repeat("x", size) } return out } func BenchmarkJoinSlow(b *testing.B) { for b.Loop() { JoinSlow(parts) } } func BenchmarkJoinGrow(b *testing.B) { for b.Loop() { JoinGrow(parts) } } func BenchmarkJoinFast(b *testing.B) { for b.Loop() { JoinFast(parts) } } var payload = make([]byte, 1<<20) func BenchmarkChecksum(b *testing.B) { for b.Loop() { Checksum(payload) } }

go test -bench=. -benchmem ./text

goos: linux
goarch: amd64
pkg: example.com/bench/text
cpu: Intel(R) Core(TM) i3-6006U CPU @ 2.00GHz
BenchmarkJoinSlow-4 17894 83189 ns/op 218496 B/op 19 allocs/op
BenchmarkJoinGrow-4 212194 5529 ns/op 20480 B/op 1 allocs/op
BenchmarkJoinFast-4 54013 23355 ns/op 79360 B/op 9 allocs/op
BenchmarkChecksum-4 564 2154032 ns/op 0 B/op 0 allocs/op
PASS
ok example.com/bench/text 5.149s

Reading the numbers

The columns are the name, the number of iterations the harness chose, the time per operation, the bytes allocated per operation and the number of allocations per operation. The suffix -4 is GOMAXPROCS, the number of threads the benchmark was allowed; a different machine gives a different suffix and different numbers.

The naive join is fifteen times slower than the grown builder and allocates 218 KB against 20 KB, because every += copies the string so far into a larger one. The middle version already helps and still allocates nine times, because a builder without a size hint doubles its buffer as it grows. That is the reason the hint is worth the four lines: one allocation for the whole result.

A benchmark number is only comparable with a run of the same benchmark on the same machine. For a decision, run it twice with -count=3 and compare the medians:

go test -bench=BenchmarkJoinGrow -benchmem -count=3 ./text

BenchmarkJoinGrow-4 122568 9561 ns/op 20480 B/op 1 allocs/op
BenchmarkJoinGrow-4 195362 5914 ns/op 20480 B/op 1 allocs/op
BenchmarkJoinGrow-4 222902 6155 ns/op 20480 B/op 1 allocs/op

The first run is the slowest of the three: the process is cold, the pages of the new allocation are faulted in and the CPU has not settled at its clock, which is why the harness warms up before it measures and why the first number is the one to throw away. The last two differ by four percent, and that is the noise of this container. A difference smaller than the spread is not a result; the tool for comparing two such runs and reporting a confidence interval is benchstat, which is not installed in this image and is not shown here.

A CPU profile

-cpuprofile writes samples taken while the benchmark runs, and go tool pprof reads the file. The checksum benchmark is pure computation, so the profile has one obvious answer:

go test -bench=BenchmarkChecksum -benchtime=500x -cpuprofile=/tmp/cpu.out ./text
go tool pprof -top -nodecount=6 /tmp/cpu.out

File: text.test
Build ID: f657235d5b9bf65a450b426c25933773ab25ed5c
Type: cpu
Time: 2026-11-18 02:37:32 UTC
Duration: 1.06s, Total samples = 1.06s (99.67%)
Showing nodes accounting for 1.06s, 100% of 1.06s total
flat flat% sum% cum cum%
1.06s 100% 100% 1.06s 100% example.com/bench/text.Checksum (inline)
0 0% 100% 1.06s 100% example.com/bench/text.BenchmarkChecksum
0 0% 100% 1.06s 100% testing.(*B).run1.func1
0 0% 100% 1.06s 100% testing.(*B).runN

flat is the time in the function itself and cum the time in it and everything it called. The hash loop is inlined, so every sample lands on Checksum and the benchmark functions above it show zero of their own. A profile of the join benchmark instead shows runtime.memmove and the garbage collector at the top, which is the honest answer for code that copies strings.

The same flag pair exists for memory with -memprofile, and go tool pprof -http=:8080 /tmp/cpu.out serves the same data as a web page with a flame graph. The container has no browser, so this page stops at the text output.

The race detector

Two goroutines add to the same integer without a lock. Every one of the ten goroutines increments a thousand times, so the correct total is ten thousand, and the detector catches the first pair that overlaps:

package race // Counter counts without a lock. type Counter struct { n int } // Increment adds one. func (c *Counter) Increment() { c.n++ } // Value returns the count. func (c *Counter) Value() int { return c.n }

package race import ( "sync" "testing" ) func TestCounter(t *testing.T) { var c Counter var wg sync.WaitGroup for range 10 { wg.Add(1) go func() { defer wg.Done() for range 1000 { c.Increment() } }() } wg.Wait() if got := c.Value(); got != 10000 { t.Errorf("Value = %d, want 10000", got) } }

go test -trimpath -race -run TestCounter ./race

==================
WARNING: DATA RACE
Read at 0x00c0000182a8 by goroutine 12:
example.com/bench/race.(*Counter).Increment()
example.com/bench/race/race.go:10 +0x9c
example.com/bench/race.TestCounter.func1()
example.com/bench/race/race_test.go:16 +0x97

Previous write at 0x00c0000182a8 by goroutine 18:
example.com/bench/race.(*Counter).Increment()
example.com/bench/race/race.go:10 +0xae
example.com/bench/race.TestCounter.func1()
example.com/bench/race/race_test.go:16 +0x97

Goroutine 12 (running) created at:
example.com/bench/race.TestCounter()
example.com/bench/race/race_test.go:13 +0x84
testing.tRunner()
testing/testing.go:2036 +0x21c
...
--- FAIL: TestCounter (0.00s)
race_test.go:22: Value = 8639, want 10000
testing.go:1712: race detected during execution of test
FAIL
FAIL example.com/bench/race 0.023s
FAIL

The report gave two races and the middle one is cut above. Three things are in it: the two accesses, with the line of the increment and the address of the field, the goroutines and the line that started them. The address changes on every run, because the heap does; the line numbers do not, which is why the report is worth reading.

The detector is not a proof. It only sees the pairs whose execution overlapped, so a run that misses the bad interleaving reports nothing. It costs about a factor of two to ten in speed and memory, which is why CI runs it on one job and the daily loop runs without it. The fix for this program is a sync.Mutex or an atomic.Int64, both later in the course: sync.Mutex and shared state and sync/atomic.

Coverage

Coverage counts the statements a test executed, not the cases it thought of. The text package has one test for JoinFast, and the other three functions are only called by benchmarks, which do not count:

go test -coverprofile=/tmp/cover.out ./text
go tool cover -func=/tmp/cover.out

ok example.com/bench/text 0.005s coverage: 19.0% of statements
example.com/bench/text/text.go:6: JoinSlow 0.0%
example.com/bench/text/text.go:15: JoinGrow 0.0%
example.com/bench/text/text.go:29: JoinFast 100.0%
example.com/bench/text/text.go:38: Checksum 0.0%
total: (statements) 19.0%

The per-function list is more useful than the total: it names the four functions and shows that one of them is tested. go tool cover -html=/tmp/cover.out writes the source with the covered lines in green and the others in red, and the total is what a CI badge usually prints, with all the distortion that implies: a getter executed by every test counts as much as a branch nobody tried.

The lessons behind these tools are benchmarks and benchstat, the race detector and coverage and profiling with pprof. The next page runs the tools that find mistakes before a test does: go vet, staticcheck and CI.

The code on this page was run with go 1.26.10 on Alpine Linux 3.24.3 (x86_64) in a Docker container, on an Intel Core i3-6006U at 2.00 GHz with 4 CPUs.

Article author: Arthur Isaev