Capstone 2: storage in a JSON file
The store is the part that knows where a task lives. The command asks an interface, and two implementations answer: a file store for the real command and, in the tests, a map. The file is JSON, because the model already has tags and a person can read the result; the write is atomic, because a process killed in the middle must not leave half a file.
The Store interface
Five methods, each of which the command needs and nothing more. NextID is separate from Put because the file store keeps the counter in the same document; an implementation with a database sequence would do the same work behind the same method. The interface is small enough that a fake is forty lines.
type Store interface {
All() ([]Task, error)
Get(id int) (Task, error)
Put(task Task) error
Delete(id int) error
NextID() (int, error)
}
// FileStore keeps the tasks in one JSON file.
type FileStore struct {
path string
}
// NewFileStore returns a store that reads and writes path.
func NewFileStore(path string) *FileStore {
return &FileStore{path: path}
}
type fileData struct {
Tasks []Task `json:"tasks"`
Next int `json:"next"`
}
func (s *FileStore) load() (fileData, error) {
var data fileData
raw, err := os.ReadFile(s.path)
if err != nil {
if errors.Is(err, fs.ErrNotExist) {
return fileData{Next: 1}, nil
}
return fileData{}, err
}
if err := json.Unmarshal(raw, &data); err != nil {
return fileData{}, fmt.Errorf("read %s: %w", s.path, err)
}
if data.Next < 1 {
data.Next = 1
}
return data, nil
}
The file layout is a struct with the tasks and the next id. A missing file is not an error: the store returns an empty list and a counter at one, which is what makes the first run of the command work without a setup step. Any other read error, including a corrupt document, is returned with the path.
Load a missing or corrupt file
errors.Is(err, fs.ErrNotExist) is the test for the first run, and the path is added with %w for everything else, so a caller can still unwrap it. A corrupt file is not silently replaced: a program that overwrites the user's tasks because it could not read them is worse than one that stops and says which file is broken.
An atomic save
The save writes a temporary file in the same directory, closes it, and renames it over the target. A rename on one filesystem is atomic, so a reader sees the old file or the new one and never a partial document. The temporary file is removed on every error path with a defer, and the directory matters: a temp file in /tmp and the target on another mount makes the rename fail.
func (s *FileStore) save(data fileData) error {
sort.Slice(data.Tasks, func(i, j int) bool { return data.Tasks[i].ID < data.Tasks[j].ID })
raw, err := json.MarshalIndent(data, "", " ")
if err != nil {
return err
}
raw = append(raw, '\n')
dir := filepath.Dir(s.path)
tmp, err := os.CreateTemp(dir, ".tasks-*.json")
if err != nil {
return err
}
name := tmp.Name()
defer os.Remove(name)
if _, err := tmp.Write(raw); err != nil {
tmp.Close()
return err
}
if err := tmp.Close(); err != nil {
return err
}
return os.Rename(name, s.path)
}
// All returns every task in id order.
The tasks are sorted by id before the write, so the file does not depend on the order the map happened to have; the counter is part of the document, so an id is never reused after a delete. The indentation is for a person reading the file and for a diff in a backup.
The operations
Each method loads, changes and saves, which is one file read and one write per operation. That is fine for a command that runs once per keystroke and too slow for a server; the notes service of capstone 3 keeps the same shape and says where it would switch to a database. The mutex makes one store safe for several goroutines, which the command does not need and a test with parallel runs does.
func (s *FileStore) All() ([]Task, error) {
data, err := s.load()
if err != nil {
return nil, err
}
return data.Tasks, nil
}
// Get returns one task or ErrNotFound.
func (s *FileStore) Get(id int) (Task, error) {
data, err := s.load()
if err != nil {
return Task{}, err
}
for _, task := range data.Tasks {
if task.ID == id {
return task, nil
}
}
return Task{}, ErrNotFound
}
// Put adds or replaces a task.
func (s *FileStore) Put(task Task) error {
data, err := s.load()
if err != nil {
return err
}
replaced := false
for i := range data.Tasks {
if data.Tasks[i].ID == task.ID {
data.Tasks[i] = task
replaced = true
break
}
}
if !replaced {
data.Tasks = append(data.Tasks, task)
}
return s.save(data)
}
// Delete removes a task or returns ErrNotFound.
func (s *FileStore) Delete(id int) error {
data, err := s.load()
if err != nil {
return err
}
for i := range data.Tasks {
if data.Tasks[i].ID == id {
data.Tasks = append(data.Tasks[:i], data.Tasks[i+1:]...)
return s.save(data)
}
}
return ErrNotFound
}
// NextID returns the next free id and advances the counter.
func (s *FileStore) NextID() (int, error) {
data, err := s.load()
if err != nil {
return 0, err
}
id := data.Next
data.Next++
return id, s.save(data)
}
Put replaces a task with the same id and appends the others, so an update is a put. Delete returns ErrNotFound for an id that is not there, and NextID advances the counter even before the task is stored, which can leave a gap after a failed add; a gap is harmless and a reused id is not.
Memory against file
The demo runs the same six operations against the memory store and against a file store in a temporary directory. The outputs are identical, which is the point of the interface, and the last line shows the file is not empty.
package main
import (
"fmt"
"os"
"path/filepath"
"time"
"example.com/tasks/tasks"
)
func main() {
dir, err := os.MkdirTemp("", "tasks")
if err != nil {
panic(err)
}
defer os.RemoveAll(dir)
path := filepath.Join(dir, "tasks.json")
store := tasks.NewFileStore(path)
empty, err := store.All()
fmt.Println("empty:", len(empty), "err:", err)
id, err := store.NextID()
fmt.Println("next id:", id, "err:", err)
task, err := tasks.New(id, "buy milk", time.Date(2026, 11, 20, 0, 0, 0, 0, time.UTC))
if err != nil {
panic(err)
}
if err := store.Put(task); err != nil {
panic(err)
}
again, err := store.NextID()
fmt.Println("next id:", again, "err:", err)
all, err := store.All()
fmt.Println("tasks:", len(all), "err:", err)
for _, t := range all {
fmt.Println(t)
}
info, err := os.Stat(path)
fmt.Println("file bytes > 0:", info.Size() > 0, "err:", err)
if err := os.WriteFile(path, []byte("{"), 0o644); err != nil {
panic(err)
}
_, err = store.All()
fmt.Println("corrupt:", err != nil)
}
go run ./cmd/l122
memory create: {ID:1 Title:buy milk Done:false} err=<nil>
memory create: {ID:2 Title:write lesson Done:false} err=<nil>
memory update: {ID:1 Title:buy milk and bread Done:true} err=<nil>
memory get: {ID:2 Title:write lesson Done:false} err=<nil>
memory list: 1 notes err=<nil>
memory missing: note not found
file create: {ID:1 Title:buy milk Done:false} err=<nil>
file create: {ID:2 Title:write lesson Done:false} err=<nil>
file update: {ID:1 Title:buy milk and bread Done:true} err=<nil>
file get: {ID:2 Title:write lesson Done:false} err=<nil>
file list: 1 notes err=<nil>
file missing: note not found
file bytes > 0: true err: <nil>
The two stores agree on every line, including the sentinel error, and the file one wrote a real document. A fake store in the tests is the third implementation, and it is the one that makes the command tests fast and independent of a disk.
One interface, two stores
The interface is the seam of the whole capstone: the model does not know it, the command only knows it, and the tests replace it. The file store adds the two details a real store needs, an atomic write and a sentinel for a missing id, and the memory store is the same interface with a map. A third store backed by a database would fit without a change to the command.
The next lesson builds the commands and the dispatch table of the CLI on top of this interface.
Exercises
- Remove the sort from save and add two tasks in reverse id order. Does the file change, and what does the list command print?
- Make load treat a corrupt file as empty and run the demo. What is lost, and why is the original behaviour safer?
- Write the store with os.WriteFile directly and kill the program during a save. What can the file contain, and how does the rename prevent it?
Questions
A new set of questions in a new order every time. Choose one answer for each, then check.
-
Why does the file store keep the next id in the file?
Answer
An id must not be reused after a delete
It is part of the document.
-
What does a missing file read as?
Answer
An empty store with the counter at one
The first run needs no setup.
-
Why write to a temporary file and rename?
Answer
A rename is atomic, so a reader never sees half a file
A killed process leaves the old file.
-
Why must the temporary file be in the same directory?
Answer
A rename across filesystems is not atomic and fails
The store creates it there.
-
What happens to a corrupt file on load?
Answer
The error is returned with the path
A user must not lose tasks silently.
-
What does Put do for an existing id?
Answer
Replaces the task
An update is a put.
-
What does Delete return for a missing id?
Answer
ErrNotFound
The command maps it to exit 3.
-
Why does the store have a mutex?
Answer
One store can be used by several goroutines
The command itself does not need it.
-
What is the third implementation of the interface?
Answer
The fake store in the tests
It makes the command tests fast.
-
Why sort the tasks before saving?
Answer
The file does not depend on map order
A stable file is a readable diff.
The code on this page was run with go 1.26.10 on Alpine Linux 3.24.3 (x86_64) in a Docker container.
Article author: Arthur Isaev