Capstone 3: the store
The service needs a store with the five operations the API calls, and two implementations cover the two modes of running it. The memory store is the default and the one the tests use; the file store is chosen by setting DB_FILE, and it survives a restart. Both are safe for the concurrent handlers the HTTP server runs.
The interface
The same interface as the CLI capstone, with the note type from lesson 129. The API depends on it and on nothing else, so a fake in the tests and a database in a later version fit without a handler change.
package store
import "errors"
// ErrNotFound is returned for an id that does not exist.
var ErrNotFound = errors.New("note not found")
// ErrEmptyTitle is returned for a note with no title.
var ErrEmptyTitle = errors.New("title is required")
// Note is one note of the service.
type Note struct {
ID int `json:"id"`
Title string `json:"title"`
Done bool `json:"done"`
}
// Store is what the API needs from its storage.
type Store interface {
List() ([]Note, error)
Get(id int) (Note, error)
Create(title string) (Note, error)
Update(id int, title string, done bool) (Note, error)
Delete(id int) error
}
The rules live in the store too: a title is trimmed and an empty one is an error, and a missing id is the sentinel. A handler cannot create a note without a title, whichever store it was given, which is the reason the rule is not repeated in the API.
The memory store
A map, a counter and a mutex. The lock covers the whole method, so a reader never sees a half-written map and two creates never take the same id. List copies the map into a slice and sorts by id, which is the order the API returns.
package store
import (
"sort"
"strings"
"sync"
)
// Memory is a Store in a map, safe for concurrent use.
type Memory struct {
mu sync.Mutex
notes map[int]Note
next int
}
// NewMemory returns an empty memory store.
func NewMemory() *Memory {
return &Memory{notes: map[int]Note{}, next: 1}
}
func (m *Memory) List() ([]Note, error) {
m.mu.Lock()
defer m.mu.Unlock()
out := make([]Note, 0, len(m.notes))
for _, note := range m.notes {
out = append(out, note)
}
sort.Slice(out, func(i, j int) bool { return out[i].ID < out[j].ID })
return out, nil
}
func (m *Memory) Get(id int) (Note, error) {
m.mu.Lock()
defer m.mu.Unlock()
note, ok := m.notes[id]
if !ok {
return Note{}, ErrNotFound
}
return note, nil
}
func (m *Memory) Create(title string) (Note, error) {
title = strings.TrimSpace(title)
if title == "" {
return Note{}, ErrEmptyTitle
}
m.mu.Lock()
defer m.mu.Unlock()
note := Note{ID: m.next, Title: title}
m.next++
m.notes[note.ID] = note
return note, nil
}
func (m *Memory) Update(id int, title string, done bool) (Note, error) {
title = strings.TrimSpace(title)
if title == "" {
return Note{}, ErrEmptyTitle
}
m.mu.Lock()
defer m.mu.Unlock()
note, ok := m.notes[id]
if !ok {
return Note{}, ErrNotFound
}
note.Title = title
note.Done = done
m.notes[id] = note
return note, nil
}
func (m *Memory) Delete(id int) error {
m.mu.Lock()
defer m.mu.Unlock()
if _, ok := m.notes[id]; !ok {
return ErrNotFound
}
delete(m.notes, id)
return nil
}
The mutex is a sync.Mutex and not a RWMutex, because the methods are short and a read still needs the map to be stable. A read-only mode with many readers and a rare write is the case for the read-write lock, and a service with one write a second does not have it.
The file store
The file store keeps the notes and the counter in one JSON document and reloads it on every operation. The mutex makes the read-modify-write of one call atomic with respect to other calls in the same process, which is the guarantee the API needs. The layout is the same as the CLI store, and the code is the same shape with the note type.
return fileData{}, err
}
var data fileData
if err := json.Unmarshal(raw, &data); err != nil {
return fileData{}, fmt.Errorf("read %s: %w", f.path, err)
}
if data.Next < 1 {
data.Next = 1
}
return data, nil
}
func (f *File) save(data fileData) error {
sort.Slice(data.Notes, func(i, j int) bool { return data.Notes[i].ID < data.Notes[j].ID })
raw, err := json.MarshalIndent(data, "", " ")
if err != nil {
return err
}
raw = append(raw, '\n')
tmp, err := os.CreateTemp(filepath.Dir(f.path), ".notes-*.json")
if err != nil {
return err
}
name := tmp.Name()
defer os.Remove(name)
if _, err := tmp.Write(raw); err != nil {
tmp.Close()
return err
}
if err := tmp.Close(); err != nil {
return err
}
return os.Rename(name, f.path)
}
func (f *File) List() ([]Note, error) {
f.mu.Lock()
defer f.mu.Unlock()
data, err := f.load()
return data.Notes, err
}
func (f *File) Get(id int) (Note, error) {
f.mu.Lock()
defer f.mu.Unlock()
data, err := f.load()
if err != nil {
return Note{}, err
}
for _, note := range data.Notes {
if note.ID == id {
return note, nil
}
}
return Note{}, ErrNotFound
}
func (f *File) Create(title string) (Note, error) {
title = strings.TrimSpace(title)
if title == "" {
return Note{}, ErrEmptyTitle
}
f.mu.Lock()
defer f.mu.Unlock()
load treats a missing file as empty and returns any other error with the path, and save writes a temporary file and renames it. A corrupt document is reported, not overwritten: a service that loses its notes because it could not parse them is worse than one that refuses to start.
The atomic save
The same trick as the CLI: a temporary file in the same directory, a close, a rename. The mutex is what makes two concurrent requests take turns; the rename is what makes a crash leave either the old document or the new one. A database gives both guarantees with more machinery, which is the trade the next section names.
data, err := f.load()
if err != nil {
return Note{}, err
}
note := Note{ID: data.Next, Title: title}
data.Next++
data.Notes = append(data.Notes, note)
if err := f.save(data); err != nil {
return Note{}, err
}
return note, nil
}
func (f *File) Update(id int, title string, done bool) (Note, error) {
title = strings.TrimSpace(title)
if title == "" {
return Note{}, ErrEmptyTitle
}
f.mu.Lock()
defer f.mu.Unlock()
data, err := f.load()
if err != nil {
return Note{}, err
}
for i := range data.Notes {
if data.Notes[i].ID == id {
data.Notes[i].Title = title
data.Notes[i].Done = done
if err := f.save(data); err != nil {
return Note{}, err
}
return data.Notes[i], nil
}
}
return Note{}, ErrNotFound
}
func (f *File) Delete(id int) error {
f.mu.Lock()
defer f.mu.Unlock()
data, err := f.load()
if err != nil {
return err
}
for i := range data.Notes {
if data.Notes[i].ID == id {
data.Notes = append(data.Notes[:i], data.Notes[i+1:]...)
return f.save(data)
}
}
return ErrNotFound
}
The delete copies the slice around the removed element and saves; the append and the copy are the two places a slice aliases its backing array, and neither escapes the store. The file is small, the code is short, and the cost is one read and one write per request, which the benchmark of lesson 127 measured for the CLI.
The same operations
The demo runs create, create, update, get, delete and a missing get against the memory store and against a file store in a temporary directory. The two agree on every line.
package main
import (
"fmt"
"os"
"path/filepath"
"example.com/notesapi/internal/store"
)
func demo(name string, st store.Store) {
first, err := st.Create(" buy milk ")
fmt.Printf("%s create: %+v err=%v\n", name, first, err)
second, err := st.Create("write lesson")
fmt.Printf("%s create: %+v err=%v\n", name, second, err)
updated, err := st.Update(first.ID, "buy milk and bread", true)
fmt.Printf("%s update: %+v err=%v\n", name, updated, err)
got, err := st.Get(second.ID)
fmt.Printf("%s get: %+v err=%v\n", name, got, err)
if err := st.Delete(first.ID); err != nil {
panic(err)
}
all, err := st.List()
fmt.Printf("%s list: %d notes err=%v\n", name, len(all), err)
_, err = st.Get(first.ID)
fmt.Printf("%s missing: %v\n", name, err)
}
func main() {
demo("memory", store.NewMemory())
dir, err := os.MkdirTemp("", "notes")
if err != nil {
panic(err)
}
defer os.RemoveAll(dir)
path := filepath.Join(dir, "notes.json")
demo("file ", store.NewFile(path))
info, err := os.Stat(path)
fmt.Println("file bytes > 0:", info.Size() > 0, "err:", err)
}
go run ./cmd/l130
memory create: {ID:1 Title:buy milk Done:false} err=<nil>
memory create: {ID:2 Title:write lesson Done:false} err=<nil>
memory update: {ID:1 Title:buy milk and bread Done:true} err=<nil>
memory get: {ID:2 Title:write lesson Done:false} err=<nil>
memory list: 1 notes err=<nil>
memory missing: note not found
file create: {ID:1 Title:buy milk Done:false} err=<nil>
file create: {ID:2 Title:write lesson Done:false} err=<nil>
file update: {ID:1 Title:buy milk and bread Done:true} err=<nil>
file get: {ID:2 Title:write lesson Done:false} err=<nil>
file list: 1 notes err=<nil>
file missing: note not found
file bytes > 0: true err: <nil>
The outputs are identical and the file is not empty, which is the contract of the interface. The demo is also the test the service does not need: the API tests use a fake, and one end-to-end run with the file store covers the difference.
What a database would change
A database would move the atomicity into a transaction, the ordering into an index and the durability into a write-ahead log, and it would let two processes share the data. The interface would not change: the five methods are the same, and the handlers would not know. What would change is the configuration, the migration story and the deployment, which is why a service starts with the memory store and moves when the data has to outlive the process.
The next lesson writes the handlers that use these methods and the statuses that go with them.
Exercises
- Run the file store demo twice in the same directory and compare the ids. What does the counter in the file do, and what would a sequence in a database do?
- Change the memory store to an RWMutex and make Get use RLock. Is the code still correct with List sorting under the lock?
- Make the file store cache the document in memory and write on a timer. What does a request see after a crash, and what does the interface still promise?
Questions
A new set of questions in a new order every time. Choose one answer for each, then check.
-
Which package defines the Store interface?
Answer
internal/store, next to the model
The API imports it.
-
Why a mutex in the memory store?
Answer
Handlers run concurrently and the map is not safe
Two creates must not share an id.
-
What order does List return?
Answer
By id
It sorts before returning.
-
What does a missing file read as in the file store?
Answer
An empty document with the counter at one
The first start needs no setup.
-
What makes a save atomic?
Answer
A temporary file and a rename
A rename on one filesystem is atomic.
-
What does the mutex protect in the file store?
Answer
The read-modify-write of one call
Two calls take turns.
-
What happens to a corrupt file?
Answer
The error is returned with the path
A user must not lose notes silently.
-
Which store is the default?
Answer
The memory store
DB_FILE selects the file store.
-
What would a database change?
Answer
Transactions, indexes and durability, not the interface
The interface is the seam.
-
Why is the file store enough for this service?
Answer
The data is small and the traffic is one process
The next step is a database when it outgrows that.
The code on this page was run with go 1.26.10 on Alpine Linux 3.24.3 (x86_64) in a Docker container.
Article author: Arthur Isaev