Capstone 3: the store

The service needs a store with the five operations the API calls, and two implementations cover the two modes of running it. The memory store is the default and the one the tests use; the file store is chosen by setting DB_FILE, and it survives a restart. Both are safe for the concurrent handlers the HTTP server runs.

The interface

The same interface as the CLI capstone, with the note type from lesson 129. The API depends on it and on nothing else, so a fake in the tests and a database in a later version fit without a handler change.

package store import "errors" // ErrNotFound is returned for an id that does not exist. var ErrNotFound = errors.New("note not found") // ErrEmptyTitle is returned for a note with no title. var ErrEmptyTitle = errors.New("title is required") // Note is one note of the service. type Note struct { ID int `json:"id"` Title string `json:"title"` Done bool `json:"done"` } // Store is what the API needs from its storage. type Store interface { List() ([]Note, error) Get(id int) (Note, error) Create(title string) (Note, error) Update(id int, title string, done bool) (Note, error) Delete(id int) error }

The rules live in the store too: a title is trimmed and an empty one is an error, and a missing id is the sentinel. A handler cannot create a note without a title, whichever store it was given, which is the reason the rule is not repeated in the API.

The memory store

A map, a counter and a mutex. The lock covers the whole method, so a reader never sees a half-written map and two creates never take the same id. List copies the map into a slice and sorts by id, which is the order the API returns.

package store import ( "sort" "strings" "sync" ) // Memory is a Store in a map, safe for concurrent use. type Memory struct { mu sync.Mutex notes map[int]Note next int } // NewMemory returns an empty memory store. func NewMemory() *Memory { return &Memory{notes: map[int]Note{}, next: 1} } func (m *Memory) List() ([]Note, error) { m.mu.Lock() defer m.mu.Unlock() out := make([]Note, 0, len(m.notes)) for _, note := range m.notes { out = append(out, note) } sort.Slice(out, func(i, j int) bool { return out[i].ID < out[j].ID }) return out, nil } func (m *Memory) Get(id int) (Note, error) { m.mu.Lock() defer m.mu.Unlock() note, ok := m.notes[id] if !ok { return Note{}, ErrNotFound } return note, nil } func (m *Memory) Create(title string) (Note, error) { title = strings.TrimSpace(title) if title == "" { return Note{}, ErrEmptyTitle } m.mu.Lock() defer m.mu.Unlock() note := Note{ID: m.next, Title: title} m.next++ m.notes[note.ID] = note return note, nil } func (m *Memory) Update(id int, title string, done bool) (Note, error) { title = strings.TrimSpace(title) if title == "" { return Note{}, ErrEmptyTitle } m.mu.Lock() defer m.mu.Unlock() note, ok := m.notes[id] if !ok { return Note{}, ErrNotFound } note.Title = title note.Done = done m.notes[id] = note return note, nil } func (m *Memory) Delete(id int) error { m.mu.Lock() defer m.mu.Unlock() if _, ok := m.notes[id]; !ok { return ErrNotFound } delete(m.notes, id) return nil }

The mutex is a sync.Mutex and not a RWMutex, because the methods are short and a read still needs the map to be stable. A read-only mode with many readers and a rare write is the case for the read-write lock, and a service with one write a second does not have it.

The file store

The file store keeps the notes and the counter in one JSON document and reloads it on every operation. The mutex makes the read-modify-write of one call atomic with respect to other calls in the same process, which is the guarantee the API needs. The layout is the same as the CLI store, and the code is the same shape with the note type.

return fileData{}, err } var data fileData if err := json.Unmarshal(raw, &data); err != nil { return fileData{}, fmt.Errorf("read %s: %w", f.path, err) } if data.Next < 1 { data.Next = 1 } return data, nil } func (f *File) save(data fileData) error { sort.Slice(data.Notes, func(i, j int) bool { return data.Notes[i].ID < data.Notes[j].ID }) raw, err := json.MarshalIndent(data, "", " ") if err != nil { return err } raw = append(raw, '\n') tmp, err := os.CreateTemp(filepath.Dir(f.path), ".notes-*.json") if err != nil { return err } name := tmp.Name() defer os.Remove(name) if _, err := tmp.Write(raw); err != nil { tmp.Close() return err } if err := tmp.Close(); err != nil { return err } return os.Rename(name, f.path) } func (f *File) List() ([]Note, error) { f.mu.Lock() defer f.mu.Unlock() data, err := f.load() return data.Notes, err } func (f *File) Get(id int) (Note, error) { f.mu.Lock() defer f.mu.Unlock() data, err := f.load() if err != nil { return Note{}, err } for _, note := range data.Notes { if note.ID == id { return note, nil } } return Note{}, ErrNotFound } func (f *File) Create(title string) (Note, error) { title = strings.TrimSpace(title) if title == "" { return Note{}, ErrEmptyTitle } f.mu.Lock() defer f.mu.Unlock()

load treats a missing file as empty and returns any other error with the path, and save writes a temporary file and renames it. A corrupt document is reported, not overwritten: a service that loses its notes because it could not parse them is worse than one that refuses to start.

The atomic save

The same trick as the CLI: a temporary file in the same directory, a close, a rename. The mutex is what makes two concurrent requests take turns; the rename is what makes a crash leave either the old document or the new one. A database gives both guarantees with more machinery, which is the trade the next section names.

data, err := f.load() if err != nil { return Note{}, err } note := Note{ID: data.Next, Title: title} data.Next++ data.Notes = append(data.Notes, note) if err := f.save(data); err != nil { return Note{}, err } return note, nil } func (f *File) Update(id int, title string, done bool) (Note, error) { title = strings.TrimSpace(title) if title == "" { return Note{}, ErrEmptyTitle } f.mu.Lock() defer f.mu.Unlock() data, err := f.load() if err != nil { return Note{}, err } for i := range data.Notes { if data.Notes[i].ID == id { data.Notes[i].Title = title data.Notes[i].Done = done if err := f.save(data); err != nil { return Note{}, err } return data.Notes[i], nil } } return Note{}, ErrNotFound } func (f *File) Delete(id int) error { f.mu.Lock() defer f.mu.Unlock() data, err := f.load() if err != nil { return err } for i := range data.Notes { if data.Notes[i].ID == id { data.Notes = append(data.Notes[:i], data.Notes[i+1:]...) return f.save(data) } } return ErrNotFound }

The delete copies the slice around the removed element and saves; the append and the copy are the two places a slice aliases its backing array, and neither escapes the store. The file is small, the code is short, and the cost is one read and one write per request, which the benchmark of lesson 127 measured for the CLI.

The same operations

The demo runs create, create, update, get, delete and a missing get against the memory store and against a file store in a temporary directory. The two agree on every line.

package main import ( "fmt" "os" "path/filepath" "example.com/notesapi/internal/store" ) func demo(name string, st store.Store) { first, err := st.Create(" buy milk ") fmt.Printf("%s create: %+v err=%v\n", name, first, err) second, err := st.Create("write lesson") fmt.Printf("%s create: %+v err=%v\n", name, second, err) updated, err := st.Update(first.ID, "buy milk and bread", true) fmt.Printf("%s update: %+v err=%v\n", name, updated, err) got, err := st.Get(second.ID) fmt.Printf("%s get: %+v err=%v\n", name, got, err) if err := st.Delete(first.ID); err != nil { panic(err) } all, err := st.List() fmt.Printf("%s list: %d notes err=%v\n", name, len(all), err) _, err = st.Get(first.ID) fmt.Printf("%s missing: %v\n", name, err) } func main() { demo("memory", store.NewMemory()) dir, err := os.MkdirTemp("", "notes") if err != nil { panic(err) } defer os.RemoveAll(dir) path := filepath.Join(dir, "notes.json") demo("file ", store.NewFile(path)) info, err := os.Stat(path) fmt.Println("file bytes > 0:", info.Size() > 0, "err:", err) }

go run ./cmd/l130

memory create: {ID:1 Title:buy milk Done:false} err=<nil>
memory create: {ID:2 Title:write lesson Done:false} err=<nil>
memory update: {ID:1 Title:buy milk and bread Done:true} err=<nil>
memory get: {ID:2 Title:write lesson Done:false} err=<nil>
memory list: 1 notes err=<nil>
memory missing: note not found
file create: {ID:1 Title:buy milk Done:false} err=<nil>
file create: {ID:2 Title:write lesson Done:false} err=<nil>
file update: {ID:1 Title:buy milk and bread Done:true} err=<nil>
file get: {ID:2 Title:write lesson Done:false} err=<nil>
file list: 1 notes err=<nil>
file missing: note not found
file bytes > 0: true err: <nil>

The outputs are identical and the file is not empty, which is the contract of the interface. The demo is also the test the service does not need: the API tests use a fake, and one end-to-end run with the file store covers the difference.

What a database would change

A database would move the atomicity into a transaction, the ordering into an index and the durability into a write-ahead log, and it would let two processes share the data. The interface would not change: the five methods are the same, and the handlers would not know. What would change is the configuration, the migration story and the deployment, which is why a service starts with the memory store and moves when the data has to outlive the process.

The next lesson writes the handlers that use these methods and the statuses that go with them.

Exercises

  1. Run the file store demo twice in the same directory and compare the ids. What does the counter in the file do, and what would a sequence in a database do?
  2. Change the memory store to an RWMutex and make Get use RLock. Is the code still correct with List sorting under the lock?
  3. Make the file store cache the document in memory and write on a timer. What does a request see after a crash, and what does the interface still promise?

Questions

A new set of questions in a new order every time. Choose one answer for each, then check.

  1. Which package defines the Store interface?

    Answer

    internal/store, next to the model

    The API imports it.

  2. Why a mutex in the memory store?

    Answer

    Handlers run concurrently and the map is not safe

    Two creates must not share an id.

  3. What order does List return?

    Answer

    By id

    It sorts before returning.

  4. What does a missing file read as in the file store?

    Answer

    An empty document with the counter at one

    The first start needs no setup.

  5. What makes a save atomic?

    Answer

    A temporary file and a rename

    A rename on one filesystem is atomic.

  6. What does the mutex protect in the file store?

    Answer

    The read-modify-write of one call

    Two calls take turns.

  7. What happens to a corrupt file?

    Answer

    The error is returned with the path

    A user must not lose notes silently.

  8. Which store is the default?

    Answer

    The memory store

    DB_FILE selects the file store.

  9. What would a database change?

    Answer

    Transactions, indexes and durability, not the interface

    The interface is the seam.

  10. Why is the file store enough for this service?

    Answer

    The data is small and the traffic is one process

    The next step is a database when it outgrows that.

The code on this page was run with go 1.26.10 on Alpine Linux 3.24.3 (x86_64) in a Docker container.

Article author: Arthur Isaev