Capstone 1: errors with positions
A parser error without a position is a sentence a user cannot use. The error type of this project carries a line, a column and a message, and every lexer and parser failure builds one. The type is small enough that the whole error story fits in one file.
Position and SyntaxError
Position is a line, a column and a byte offset, all counted from the start of the document. The line and column start at one, which is what a text editor shows, and the offset is there for a caller that wants to slice the source. SyntaxError embeds a position and a message, and its Error method is the only formatting the package does.
package jsonx
import "fmt"
// Position is a line and column in the source, both starting at 1.
type Position struct {
Line int
Column int
Offset int
}
// String renders the position the way an error message names it.
func (p Position) String() string {
return fmt.Sprintf("line %d, column %d", p.Line, p.Column)
}
// SyntaxError is a parse or lex error with the place it happened.
type SyntaxError struct {
Pos Position
Msg string
}
func (e *SyntaxError) Error() string {
return fmt.Sprintf("%s: %s", e.Pos, e.Msg)
}
A program compares with errors.As to get the position back, and a test compares the whole string because a message is part of the contract. The type is a pointer, so a nil error stays nil and a failure is always a non-nil *SyntaxError.
The messages
Each parse function returns a message that names what it wanted and what it found. The value switch has one for an unexpected token and one for the end of input; the object loop has one for a key, one for the colon, one for a duplicate and one for an unterminated object.
func (p *parser) value() (Value, error) {
switch p.tok.Kind {
case String:
v := StringValue(p.tok.Text)
return v, p.next()
case Number:
var f float64
if _, err := fmt.Sscanf(p.tok.Text, "%g", &f); err != nil {
return NullValue(), &SyntaxError{Pos: p.tok.Pos, Msg: "invalid number " + p.tok.Text}
}
return NumberValue(f), p.next()
case True:
return BoolValue(true), p.next()
case False:
return BoolValue(false), p.next()
case Null:
return NullValue(), p.next()
case BraceOpen:
return p.object()
case BracketOpen:
return p.array()
case Invalid:
return NullValue(), &SyntaxError{Pos: p.lex.pos(), Msg: "unexpected end of input"}
default:
return NullValue(), &SyntaxError{Pos: p.tok.Pos, Msg: fmt.Sprintf("unexpected %s", p.tok.Kind)}
}
}
func (p *parser) object() (Value, error) {
open := p.tok.Pos
if err := p.next(); err != nil {
return NullValue(), err
}
fields := map[string]Value{}
first := true
for {
if p.tok.Kind == BraceClose {
if !first {
return NullValue(), &SyntaxError{Pos: p.tok.Pos, Msg: "trailing comma in object"}
}
return ObjectValue(fields), p.next()
}
first = false
if p.tok.Kind != String {
return NullValue(), &SyntaxError{Pos: p.tok.Pos, Msg: fmt.Sprintf("expected a key, got %s", p.tok.Kind)}
}
key := p.tok.Text
keyPos := p.tok.Pos
if err := p.next(); err != nil {
return NullValue(), err
}
if p.tok.Kind != Colon {
return NullValue(), &SyntaxError{Pos: p.tok.Pos, Msg: fmt.Sprintf("expected ':' after %q", key)}
}
if err := p.next(); err != nil {
return NullValue(), err
}
value, err := p.value()
if err != nil {
The %s of a Kind uses the String method from the tokenizer, so the message says unexpected } and not unexpected 11. The position is the token that failed, not the one before it, and the unterminated-container errors point at the opening bracket because a reader can find that character.
Five bad documents
The command parses five broken documents and prints each error. The first is a missing value, the second a trailing comma, the third an unfinished array, the fourth a truncated literal and the fifth an unterminated string.
package main
import (
"fmt"
"example.com/jsonx/jsonx"
)
func main() {
for _, src := range []string{
`{"a": }`,
`{"a": 1,}`,
`[1, 2`,
`nul`,
`"open`,
} {
_, err := jsonx.Parse([]byte(src))
fmt.Printf("%-12s -> %v\n", src, err)
}
}
go run ./cmd/l117
{"a": } -> line 1, column 7: unexpected }
{"a": 1,} -> line 1, column 9: trailing comma in object
[1, 2 -> line 1, column 1: unterminated array
nul -> line 1, column 1: invalid literal, expected "null"
"open -> line 1, column 1: unterminated string
The column of the trailing comma is the closing brace, because that is the token the loop saw after the comma. The other positions are the first character of the construct, which is the character a user selects in an editor. A message that names both the position and the expectation is the whole difference between an error and a mystery.
An error a user can act on
Three rules make the errors useful. One type with a position for every failure, so a caller has one thing to unwrap. A message that names what was expected and what was found. And a position at the token that caused the failure, not at the end of the file. A handler maps the error to a 400 with the position in the response, and the command line tool prints it on stderr and exits 2.
The next lesson turns these messages into tests: a table of bad documents with the substring each must contain, and a fuzz target that checks the parser never panics and always round-trips.
Exercises
- Parse a document with an error on the third line and read the position. Which line and column does it name, and how did the lexer count them?
- Wrap the error with fmt.Errorf("config: %w", err) and unwrap it with errors.As. What does the caller get back, and why is the wrapping useful?
- Add an error for a number that overflows a float64 and read the message. Which position does it name?
Questions
A new set of questions in a new order every time. Choose one answer for each, then check.
-
What does Position hold?
Answer
A line, a column and a byte offset
The offset is for a caller that slices the source.
-
What does SyntaxError.Error print?
Answer
The position and the message
line 1, column 7: unexpected }
-
How does a caller get the position back?
Answer
With errors.As to *SyntaxError
The type carries the position.
-
Which token does an unterminated array error name?
Answer
The opening bracket
A reader can find it.
-
What does a trailing comma error name?
Answer
The closing brace after the comma
It is the token the loop saw.
-
Why name both the expectation and what was found?
Answer
The reader knows what to change and where
It is the difference between an error and a mystery.
-
What status does a handler give a parse error?
Answer
400
The client sent a broken document.
-
What does the command line tool do with a parse error?
Answer
Prints it on stderr and exits 2
A non-zero status is how a shell learns.
-
Why is the error a pointer?
Answer
So a nil error stays nil
A non-nil failure is always a *SyntaxError.
-
Which line starts counting at one?
Answer
Line and column, like an editor
The offset starts at zero.
The code on this page was run with go 1.26.10 on Alpine Linux 3.24.3 (x86_64) in a Docker container.
Article author: Arthur Isaev