Capstone 1: the writer

The writer turns a value into bytes. There are two forms: the compact one a wire wants and the indented one a person reads. Both are the same recursion with a different amount of white space, and both append to a byte slice the caller owns, which is the shape that avoids a string per value.

AppendJSON and JSON

AppendJSON is the engine: it switches on the kind and appends the representation, recursing into arrays and objects. JSON is the one-line wrapper that starts with a nil slice. The keys of an object are already sorted by the constructor, so the encoding is stable across runs and machines.

func (v Value) JSON() []byte { return v.AppendJSON(nil, "") } // Indent returns the encoding with newlines and the given indent per level. func (v Value) Indent(indent string) []byte { return v.appendIndent(nil, indent, 0) } // AppendJSON appends the compact encoding to dst. func (v Value) AppendJSON(dst []byte, _ string) []byte { switch v.kind { case NullKind: return append(dst, "null"...) case BoolKind: return strconv.AppendBool(dst, v.b) case NumberKind: return appendNumber(dst, v.n) case StringKind: return appendString(dst, v.s) case ArrayKind: dst = append(dst, '[') for i, item := range v.arr { if i > 0 { dst = append(dst, ',') } dst = item.AppendJSON(dst, "") } return append(dst, ']') case ObjectKind: dst = append(dst, '{') for i, key := range v.keys { if i > 0 { dst = append(dst, ',') } dst = appendString(dst, key) dst = append(dst, ':') dst = v.obj[key].AppendJSON(dst, "") } return append(dst, '}') default: return dst } }

A string is quoted and escaped by appendString, a bool by strconv.AppendBool, a number by strconv.AppendFloat and a null is four bytes. The recursion is on the call stack, so a document that is ten thousand levels deep is a stack overflow here too, the same limit the parser has.

Indent

The indented form repeats the same walk with a newline and a padding string after every comma and bracket. It is a second function rather than a flag so the hot path has no branch per value, and the empty array and object get their own short form on one line.

func (v Value) appendIndent(dst []byte, indent string, depth int) []byte { pad := strings.Repeat(indent, depth+1) closePad := strings.Repeat(indent, depth) switch v.kind { case ArrayKind: if len(v.arr) == 0 { return append(dst, "[]"...) } dst = append(dst, '[') for i, item := range v.arr { if i > 0 { dst = append(dst, ',') } dst = append(dst, '\n') dst = append(dst, pad...) dst = item.appendIndent(dst, indent, depth+1) } dst = append(dst, '\n') dst = append(dst, closePad...) return append(dst, ']') case ObjectKind: if len(v.keys) == 0 { return append(dst, "{}"...) } dst = append(dst, '{') for i, key := range v.keys { if i > 0 { dst = append(dst, ',') } dst = append(dst, '\n') dst = append(dst, pad...) dst = appendString(dst, key) dst = append(dst, ':', ' ') dst = v.obj[key].appendIndent(dst, indent, depth+1) } dst = append(dst, '\n') dst = append(dst, closePad...) return append(dst, '}') default: return v.AppendJSON(dst, "") } }

The padding is built once per level with strings.Repeat, which allocates, so the indented form is for a file and not for a million messages. The compact form allocates one buffer and grows it; a caller that knows the size can pass a slice with capacity to AppendJSON directly.

String escaping

A JSON string must escape the quote, the backslash and every character below 0x20; everything else can be written as UTF-8. The function below handles the named escapes, falls back to \u00XX for the other control characters and copies a multibyte character as it is.

func appendNumber(dst []byte, f float64) []byte { return strconv.AppendFloat(dst, f, 'g', -1, 64) } func appendString(dst []byte, s string) []byte { dst = append(dst, '"') for _, r := range s { switch r { case '"': dst = append(dst, '\\', '"') case '\\': dst = append(dst, '\\', '\\') case '\n': dst = append(dst, '\\', 'n') case '\r': dst = append(dst, '\\', 'r') case '\t': dst = append(dst, '\\', 't') default: if r < 0x20 { dst = append(dst, []byte(fmt.Sprintf("\\u%04x", r))...) continue } dst = append(dst, string(r)...) } } return append(dst, '"') }

The fmt.Sprintf for a control character allocates, which is acceptable because a control character in a string is rare. A writer that must be allocation-free for every input uses a lookup table; the parser and the writer then agree on exactly which escapes are legal, which the round trip in the demo checks.

Number formatting

strconv.AppendFloat with a format of g and a precision of -1 writes the shortest decimal that parses back to the same float64. That is why 1.5 stays 1.5 and 1e21 becomes 1e+21, and it is the same choice encoding/json makes. The parser reads the text back with %g, so a number survives the round trip even if its digits are not the ones a human typed.

The round trip

The command parses a document, prints the compact form, prints the indented form, parses the compact bytes again and compares the two encodings. The comparison is the test the writer owes the parser: what it writes must be what the parser accepts.

package main import ( "fmt" "example.com/jsonx/jsonx" ) func main() { src := []byte(`{"b": [1, 2], "a": {"x": true, "y": "hi\n"}}`) v, err := jsonx.Parse(src) fmt.Println("error:", err) fmt.Println("compact:", string(v.JSON())) fmt.Println("indent:") fmt.Println(string(v.Indent(" "))) round, err := jsonx.Parse(v.JSON()) fmt.Println("round trip error:", err) fmt.Println("equal:", string(round.JSON()) == string(v.JSON())) }

go run ./cmd/l116

error: <nil>
compact: {"a":{"x":true,"y":"hi\n"},"b":[1,2]}
indent:
{
"a": {
"x": true,
"y": "hi\n"
},
"b": [
1,
2
]
}
round trip error: <nil>
equal: true

The compact line is one line, the indented one is one field per line, and the round trip is equal. A string with a newline came back as \n in both encodings and as the same value after the second parse, which is the property the tests in lesson 118 turn into a fuzz target.

Bytes, not a document

The writer returns bytes and never a formatted string, so a program can append it to a buffer, write it to a socket or embed it in a larger document. That is also why it is a value method and not a stateful encoder: there is nothing to flush, close or reuse, and the same value can be written any number of times.

The next lesson adds the positions to the errors, which is what turns a parser failure into a message a user can act on.

Exercises

  1. Write a string with a tab, a newline and a quote and read the bytes. Which escape does each become, and does the parser read them back?
  2. Write 0.1, 1e21 and 1e-7 and parse them again. Which digits survive, and why is the shortest form the right one?
  3. Append an empty object and an empty array in the indented form. Which short form does each use, and what would a line per bracket look like?

Questions

A new set of questions in a new order every time. Choose one answer for each, then check.

  1. What does AppendJSON append to?

    Answer

    A byte slice the caller passes

    JSON starts it with a nil slice.

  2. Why is the compact encoding stable?

    Answer

    The object keys are sorted by the constructor

    Map order is random.

  3. Which characters must a JSON string escape?

    Answer

    The quote, the backslash and control characters

    UTF-8 is legal as it is.

  4. Which float format does the writer use?

    Answer

    strconv.AppendFloat with g and precision -1

    It is the shortest round-tripping form.

  5. What does the indented form trade?

    Answer

    Allocations for readability

    It builds padding per level.

  6. Why is the writer stateless?

    Answer

    There is nothing to flush or reuse

    The same value can be written many times.

  7. What does the round trip check?

    Answer

    The parser accepts what the writer writes

    It is the contract between the two.

  8. What does a control character become?

    Answer

    A \u00XX escape

    JSON forbids raw control characters.

  9. What is the depth limit of the writer?

    Answer

    The stack, like the parser

    A deep document overflows the stack.

  10. Which values need no recursion?

    Answer

    The scalars: null, bool, number and string

    The containers recurse.

The code on this page was run with go 1.26.10 on Alpine Linux 3.24.3 (x86_64) in a Docker container.

Article author: Arthur Isaev