How encoding/json uses reflection
encoding/json is a reflection walk over a type with a cache in front of it. The walk reads the exported fields, the tags and the kinds, and the cache keeps the plan per type so the second message of a million costs a map lookup and a fill. A custom method can replace the walk for one type, which is how a value with a unit or a time in a special layout is encoded.
The field walk
Marshal reads every exported field, takes the name from the json tag and skips a field whose tag is a dash. Unmarshal matches the keys to the same names and ignores an unknown key or a field it cannot see. The struct below has a dash field, an unexported field and a key that does not exist, and only the two real fields survive the round trip.
package main
import (
"encoding/json"
"fmt"
)
type User struct {
Name string `json:"name"`
Age int `json:"age"`
Secret string `json:"-"`
private string
}
func main() {
u := User{Name: "ada", Age: 36, Secret: "hidden", private: "unexported"}
data, err := json.Marshal(u)
fmt.Println("marshal:", string(data))
fmt.Println("error:", err)
var back User
err = json.Unmarshal([]byte(`{"name":"grace","age":45,"secret":"ignored"}`), &back)
fmt.Printf("unmarshal: %+v\n", back)
fmt.Println("error:", err)
}
go build -o app main.go && ./app
marshal: {"name":"ada","age":36}
error: <nil>
unmarshal: {Name:grace Age:45 Secret: private:}
error: <nil>
The marshal output has two keys, because Secret is a dash and private is unexported. The unmarshal filled the name and the age and left the other two empty; the unknown secret key was ignored. That is the whole behaviour of the package on a struct, and the rest is the tags and the kinds.
A custom marshaller
A type with a MarshalJSON method replaces the walk for itself. The method returns the bytes it wants, and the package trusts them: it does not check that they are valid JSON. The temperature below writes a string with a unit, which is a value a client can read without knowing the struct.
package main
import (
"encoding/json"
"fmt"
)
type Temperature float64
func (t Temperature) MarshalJSON() ([]byte, error) {
return []byte(fmt.Sprintf("%q", fmt.Sprintf("%.1f C", float64(t)))), nil
}
func main() {
data, err := json.Marshal(struct {
Temp Temperature `json:"temp"`
}{Temp: 21.5})
fmt.Println(string(data))
fmt.Println("error:", err)
}
go build -o app main.go && ./app
{"temp":"21.5 C"}
error: <nil>
The struct's walk reached the field and called the method, and the result is a JSON string. The method must return an error for a value it cannot represent, and a panic inside it becomes an error from Marshal, not a crash: the package recovers and reports json: error calling MarshalJSON.
A custom unmarshaller
UnmarshalJSON gets the raw bytes of the value and writes through a pointer. The method below reads a string, strips the unit and parses the number, so the same wire value round-trips through both custom methods. A decoder that finds the method does not walk the type at all.
package main
import (
"encoding/json"
"fmt"
"strconv"
"strings"
)
type Temperature float64
func (t *Temperature) UnmarshalJSON(data []byte) error {
var s string
if err := json.Unmarshal(data, &s); err != nil {
return err
}
s = strings.TrimSuffix(s, " C")
value, err := strconv.ParseFloat(s, 64)
if err != nil {
return err
}
*t = Temperature(value)
return nil
}
func main() {
var v struct {
Temp Temperature `json:"temp"`
}
err := json.Unmarshal([]byte(`{"temp":"21.5 C"}`), &v)
fmt.Println("temp:", float64(v.Temp))
fmt.Println("error:", err)
}
go build -o app main.go && ./app
temp: 21.5
error: <nil>
The value arrived as a string and the field holds the number at the end. A method that returns an error makes Unmarshal return it with the field path, which is how a client learns which value was wrong. The pair is the documented way to give a type a wire format of its own without a separate DTO.
The walk by hand
The function below is the small core of the package: reflect over the type, read the tag, switch on the kind, write the key and the value. Its output matches json.Marshal for this struct, which is the point: the package is not magic, it is this loop with a cache and a lot of edge cases.
package main
import (
"encoding/json"
"fmt"
"reflect"
"strings"
)
type Note struct {
ID int `json:"id"`
Title string `json:"title"`
Note string `json:"-"`
}
func encode(v any) string {
rv := reflect.ValueOf(v)
rt := rv.Type()
var parts []string
for i := 0; i < rt.NumField(); i++ {
field := rt.Field(i)
tag := field.Tag.Get("json")
if tag == "-" {
continue
}
name := field.Name
if tag != "" {
name = strings.Split(tag, ",")[0]
}
value := rv.Field(i)
if value.Kind() == reflect.Int {
parts = append(parts, fmt.Sprintf("%q:%d", name, value.Int()))
} else {
parts = append(parts, fmt.Sprintf("%q:%q", name, value.String()))
}
}
return "{" + strings.Join(parts, ",") + "}"
}
func main() {
n := Note{ID: 1, Title: "buy milk", Note: "ignored"}
expected, err := json.Marshal(n)
if err != nil {
panic(err)
}
fmt.Println("encoding/json:", string(expected))
fmt.Println("reflection: ", encode(n))
}
go build -o app main.go && ./app
encoding/json: {"id":1,"title":"buy milk"}
reflection: {"id":1,"title":"buy milk"}
The two lines are the same, and the hand-written version already has a bug the package does not: a string with a quote is not escaped. That is the lesson of re-implementing it: the loop is short and the escaping table, the float formats and the recursion are long. The package caches the plan per type, which is why the real one is fast enough to use by default.
A field deep in a slice
The error path names the Go type, the field and the type it expected, and for a nested value it names the outer field too. The message below says Item.items.age: the field age of an Item inside the items array. A client that logs the message can find the value without a debugger.
package main
import (
"encoding/json"
"fmt"
)
type Item struct {
Age int `json:"age"`
}
type Response struct {
Items []Item `json:"items"`
}
func main() {
var r Response
err := json.Unmarshal([]byte(`{"items":[{"age":"old"}]}`), &r)
fmt.Println("error:", err)
}
go build -o app main.go && ./app
error: json: cannot unmarshal string into Go struct field Item.items.age of type int
The message is produced by the field walk after the decoder has descended into the slice, and it is the same text whether the value is at the top or three levels down. A handler maps it to a 400 and a test asserts on the field name, which is stable, rather than on the whole sentence.
What the package caches
The first marshal of a type builds a plan: the fields to include, their names, their order and the encoders for their kinds. The plan is cached per type, and every later value of that type reuses it. The cache is why a single struct type encodes at a steady speed and why a program that builds types at run time pays for each one. It is also why the tag of a field is a compatibility promise: the plan is made once per type and the keys are fixed by it.
The next lesson leaves the encoder for the memory model, the rules that say when one goroutine sees a write made by another. The reflection here is all on one goroutine; the next rules are about the ones that are not.
Exercises
- Add a MarshalJSON to a struct instead of a float type and read the output of a field that holds it. Which method wins, the struct's or the field walk?
- Remove the json tag from the Note struct and run the hand-written encoder. Which keys change, and does encoding/json agree?
- Give the custom unmarshaller a value without the unit and read the error. What does the message name, and where would a handler turn it into a 400?
Questions
A new set of questions in a new order every time. Choose one answer for each, then check.
-
Which fields does encoding/json marshal?
Answer
The exported fields, with the json tag names
A dash tag skips one and unexported fields never take part.
-
What does a dash in a json tag do on Unmarshal?
Answer
The field is ignored
The key never matches.
-
What does MarshalJSON replace?
Answer
The field walk for that type
The method returns the bytes for the value.
-
What does a panic inside MarshalJSON become?
Answer
An error from Marshal
The package recovers and reports it.
-
What does UnmarshalJSON receive?
Answer
The raw bytes of the value
It parses what it needs and writes through the pointer.
-
What does the hand-written encoder miss?
Answer
Escaping a string with a quote
That is why the real package is longer.
-
What does the nested error name?
Answer
The Go type, the outer field and the inner field
It is produced after descending into the slice.
-
What does encoding/json cache per type?
Answer
The field plan with names and encoders
The first marshal builds it.
-
Why is a tag a compatibility promise?
Answer
The cached plan fixes the keys for the type
A rename changes the wire format.
-
When is a custom marshaller the right tool?
Answer
When the wire format differs from the struct layout
A unit or a special date is the classic case.
The code on this page was run with go 1.26.10 on Alpine Linux 3.24.3 (x86_64) in a Docker container.
Article author: Arthur Isaev