encoding/csv
CSV is the format of every export a spreadsheet produced, and its rules are small: fields are separated by a comma, a field with a comma, a quote or a newline is wrapped in quotes, and a quote inside such a field is doubled. encoding/csv implements those rules in both directions, so a program does not need a regular expression for a format that has one.
ReadAll and quoted fields
ReadAll returns every record as a slice of fields. The second line has a quoted comma, and the third has doubled quotes, and both come back as one field each. %q shows where the quotes are after decoding, which the raw text does not.
package main
import (
"encoding/csv"
"fmt"
"strings"
)
func main() {
data := `name,note
ada,"hello, world"
grace,"says ""hi"""
`
r := csv.NewReader(strings.NewReader(data))
records, err := r.ReadAll()
for _, rec := range records {
fmt.Printf("%d %q\n", len(rec), rec)
}
fmt.Println("error:", err)
}
go build -o app main.go && ./app
2 ["name" "note"]
2 ["ada" "hello, world"]
2 ["grace" "says \"hi\""]
error: <nil>
Each record has two fields, and the quoting is gone from the values. A reader with a header is the usual case: the first record names the columns and the program maps each field to a struct field, which is what the capstone tasks CLI does with its import command.
The field count
By default the reader takes the field count of the first record and reports any other record as an error, with the line number. That catches a row with a missing comma at the point where it happens, and the record before it was already returned. Setting FieldsPerRecord to a positive number checks every record against it, and -1 turns the check off.
package main
import (
"encoding/csv"
"fmt"
"io"
"strings"
)
func main() {
r := csv.NewReader(strings.NewReader("a,b\nc\n"))
for {
rec, err := r.Read()
if err == io.EOF {
break
}
if err != nil {
fmt.Println("error:", err)
break
}
fmt.Println(rec)
}
}
go build -o app main.go && ./app
[a b]
error: record on line 2: wrong number of fields
The first record came back and the second stopped the loop. A program that skips bad rows reads with FieldsPerRecord = -1 and checks each record itself, and a program that reports them keeps the error and the line, which is what a user needs to fix the export.
A Writer
A csv.Writer quotes a field when it has to and writes the record with the line ending the format uses. Flush is required, as with every buffered writer, and Error reports a failure from the flush, because that is when the underlying writer usually fails. The output below quotes the field with a comma and leaves the plain ones alone.
package main
import (
"bytes"
"encoding/csv"
"fmt"
)
func main() {
var buf bytes.Buffer
w := csv.NewWriter(&buf)
if err := w.Write([]string{"name", "note"}); err != nil {
panic(err)
}
if err := w.Write([]string{"ada", "hello, world"}); err != nil {
panic(err)
}
w.Flush()
fmt.Println("error:", w.Error())
fmt.Print(buf.String())
}
go build -o app main.go && ./app
error: <nil>
name,note
ada,"hello, world"
The writer made the file that the reader in the first section parses. A field with a newline is quoted the same way, and a field with a quote gets the quote doubled, so a round trip through the writer and the reader returns the same strings.
Comma, comment and space
A reader can change its comma, skip comment lines and trim the space before a field, which covers the exports from systems that write ; as a separator or pad every column. The program below trims the space after the comma, so age is a field name and not age.
package main
import (
"encoding/csv"
"fmt"
"strings"
)
func main() {
data := "name, age\nada, 36\n"
r := csv.NewReader(strings.NewReader(data))
r.TrimLeadingSpace = true
records, err := r.ReadAll()
for _, rec := range records {
fmt.Printf("%q\n", rec)
}
fmt.Println("error:", err)
}
go build -o app main.go && ./app
["name" "age"]
["ada" "36"]
error: <nil>
Comma = ';' changes the separator and Comment = '#' skips a line that starts with that character; neither changes the quoting rules. Space inside a quoted field is kept, which is the difference between TrimLeadingSpace and strings.TrimSpace on the result, and the reason a field that must keep its leading space is quoted by the writer.
CSV is strings
Every field the reader returns is a string, and strconv.Atoi or time.Parse turns it into the type the program wants. A number with a comma as a thousands separator cannot be read as a number, and a date in a local format needs the layout from the previous lesson. A program that stores the raw record and the parsed value together can report the line that failed to convert.
Two limits are worth knowing. The reader handles a quoted newline but not Windows line endings inside a field mixed with Unix ones, and ReadAll holds the whole file, so a million-row export is read record by record with Read in a loop. The tasks capstone streams its export for that reason.
Exercises
- Write a field that contains a newline and a quote with the writer and read it back. How many quotes does the file hold, and what does the reader return?
- Set FieldsPerRecord = 1 on a file whose first row has two fields. Which line fails, and what does the error say?
- Read a semicolon-separated file with Comma = ';'. What does a comma inside a field do then, and why is it not quoted?
Questions
A new set of questions in a new order every time. Choose one answer for each, then check.
-
How is a comma inside a CSV field written?
Answer
The field is quoted
A quote inside a quoted field is doubled.
-
What does ReadAll return?
Answer
A slice of records, each a slice of strings
Every field is a string.
-
What field count does the reader check by default?
Answer
The count of the first record
FieldsPerRecord changes it.
-
What does the wrong-number error name?
Answer
The line number
That is what a user needs to fix the export.
-
Why call Flush on a csv.Writer?
Answer
The writer buffers and Error reports the flush failure
Every buffered writer needs it.
-
What does TrimLeadingSpace do?
Answer
Removes the space before each field
Space inside quotes is kept.
-
How is a semicolon-separated file read?
Answer
Set Reader.Comma to a semicolon
The quoting rules stay the same.
-
What type is a CSV field?
Answer
A string
The program converts what it needs.
-
Why read a large export with Read and not ReadAll?
Answer
ReadAll holds the whole file in memory
One record at a time keeps memory flat.
-
What does Comma = ';' change?
Answer
The field separator only
A comma inside a field then needs no quote.
The code on this page was run with go 1.26.10 on Alpine Linux 3.24.3 (x86_64) in a Docker container.
Article author: Arthur Isaev