encoding/csv

CSV is the format of every export a spreadsheet produced, and its rules are small: fields are separated by a comma, a field with a comma, a quote or a newline is wrapped in quotes, and a quote inside such a field is doubled. encoding/csv implements those rules in both directions, so a program does not need a regular expression for a format that has one.

ReadAll and quoted fields

ReadAll returns every record as a slice of fields. The second line has a quoted comma, and the third has doubled quotes, and both come back as one field each. %q shows where the quotes are after decoding, which the raw text does not.

package main import ( "encoding/csv" "fmt" "strings" ) func main() { data := `name,note ada,"hello, world" grace,"says ""hi""" ` r := csv.NewReader(strings.NewReader(data)) records, err := r.ReadAll() for _, rec := range records { fmt.Printf("%d %q\n", len(rec), rec) } fmt.Println("error:", err) }

go build -o app main.go && ./app

2 ["name" "note"]
2 ["ada" "hello, world"]
2 ["grace" "says \"hi\""]
error: <nil>

Each record has two fields, and the quoting is gone from the values. A reader with a header is the usual case: the first record names the columns and the program maps each field to a struct field, which is what the capstone tasks CLI does with its import command.

The field count

By default the reader takes the field count of the first record and reports any other record as an error, with the line number. That catches a row with a missing comma at the point where it happens, and the record before it was already returned. Setting FieldsPerRecord to a positive number checks every record against it, and -1 turns the check off.

package main import ( "encoding/csv" "fmt" "io" "strings" ) func main() { r := csv.NewReader(strings.NewReader("a,b\nc\n")) for { rec, err := r.Read() if err == io.EOF { break } if err != nil { fmt.Println("error:", err) break } fmt.Println(rec) } }

go build -o app main.go && ./app

[a b]
error: record on line 2: wrong number of fields

The first record came back and the second stopped the loop. A program that skips bad rows reads with FieldsPerRecord = -1 and checks each record itself, and a program that reports them keeps the error and the line, which is what a user needs to fix the export.

A Writer

A csv.Writer quotes a field when it has to and writes the record with the line ending the format uses. Flush is required, as with every buffered writer, and Error reports a failure from the flush, because that is when the underlying writer usually fails. The output below quotes the field with a comma and leaves the plain ones alone.

package main import ( "bytes" "encoding/csv" "fmt" ) func main() { var buf bytes.Buffer w := csv.NewWriter(&buf) if err := w.Write([]string{"name", "note"}); err != nil { panic(err) } if err := w.Write([]string{"ada", "hello, world"}); err != nil { panic(err) } w.Flush() fmt.Println("error:", w.Error()) fmt.Print(buf.String()) }

go build -o app main.go && ./app

error: <nil>
name,note
ada,"hello, world"

The writer made the file that the reader in the first section parses. A field with a newline is quoted the same way, and a field with a quote gets the quote doubled, so a round trip through the writer and the reader returns the same strings.

Comma, comment and space

A reader can change its comma, skip comment lines and trim the space before a field, which covers the exports from systems that write ; as a separator or pad every column. The program below trims the space after the comma, so age is a field name and not age.

package main import ( "encoding/csv" "fmt" "strings" ) func main() { data := "name, age\nada, 36\n" r := csv.NewReader(strings.NewReader(data)) r.TrimLeadingSpace = true records, err := r.ReadAll() for _, rec := range records { fmt.Printf("%q\n", rec) } fmt.Println("error:", err) }

go build -o app main.go && ./app

["name" "age"]
["ada" "36"]
error: <nil>

Comma = ';' changes the separator and Comment = '#' skips a line that starts with that character; neither changes the quoting rules. Space inside a quoted field is kept, which is the difference between TrimLeadingSpace and strings.TrimSpace on the result, and the reason a field that must keep its leading space is quoted by the writer.

CSV is strings

Every field the reader returns is a string, and strconv.Atoi or time.Parse turns it into the type the program wants. A number with a comma as a thousands separator cannot be read as a number, and a date in a local format needs the layout from the previous lesson. A program that stores the raw record and the parsed value together can report the line that failed to convert.

Two limits are worth knowing. The reader handles a quoted newline but not Windows line endings inside a field mixed with Unix ones, and ReadAll holds the whole file, so a million-row export is read record by record with Read in a loop. The tasks capstone streams its export for that reason.

Exercises

  1. Write a field that contains a newline and a quote with the writer and read it back. How many quotes does the file hold, and what does the reader return?
  2. Set FieldsPerRecord = 1 on a file whose first row has two fields. Which line fails, and what does the error say?
  3. Read a semicolon-separated file with Comma = ';'. What does a comma inside a field do then, and why is it not quoted?

Questions

A new set of questions in a new order every time. Choose one answer for each, then check.

  1. How is a comma inside a CSV field written?

    Answer

    The field is quoted

    A quote inside a quoted field is doubled.

  2. What does ReadAll return?

    Answer

    A slice of records, each a slice of strings

    Every field is a string.

  3. What field count does the reader check by default?

    Answer

    The count of the first record

    FieldsPerRecord changes it.

  4. What does the wrong-number error name?

    Answer

    The line number

    That is what a user needs to fix the export.

  5. Why call Flush on a csv.Writer?

    Answer

    The writer buffers and Error reports the flush failure

    Every buffered writer needs it.

  6. What does TrimLeadingSpace do?

    Answer

    Removes the space before each field

    Space inside quotes is kept.

  7. How is a semicolon-separated file read?

    Answer

    Set Reader.Comma to a semicolon

    The quoting rules stay the same.

  8. What type is a CSV field?

    Answer

    A string

    The program converts what it needs.

  9. Why read a large export with Read and not ReadAll?

    Answer

    ReadAll holds the whole file in memory

    One record at a time keeps memory flat.

  10. What does Comma = ';' change?

    Answer

    The field separator only

    A comma inside a field then needs no quote.

The code on this page was run with go 1.26.10 on Alpine Linux 3.24.3 (x86_64) in a Docker container.

Article author: Arthur Isaev

Banner image