CSV in depth

Installing the gem

csv is a bundled gem: it is in a full Ruby install and a minimal one needs the install. The gem is pure Ruby, so the command works anywhere:

gem install csv

1 gem installed

generate and parse

CSV.generate yields a writer, and each << adds a row. CSV.parse goes the other way and returns arrays of strings:

ruby -e 'require "csv"; puts CSV.generate { |c| c << ["a", 1] }'

a,1

ruby -e 'require "csv"; p CSV.parse("a,1\n")[0][1]'

"1"

The parse result is strings even for a number, because CSV has no types. Converting is the caller's job, which is what the converters in the next section do.

Headers and converters

With headers: true the first row becomes the keys, and each row answers to a column name. to_h turns it into a hash:

ruby -e 'require "csv"; rows = CSV.parse("name,age\nAda,36\n", headers: true); r = rows.first; p r["name"]; p r.to_h'

"Ada" {"name" => "Ada", "age" => "36"}

converters: :numeric converts a field that looks like a number, which is the usual first step before arithmetic. The conversion is a guess: a zip code with a leading zero becomes a number and loses the zero, so a column that must stay text keeps the string.

ruby -e 'require "csv"; p CSV.parse("a,1\n", converters: :numeric)'

[["a", 1]]

Files and streaming

CSV.open writes a file, and CSV.foreach reads one row at a time without loading the file:

require "csv" CSV.open("out.csv", "w") do |csv| csv << ["name", "age"] csv << ["Ada", 36] end print File.read("out.csv") CSV.foreach("out.csv", headers: true) { |row| puts row["name"] }

ruby csv_file.rb

name,age Ada,36 Ada

foreach keeps memory flat for a large export, which is the difference between a job that runs and one the host kills. The block form of open closes the file, the same guarantee as the file lesson.

Quoting

A field with a comma or a quote is written inside quotes, and a quote inside it is doubled. That rule is why a line built with join(",") is not CSV:

ruby -e 'require "csv"; print CSV.generate { |c| c << ["a,b", "c\"d"] }'

"a,b","c""d"

The writer knows the rule, the reader knows it, and a spreadsheet knows it. A hand-built line fails on the first value with a comma in it, which is usually a person's name.

What to watch for

CSV is not an encoding-aware format. A file written on Windows may start with a byte-order mark, which becomes part of the first header; read it with encoding: "bom|utf-8" when the source is a spreadsheet.

CSV.parse loads the whole input and CSV.foreach does not. For anything that grows with the data, use the streaming form and a block.

Next: XML and HTML

Article author: Arthur Isaev