Data.Text
Text is not String
String is a list of Char: simple, lazy, and expensive, because every character is a cons cell. Data.Text is a packed array of UTF-16 code units with a length and an offset, so a character count is a subtraction of two integers instead of a traversal:
pack :: String -> Text
unpack :: Text -> String
length :: Text -> Int
splitOn :: Text -> Text -> [Text]
The catch is that Text is strict in its content and has its own module of functions: map, filter and the list functions do not work on it untouched. The two conversions are pack and unpack, and a program that keeps Text at its edges and converts once is the usual shape.
pack, unpack and the functions
module Main (main) where
import qualified Data.Text as T
main :: IO ()
main = do
let text = T.pack "Hello, Haskell world"
print (T.length text)
print (T.toUpper text)
print (T.splitOn (T.pack ",") text)
print (T.words text)
print (T.replace (T.pack "Haskell") (T.pack "GHC") text)
print (T.isPrefixOf (T.pack "Hello") text)
ghc -Wall -O1 -o text text.hs
./text
20
"HELLO, HASKELL WORLD"
["Hello"," Haskell world"]
["Hello,","Haskell","world"]
"Hello, GHC world"
True
Every function takes and returns Text, so each literal is wrapped once with pack and the printed form is the one of Text, with the quotes of the Show instance. The length is 20 characters, not 20 bytes and not 20 cons cells, and the split kept the space after the comma, which is why the second piece starts with one.
Trimming and splitting fields
A line of input usually needs trimming before it is compared, and empty fields are usually dropped. Both are one function each:
module Main (main) where
import qualified Data.Text as T
type Field = T.Text
trim :: Field -> Field
trim = T.strip
clean :: [Field] -> [Field]
clean = filter (not . T.null) . map trim
main :: IO ()
main = do
let raw = map T.pack [" ada ", "", " grace "]
print (clean raw)
print (T.intercalate (T.pack ";") (clean raw))
print (T.length (T.concat (clean raw)))
print (T.unpack (T.take 2 (T.pack "abcdef")))
ghc -Wall -O1 -o fields fields.hs
./fields
["ada","grace"]
"ada;grace"
8
"ab"
The composition reads right to left: every field is trimmed, then the empty ones are dropped, and only then is the list printed. The length of the two joined names is 8 characters, so the concatenation happened after the trimming and the spaces are gone. T.take counts characters and not code points beyond the basic plane, which is the usual caveat of UTF-16 and worth knowing before a substring of an emoji.
String, Text or ByteString
- String for signatures, small literals and teaching code; the list functions apply and the laziness is sometimes useful.
- Text for text a program reads, stores or prints in quantity: UTF-16, strict, with a rich module of its own.
- ByteString for bytes that are not text, or for text in a known encoding where a byte count is what matters.
A file read with readFile gives a String and a file read with Data.Text.IO.readFile gives a Text; both are in the boot libraries. The next lesson takes the byte-level view with ByteString, and the capstone reads a real file with the type that fits.
Exercises
- Count the words of a Text with T.words and compare with the String version.
- Split a line on ; and trim every field.
- Convert a Text to a String, upper-case it and convert back.
- Explain why T.length is not the number of bytes of the text on disk.
- Replace every occurrence of a word with a marker and print the result.
- Decide which type a program that reads a log file and counts its lines should use and why.
Questions
A new set of questions in a new order every time. Choose one answer for each, then check.
-
What is
Stringin Haskell?Answer
A list of
CharString is
[Char], so every character is a cons cell, which is why Text exists. -
How many characters are in
T.pack "abc"?Answer
3
T.length counts characters, and abc has three.
-
What does
T.splitOn (T.pack ",")return?Answer
A list of Text pieces around every comma
splitOn cuts the text at each occurrence of the delimiter and keeps the pieces.
-
What does this program print?
module Main (main) where import qualified Data.Text as T main :: IO () main = print (T.length (T.pack "hello"))Answer
5
length gives an Int, so print shows 5 without quotes.
-
Why does
map toUppernot work on aTextdirectly?Answer
Text is not a list, so the list function needs T.map
Text has its own map, and the list version needs unpack and pack around it.
-
What does this program print?
module Main (main) where import qualified Data.Text as T main :: IO () main = print (T.words (T.pack "a b c"))Answer
["a","b","c"]
T.words splits on runs of whitespace and drops the empty pieces.
-
What does
T.stripremove?Answer
Whitespace from both ends
strip trims the ends, while T.filter or T.replace would touch the inside.
-
Which type is best for a large text file read once?
Answer
TextText is packed and strict and has the functions, while a String pays a cons cell per character.
-
What does
T.unpackreturn?Answer
A
Stringunpack converts Text back to the list-of-Char representation.
-
What does
T.intercalate sep piecesdo?Answer
Joins the pieces with sep between them
It is the join of a list of Text values, as the field program showed with a semicolon.
The code on this page was built with GHC 9.10.3 and run on Alpine Linux 3.24.3 (x86_64) in a Docker container.
Article author: Arthur Isaev