Splitting and joining text

words and unwords

A String is a list of Char, so the list functions of the last lessons work on text directly. Two of them are the pair every text program starts with: words splits on runs of white space and drops the empty pieces, and unwords joins with one space between the pieces.

words "the quick brown fox" -- ["the","quick","brown","fox"] unwords ["a", "b", "c"] -- "a b c" words "" -- [] unwords [] -- ""

The two functions are exact inverses on lists of non-empty words without spaces: unwords (words s) returns s with its runs of white space collapsed to one space, which is not the same string when the original had more than one. That is the usual behaviour of a normalising join, and it is worth knowing before a diff surprises you.

lines and unlines

lines splits on newline characters and drops the newline itself; unlines appends a newline to every piece. The asymmetry is real: unlines of three pieces ends with a newline, so unlines followed by lines is the identity, and the other order is not.

lines "one\ntwo\nthree" -- ["one","two","three"] lines "one\ntwo\n" -- ["one","two"] unlines ["x", "y"] -- "x\ny\n" putStrLn (unlines ["x", "y"]) -- prints x, y and an empty line

The last line is the trap: putStrLn adds its own newline, so a string that already ends with one produces a blank line. Use putStr with unlines, or mapM_ putStrLn on the list, and the output is exactly as many lines as the list has elements.

break and span

break p splits a list at the first element that satisfies p and returns a pair: the prefix before it and the rest starting with it. span p is the same with the predicate inverted: the longest prefix that satisfies p, then the rest.

break (== ' ') "hi there" -- ("hi"," there") span (< 3) [1, 2, 3, 1] -- ([1,2],[3,1]) break (> 3) [1, 2, 3, 4] -- ([1,2,3],[4]) span (< 3) [1, 2] -- ([1,2],[])

The pair is the whole answer: the prefix, and the remainder that still starts with the element that stopped the search. That is why break is the right tool for a splitter: the delimiter is at the head of the second component, and dropping it is one pattern match. The next lesson on where and recursion already used that shape in the splitOn of the composition lesson.

A word count

Three functions built from the ones above: count the words, find the longest one, and reverse every word in the text.

module Main (main) where wordCount :: String -> Int wordCount = length . words longestWord :: String -> String longestWord text = foldr pick "" (words text) where pick word best = if length word > length best then word else best main :: IO () main = do let text = "the quick brown fox jumps over the lazy dog" print (wordCount text) putStrLn (longestWord text) putStrLn (unwords (map reverse (words text)))

ghc -Wall -O1 -o wordcount wordcount.hs
./wordcount

9
jumps
eht kciuq nworb xof spmuj revo eht yzal god

The count is the length of the list of words, which is the composition of the last lesson in one line. The longest word is a fold with a comparison, and the tie goes to the first word because > is strict. The third line maps reverse over the words and joins them again, which is the kind of pipeline that would be a loop with a temporary buffer in another language.

A shorter program shows the four functions of the first sections in one file, including the blank line that putStrLn and unlines produce together:

module Main (main) where main :: IO () main = do print (words "the quick brown fox" :: [String]) putStrLn (unwords ["a", "b", "c"]) print (lines "one\ntwo\nthree" :: [String]) putStrLn (unlines ["x", "y"]) print (break (== ' ') "hi there") print (span (< 3) [1, 2, 3, 1] :: ([Int], [Int]))

ghc -Wall -O1 -o text text.hs
./text

["the","quick","brown","fox"]
a b c
["one","two","three"]
x
y

("hi"," there")
([1,2],[3,1])

The empty line after y is the one from unlines plus putStrLn: the string is "x\ny\n" and the print adds a third newline. The lesson on IO shows putStr, the function that does not add one.

Exercises

  1. Count the words of a three-line string with length . words and check the answer by hand.
  2. Print lines of a string that ends with a newline and compare it with one that does not.
  3. Use break to split a line at the first comma and print both parts.
  4. Write splitOn from the composition lesson again, now with break and a recursive call.
  5. Find the longest word of a sentence with maximum on the lengths and then with the fold of the program; say which is total.
  6. Replace putStrLn (unlines xs) with putStr (unlines xs) and explain the missing blank line.

Questions

A new set of questions in a new order every time. Choose one answer for each, then check.

  1. What does words "the quick brown" return?

    Answer

    ["the","quick","brown"]

    words splits on runs of white space and drops the empty pieces.

  2. What does unwords ["a", "b"] return?

    Answer

    "a b"

    unwords joins the pieces with one space between them.

  3. What does lines "one\ntwo\n" return?

    Answer

    ["one","two"]

    lines drops the newline characters, so the trailing newline produces no empty piece.

  4. What does unlines ["x", "y"] return?

    Answer

    "x\ny\n"

    unlines appends a newline to every piece, including the last.

  5. What does break (== &#39; &#39;) "hi there" return?

    Answer

    ("hi"," there")

    break stops at the space and keeps it at the head of the second component.

  6. What does span (< 3) [1, 2, 3, 1] return?

    Answer

    ([1,2],[3,1])

    span keeps the longest prefix that satisfies the predicate and returns the rest.

  7. Why does putStrLn (unlines ["x", "y"]) print a blank line at the end?

    Answer

    unlines already ends with a newline and putStrLn adds another

    The string is "x\ny\n" and putStrLn appends its own newline, which makes an empty line.

  8. What does this program print? module Main (main) where main :: IO () main = print (words "a b c" :: [String])

    Answer

    ["a","b","c"]

    words returns a list of three strings, and print shows a list with quotes.

  9. What does wordCount = length . words compute?

    Answer

    The number of words in the string

    words turns the text into a list of words and length counts them.

  10. What does take 2 (words "one two three") return?

    Answer

    ["one","two"]

    words builds the whole list lazily and take asks for the first two elements.

The code on this page was built with GHC 9.10.3 and run on Alpine Linux 3.24.3 (x86_64) in a Docker container.

Article author: Arthur Isaev