ByteString

Bytes and not characters

A ByteString is a sequence of bytes with a length. It knows nothing about characters, encodings or lines: BS.length counts bytes, and the byte 10 is a newline only if the program decides so. That is exactly what a network protocol, a compressed file or an image needs.

pack :: [Word8] -> ByteString unpack :: ByteString -> [Word8] length :: ByteString -> Int hPut :: Handle -> ByteString -> IO ()

For text that is known to be ASCII or Latin-1, the module Data.ByteString.Char8 adds a view where a Char is one byte: pack from a String, unpack back, lines and words for ASCII text. It is a convenience, not an encoding: a Cyrillic letter is not one byte and the view silently keeps only the low byte.

Strict and lazy ByteString

The package has two types with the same name in two modules:

The functions are the same and the modules are separate, so an import list says which one the code uses. The program below uses the strict one, because the examples are small.

Lines of bytes

module Main (main) where import qualified Data.ByteString as BS import qualified Data.ByteString.Char8 as BC main :: IO () main = do let bytes = BS.pack [72, 105, 10] print bytes print (BS.length bytes) print (BC.unpack (BC.pack "abc")) print (BS.unpack (BS.take 2 bytes)) print (BS.replicate 3 65)

ghc -Wall -O1 -o bytes bytes.hs
./bytes

"Hi\n"
3
"abc"
[72,105]
"AAA"

The Show instance prints a ByteString as a string with the non-printable bytes escaped, which is why the newline appears as \n and not as a line break. The length is 3 bytes for the two letters and the newline, and 65 is the byte of the letter A, which replicate repeated three times.

module Main (main) where import qualified Data.ByteString as BS import qualified Data.ByteString.Char8 as BC main :: IO () main = do let text = BC.pack "line one\nline two\n" rows = BC.lines text print rows print (map BC.length rows) print (BS.length text) print (BS.elem 10 text)

ghc -Wall -O1 -o lines lines.hs
./lines

["line one","line two"]
[8,8]
18
True

BC.lines splits on the byte 10 and drops the newline from each piece, so the two lines are 8 bytes each while the whole buffer is 18: two lines of 8, the newline after each, and the text has two newlines and no trailing empty line. BS.elem 10 answers the byte question directly.

Reading and writing bytes

A byte-for-byte copy is three lines, and the type keeps text out of it:

module Main (main) where import qualified Data.ByteString as BS main :: IO () main = do BS.writeFile "/tmp/bytestring-demo.bin" (BS.pack [104, 101, 108, 108, 111, 10]) input <- BS.readFile "/tmp/bytestring-demo.bin" BS.writeFile "/tmp/bytestring-copy.bin" input copy <- BS.readFile "/tmp/bytestring-copy.bin" print (BS.length input, BS.length copy, input == copy)

ghc -Wall -O1 -o copy copy.hs
./copy

(6,6,True)

The program reads the file into one strict ByteString and writes it back, and the equality says the copy is byte for byte the same. A lazy ByteString would stream the copy with a bounded amount of memory, which is the version to use for a file that does not fit: the interface is the same and the import is the other module. Strict and lazy do not mix without toStrict and fromStrict, and that conversion is the sign that the choice was made in the wrong place.

Exercises

  1. Build a ByteString of the bytes of your name in ASCII and print its length.
  2. Split a two-line buffer with BC.lines and join it back with BC.unlines, then compare with the original.
  3. Explain why BS.length (BC.pack "привет") is not the number of characters.
  4. Copy a file with the lazy module and say what changes in memory use.
  5. Check whether a buffer contains the byte 0 and say which function you used.
  6. Decide whether a JSON body from the network should be a ByteString or a Text and say why.

Questions

A new set of questions in a new order every time. Choose one answer for each, then check.

  1. What does BS.length count?

    Answer

    Bytes

    A ByteString is a byte sequence and knows nothing about characters.

  2. What is Data.ByteString.Char8 for?

    Answer

    A view where one Char is one byte, for ASCII and Latin-1 text

    It is a convenience view over bytes for text that fits in one byte per character.

  3. What is the difference between Data.ByteString and Data.ByteString.Lazy?

    Answer

    The strict one is one chunk and the lazy one is a list of chunks

    The lazy type streams a large input as a list of strict chunks.

  4. What does this program print? module Main (main) where import qualified Data.ByteString as BS main :: IO () main = print (BS.length (BS.pack [65, 66, 67]))

    Answer

    3

    The buffer has three bytes; the byte values are not summed.

  5. How does Show print a ByteString?

    Answer

    As a string with non-printable bytes escaped

    The instance prints the printable characters and escapes the rest, which is why the newline showed as \n.

  6. What does this program print? module Main (main) where import qualified Data.ByteString.Char8 as BC main :: IO () main = print (BC.lines (BC.pack "a\nb\n"))

    Answer

    ["a","b"]

    lines splits on the newline byte and drops the separators, so a trailing newline adds no empty piece.

  7. Which byte is the letter A in ASCII?

    Answer

    65

    The program used BS.replicate 3 65 and printed AAA.

  8. What is a ByteString the right type for?

    Answer

    Bytes of a file, a socket or a protocol, where text is not the question

    It is the byte-level type; text that needs characters is Text.

  9. What does BS.readFile return?

    Answer

    IO ByteString

    Reading a file is an effect, so the result is an IO action that gives the strict ByteString.

  10. Why is BC.pack "привет" a poor idea?

    Answer

    The Cyrillic letters do not fit in one byte, so the view loses information

    Char8 keeps the low byte of each Char, which is only correct for one-byte encodings.

The code on this page was built with GHC 9.10.3 and run on Alpine Linux 3.24.3 (x86_64) in a Docker container.

Article author: Arthur Isaev

Banner image