char, bytes and bit flags

In Rust two small types carry most of the text and flag work: char, a Unicode scalar value of four bytes, and u8, a byte. They are different on purpose. A string is bytes, a character is a code point, and the conversion between them is the source of most of the surprises in text handling.

The same byte type is what bit flags are built from. A permission set is an integer with one bit per permission, and the operators from the lesson on operators set, clear and test them. The last section reads the error that indexing a string by number produces, which is the compiler refusing to pretend that a byte is a character.

char is a scalar value

A char is written with single quotes and holds one Unicode scalar value, not one byte. The letter e with an acute accent is two bytes in UTF-8 and one char, so len_utf8 returns two while the value is one character.

The program prints the character, its byte length, whether it is alphabetic and its upper-case form. The upper-case conversion returns an iterator because one character can become several.

fn main() { let letter = 'é'; println!("{} {}", letter, letter.len_utf8()); println!("{} {}", letter.is_alphabetic(), letter.to_uppercase()); }

rustc --edition 2024 char_props.rs
./char_props

é 2
true É

A char is four bytes in memory because it must hold any scalar value, which is why an array of characters uses four times the space of the same text in UTF-8. The lesson on UTF-8 compares the two counts.

The bytes of a string

bytes() walks the UTF-8 encoding of a string one byte at a time, and len() returns the number of bytes, not characters. For ASCII the two numbers agree, which is why the difference is easy to miss until a name or an accent arrives.

The program prints six bytes for a five-letter word, because the accented letter takes two. The byte line shows the two values that make it up, and the last one is the plain ASCII letter.

fn main() { let text = "héllo"; println!("bytes: {}", text.len()); for byte in text.bytes() { print!("{byte} "); } println!(); }

rustc --edition 2024 bytes.rs
./bytes

bytes: 6
104 195 169 108 108 111

The numbers are printed with print! so they share a line, and the empty println!() closes it. The lesson on string slices counts characters instead.

Bit flags

A permission set is a u8 with one bit per permission. Constants give the bits names, | combines them, & tests one and !mask clears it. This is the pattern behind file modes and option flags in every systems program.

The program starts with read and write, checks two permissions, then adds execute and removes write. The binary print at the end shows the three bits that remain.

const READ: u8 = 0b0001; const WRITE: u8 = 0b0010; const EXEC: u8 = 0b0100; fn main() { let mut permissions = READ | WRITE; println!("read: {}", permissions & READ != 0); println!("exec: {}", permissions & EXEC != 0); permissions |= EXEC; permissions &= !WRITE; println!("{:04b}", permissions); }

rustc --edition 2024 flags.rs
./flags

read: true
exec: false
0101

The constants are const items, so they cost nothing at run time. The lesson on const items explains why they can be used in a pattern as well.

Set, clear and test a bit

Bit three is 1u8 << 3, which is eight. The shift builds the mask, the bitwise and tests it, and the inverted mask clears it from a byte of ones. All three operations are one instruction each.

The program prints the value and its binary form, then the byte with bit three cleared. The typed suffix on the literal is what makes the shift a u8 shift instead of an i32 one.

fn main() { let bit = 1u8 << 3; println!("{bit} {:08b}", bit); let cleared = 0b1111_1111u8 & !bit; println!("{:08b}", cleared); }

rustc --edition 2024 shifts.rs
./shifts

8 00001000
11110111

A shift by the width of the type or more is an error for a constant and a panic in a debug build for a value, which the lesson on operators shows.

Why text cannot be indexed

text[0] looks harmless and is E0277 in Rust, because a byte index is not a character index and the compiler refuses to guess which one was meant. The help points at chars().nth() and bytes().nth() as the two explicit choices.

The program asks for the first byte of a string. The message is long because it explains the trait that would be needed and lists the types that implement it, which is unusual detail for an index error.

fn main() { let text = String::from("hello"); println!("{}", text[0]); }

rustc --edition 2024 index_err.rs

error[E0277]: the type `str` cannot be indexed by `{integer}`
--> index_err.rs:3:25
|
3 | println!("{}", text[0]);
| ^ string indices are ranges of `usize`
|
= help: the trait `SliceIndex<str>` is not implemented for `{integer}`
= note: you can use `.chars().nth()` or `.bytes().nth()`
for more information, see chapter 8 in The Book: <https://doc.rust-lang.org/book/ch08-02-strings.html#indexing-into-strings>
help: `usize` implements trait `SliceIndex<T>`
--> /rustc/31fca3adb283cc9dfd56b49cdee9a96eb9c96ffd/library/core/src/slice/index.rs:214:0
|
= note: `SliceIndex<[T]>`
--> /rustc/31fca3adb283cc9dfd56b49cdee9a96eb9c96ffd/library/core/src/bstr/traits.rs:197:0
|
= note: `SliceIndex<ByteStr>`
= note: required for `String` to implement `Index<{integer}>`

error: aborting due to 1 previous error

For more information about this error, try `rustc --explain E0277`.

The lesson on string slices shows the range form that does work, and why its indices must fall on character boundaries.

Exercises

  1. Print the byte length of your name and the number of characters. Where do the two differ?
  2. Add a fourth permission to the flag set and test it before and after it is set.
  3. Clear bit zero of an odd number and print the result in binary. Which operator does the job?

Questions

A new set of questions in a new order every time. Choose one answer for each, then check.

  1. How many bytes does a char occupy in memory?

    Answer

    Four

    A char must hold any Unicode scalar value, so it is a fixed four-byte type.

  2. What does text.len() return for a String?

    Answer

    The number of bytes in UTF-8

    len counts bytes, which equals characters only for ASCII text.

  3. How is a bit flag usually represented?

    Answer

    An integer with one bit per flag

    One bit per flag lets the operators set, clear and test several flags at once.

  4. Why does text[0] not compile for a String?

    Answer

    Because a byte index is not a character index, and the compiler will not guess

    The string is UTF-8, so the compiler requires an explicit choice of chars or bytes.

  5. What does this program print? fn main() { let letter = 'é'; println!("{} {}", letter, letter.len_utf8()); }

    Answer

    é 2

    The accented letter is one char and two bytes in UTF-8.

  6. What does this program print? fn main() { let text = "héllo"; println!("bytes: {}", text.len()); println!("chars: {}", text.chars().count()); }

    Answer

    bytes: 6 / chars: 5

    The accented letter takes two bytes but is one character, so the counts differ by one.

  7. What does this program print? const READ: u8 = 0b0001; const WRITE: u8 = 0b0010; fn main() { let mut permissions = READ | WRITE; println!("{}", permissions & READ != 0); permissions &= !WRITE; println!("{:04b}", permissions); }

    Answer

    true / 0001

    Read is set, write is cleared, and the binary print shows only the read bit.

  8. What does this program print? fn main() { let bit = 1u8 << 3; println!("{bit} {:08b}", bit); }

    Answer

    8 00001000

    Shifting one left by three gives eight, which is bit three in binary.

  9. What is the first line of the compiler output for this program? fn main() { let text = String::from("hello"); println!("{}", text[0]); }

    Answer

    error[E0277]: the type str cannot be indexed by {integer}

    The index trait is not implemented for an integer on str, and the help names chars and bytes.

  10. What does this program print? fn main() { let text = "aé b"; for (index, ch) in text.char_indices() { println!("{index}: {ch}"); } }

    Answer

    0: a / 1: é / 3: / 4: b

    The indices are byte positions, and the accented letter takes bytes one and two.

The code on this page was built with rustc 1.96.1 and run on Alpine Linux 3.24.3 (x86_64) in a Docker container.

Article author: Arthur Isaev

Banner image