Bytes and binary files
A binary file is a sequence of bytes with a layout the program defines, and every field in it is a decision the programmer has to write down: its width, its byte order and what a missing value looks like. Rust reads and writes it as &[u8] and Vec<u8>, and numbers are converted to bytes explicitly with to_le_bytes and its partners. Nothing writes a struct directly, because the layout of a struct is not guaranteed.
The lesson writes a byte vector, adds a length header before a payload, and converts numbers in both directions. The last section is the error for passing a number to a function that wants bytes, which is the first mistake a text-file programmer makes with a binary file.
Raw bytes
fs::write accepts any byte slice, and fs::read returns a Vec<u8>. Both report an io error, like the text functions, and the vector is the whole file.
The program writes four bytes and reads them back, which is the whole cycle a binary format needs before it is worth designing. The output is the length and the four values, including the two hundred and fifty-five that a signed byte could not hold.
use std::fs;
fn main() {
let data: Vec<u8> = vec![0, 1, 2, 255];
fs::write("raw.bin", &data).unwrap();
let read = fs::read("raw.bin").unwrap();
println!("{} {:?}", read.len(), read);
}
rustc --edition 2024 bytes.rs
./bytes
4 [0, 1, 2, 255]
The file is the same on every machine, because a byte is a byte: there is no encoding and no newline translation. The bytes are printed in decimal because the debug form of a byte is a number. A viewer that shows them as characters would show a control character, a null and a replacement glyph.
A length header
A file with several messages needs to know where each one ends. The usual answer is a length before the payload, written as a fixed-width integer and read with read_exact, which fills the whole buffer or fails.
The program writes the length of the word rust, then the word, then reads it back. The output is four and the word, which proves both sides agree on the format.
use std::fs::File;
use std::io::{Read, Write};
fn main() {
let text = b"rust";
let mut file = File::create("message.bin").unwrap();
file.write_all(&(text.len() as u32).to_le_bytes()).unwrap();
file.write_all(text).unwrap();
let mut input = File::open("message.bin").unwrap();
let mut length = [0u8; 4];
input.read_exact(&mut length).unwrap();
let length = u32::from_le_bytes(length) as usize;
let mut body = vec![0u8; length];
input.read_exact(&mut body).unwrap();
println!("{} {}", length, String::from_utf8(body).unwrap());
}
rustc --edition 2024 length_prefix.rs
./length_prefix
4 rust
The header costs four bytes per message, and a format with many small messages pays for it. A length read from a file is untrusted, which is why a real format caps it before allocating. The lesson on error types makes that limit a named error.
Numbers and their bytes
to_le_bytes writes the least significant byte first, and from_le_bytes reads the same order back. The big-endian pair exists as well, and a format must pick one and say so.
The program converts a number to four bytes, back again, and reads a big-endian seven. The byte list and the two numbers are the output.
fn main() {
let number: u32 = 0x1234_5678;
let bytes = number.to_le_bytes();
println!("{:?}", bytes);
println!("{:08x}", u32::from_le_bytes(bytes));
println!("{}", i32::from_be_bytes([0, 0, 0, 7]));
}
rustc --edition 2024 endian.rs
./endian
[120, 86, 52, 18]
12345678
7
A float converts with the same methods and four or eight bytes, and a bool needs a decision because its size is not fixed by the language. The byte order matters as soon as two machines exchange a file, and the choice belongs in the format documentation. Rust makes it explicit instead of hiding it in a cast.
A number is not a byte slice
write_all takes &[u8], so passing an i32 is E0308. The conversion is to_le_bytes and a borrow of the array, which states the width and the order in one expression.
The program writes a number without converting it. The error names the expected slice and the found integer, and the fix is one call.
use std::fs::File;
use std::io::Write;
fn main() {
let mut file = File::create("number.bin").unwrap();
let number: i32 = 42;
file.write_all(number).unwrap();
println!("done");
}
rustc --edition 2024 write_err.rs
error[E0308]: mismatched types
--> write_err.rs:7:20
|
7 | file.write_all(number).unwrap();
| --------- ^^^^^^ expected `&[u8]`, found `i32`
| |
| arguments to this method are incorrect
|
note: method defined here
--> /rustc/31fca3adb283cc9dfd56b49cdee9a96eb9c96ffd/library/std/src/io/mod.rs:1857:7
error: aborting due to 1 previous error
For more information about this error, try `rustc --explain E0308`.
A struct needs a defined layout before its bytes mean anything, and the lesson on memory layout shows the attribute for that. Writing the bytes of a struct with a pointer would produce a file nobody can read, which is why the explicit conversion is the rule and not a limitation.
Exercises
- Write five numbers as bytes and read them back with read_exact.
- Store a string with a length header and read it in a second program.
- Write the same number little-endian and big-endian and compare the byte lists.
Questions
A new set of questions in a new order every time. Choose one answer for each, then check.
-
What does fs::read return?
Answer
A Result with a Vec of bytes
The function returns the bytes of the file, or the io error.
-
Why does a binary format write a length before the payload?
Answer
So the reader knows where the payload ends
Without a length the reader cannot tell where the next field begins.
-
What does to_le_bytes promise?
Answer
The least significant byte comes first
Little-endian puts the low byte first, and from_le_bytes reverses it.
-
What does read_exact do when the buffer cannot be filled?
Answer
It returns an error
The contract is the whole buffer, and a short read is an error.
-
What does this program print?
use std::fs; fn main() { let data: Vec<u8> = vec![0, 1, 2, 255]; fs::write("raw.bin", &data).unwrap(); let read = fs::read("raw.bin").unwrap(); println!("{} {:?}", read.len(), read); }Answer
4 [0, 1, 2, 255]
The four bytes come back unchanged, including the maximum byte value.
-
What does this program print?
fn main() { let number: u32 = 0x1234_5678; let bytes = number.to_le_bytes(); println!("{:?}", bytes); println!("{:08x}", u32::from_le_bytes(bytes)); }Answer
[120, 86, 52, 18] / 12345678
The low byte comes first, and the reverse conversion restores the same number.
-
What does this program print?
fn main() { println!("{}", i32::from_be_bytes([0, 0, 0, 7])); }Answer
7
Big-endian puts the most significant byte first, so the value is seven.
-
What does this program print?
use std::fs::File; use std::io::{Read, Write}; fn main() { let text = b"rust"; let mut file = File::create("message.bin").unwrap(); file.write_all(&(text.len() as u32).to_le_bytes()).unwrap(); file.write_all(text).unwrap(); let mut input = File::open("message.bin").unwrap(); let mut length = [0u8; 4]; input.read_exact(&mut length).unwrap(); println!("{}", u32::from_le_bytes(length)); }Answer
4
The header stores the length of the word, which is four bytes.
-
What is the first line of the compiler output for this program?
use std::fs::File; use std::io::Write; fn main() { let mut file = File::create("number.bin").unwrap(); let number: i32 = 42; file.write_all(number).unwrap(); }Answer
error[E0308]: mismatched types
The method wants a byte slice and the call passes an integer.
-
What does this program print?
use std::fs; fn main() { fs::write("copy.bin", b"abc").unwrap(); let read = fs::read("copy.bin").unwrap(); println!("{:?}", String::from_utf8(read).unwrap()); }Answer
"abc"
The bytes are valid UTF-8, and the debug print of the string shows its quotes.
The code on this page was built with rustc 1.96.1 and run on Alpine Linux 3.24.3 (x86_64) in a Docker container.
Article author: Arthur Isaev