io.Reader and io.Writer
Two interfaces carry almost all the input and output of Go: io.Reader is one method that fills a byte slice, and io.Writer is one method that takes one. Files, connections, buffers, compressors and hashers implement them, which is why io.Copy can move bytes between any two of them without knowing what either is.
io.Copy
A strings.Reader is a reader over a string and a bytes.Buffer is both a reader and a writer. io.Copy reads a chunk, writes it, and repeats until the reader returns io.EOF, so the copy works in a fixed buffer of its own and never holds the whole input. It returns the number of bytes copied.
package main
import (
"bytes"
"fmt"
"io"
"strings"
)
func main() {
src := strings.NewReader("hello, io")
var dst bytes.Buffer
n, err := io.Copy(&dst, src)
fmt.Println("copied:", n, "err:", err)
fmt.Println("buffer:", dst.String())
}
go build -o app main.go && ./app
copied: 9 err: <nil>
buffer: hello, io
The copy returned nine bytes and no error, and the buffer holds the text. A destination that is a file, a connection or a compressor makes the same call with no other change, which is the practical value of the two interfaces: the code that copies does not know what it copies from or to.
io.LimitReader
LimitReader wraps a reader and returns EOF after a number of bytes, which is how a program reads a header without trusting the size that follows it. The wrapper is itself a reader, so it composes with everything else, and the ten-byte source below gives up only four.
package main
import (
"fmt"
"io"
"strings"
)
func main() {
src := strings.NewReader("0123456789")
limited := io.LimitReader(src, 4)
data, err := io.ReadAll(limited)
fmt.Println("data:", string(data), "err:", err)
}
go build -o app main.go && ./app
data: 0123 err: <nil>
The four bytes came out and the read ended cleanly, because a limited reader reports EOF rather than an error when the limit is reached. io.ReadFull is the strict version: it returns an error when fewer bytes arrive than asked, which is what a fixed-size header needs.
A Reader written by hand
Anything with a Read([]byte) (int, error) method is a reader. The one below fills the slice with x until a counter runs out and then returns io.EOF. A single Read must not promise more than it puts in p, and it may return fewer bytes than the slice holds along with a nil error, which is the part a hand-written reader gets wrong.
package main
import (
"fmt"
"io"
)
type repeatReader struct {
remaining int
}
func (r *repeatReader) Read(p []byte) (int, error) {
if r.remaining == 0 {
return 0, io.EOF
}
n := len(p)
if n > r.remaining {
n = r.remaining
}
for i := 0; i < n; i++ {
p[i] = 'x'
}
r.remaining -= n
return n, nil
}
func main() {
data, err := io.ReadAll(&repeatReader{remaining: 5})
fmt.Println("data:", string(data), "err:", err)
}
go build -o app main.go && ./app
data: xxxxx err: <nil>
io.ReadAll hid the chunking: it called Read with a growing buffer until EOF and joined the pieces. A reader that returns 0, nil forever makes the loop spin, and a reader that returns an error before EOF makes ReadAll return the bytes so far with the error, which is what a partial download looks like.
A Writer written by hand
A writer needs one method, Write([]byte) (int, error), and it must report how many bytes of the slice it consumed. The counter below takes everything and adds the length to a field, and fmt.Fprintf uses it through the interface without knowing the type.
package main
import "fmt"
type counter struct {
bytes int
}
func (c *counter) Write(p []byte) (int, error) {
c.bytes += len(p)
return len(p), nil
}
func main() {
var c counter
n, err := fmt.Fprintf(&c, "counted %d bytes", 12)
fmt.Println("write:", n, "err:", err)
fmt.Println("total:", c.bytes)
}
go build -o app main.go && ./app
write: 16 err: <nil>
total: 16
The formatted line is sixteen characters and both the return value and the counter agree. A writer that consumes fewer bytes than it was given must return an error, because the caller has no way to retry the rest; that rule is the whole contract of the interface.
The two contracts
A reader returns the bytes it read and an error; io.EOF at the end is not a failure. A writer returns the bytes it wrote and an error; a short write without an error is a bug. Both interfaces are small enough to implement in a few lines, which is why the standard library has dozens of wrappers: a io.TeeReader writes what it reads to a second writer, a bufio.Reader buffers a reader, and a compressor turns a writer into another writer.
The next lesson buffers both directions. The one after that treats a file as a reader, a writer and a filesystem, and the capstone HTTP service uses the same two interfaces for its request bodies and responses.
Exercises
- Run the copy with a io.TeeReader in front of the source and see what the second writer receives. Which method does the tee implement?
- Make the hand-written reader return 0, nil once before EOF and run the program. What does io.ReadAll do with it, and why is it legal?
- Give the counter a limit that returns a short write and read the error from fmt.Fprintf. What does Fprintf do when the writer refuses part of the line?
Questions
A new set of questions in a new order every time. Choose one answer for each, then check.
-
What is the method of io.Reader?
Answer
Read(p []byte) (int, error)
It fills the slice and reports how many bytes it used.
-
What is the method of io.Writer?
Answer
Write(p []byte) (int, error)
It consumes the slice and reports how much.
-
How does io.Copy know when to stop?
Answer
The reader returns io.EOF
EOF is the normal end of a reader.
-
What does io.LimitReader do at the limit?
Answer
Returns io.EOF
The limited input simply ends.
-
What may a single Read return?
Answer
Fewer bytes than the slice with a nil error
The caller repeats until EOF.
-
What must a writer report when it consumes fewer bytes than given?
Answer
An error
A short write without an error is a bug.
-
Which pairs work with io.Copy?
Answer
A reader and a writer, whatever their types
The interfaces are the whole contract.
-
What is io.TeeReader for?
Answer
Writing what is read to a second writer
It is a reader and a side channel.
-
What does io.ReadAll return when a reader fails before EOF?
Answer
The bytes so far and the error
A partial download looks exactly like that.
-
Which type is both a Reader and a Writer?
Answer
bytes.Buffer
A buffer can be read from and written to.
The code on this page was run with go 1.26.10 on Alpine Linux 3.24.3 (x86_64) in a Docker container.
Article author: Arthur Isaev