Graceful shutdown
A server that stops on a signal should finish the requests it accepted and refuse new ones, not close the sockets under a handler that is halfway through a write. Server.Shutdown does exactly that: it closes the listener, waits for the active handlers, and returns when they are done or when its context expires. This lesson runs the shutdown against a request that is deliberately slow.
Shutdown waits for a request
The handler signals that it started, then sleeps thirty milliseconds. The client runs in a goroutine, and the main goroutine calls Shutdown as soon as the handler is in flight. Shutdown returns only after the handler finished, so the response is complete and the call took about as long as the handler.
package main
import (
"context"
"fmt"
"io"
"net"
"net/http"
"time"
)
func main() {
started := make(chan struct{})
done := make(chan string)
mux := http.NewServeMux()
mux.HandleFunc("GET /slow", func(w http.ResponseWriter, r *http.Request) {
close(started)
time.Sleep(30 * time.Millisecond)
fmt.Fprintln(w, "slow response")
})
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
panic(err)
}
srv := &http.Server{Handler: mux}
go srv.Serve(ln)
go func() {
resp, err := http.Get("http://" + ln.Addr().String() + "/slow")
if err != nil {
done <- "error: " + err.Error()
return
}
body, _ := io.ReadAll(resp.Body)
resp.Body.Close()
done <- string(body)
}()
<-started
ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
start := time.Now()
err = srv.Shutdown(ctx)
fmt.Println("shutdown error:", err)
fmt.Println("shutdown under 100ms:", time.Since(start) < 100*time.Millisecond)
fmt.Print("response: ", <-done)
}
go build -o app main.go && ./app
shutdown error: <nil>
shutdown under 100ms: true
response: slow response
The response arrived complete, and the shutdown took thirty-odd milliseconds because it waited for the handler. That is the whole promise: a deploy that restarts a pod does not cut a request in half, as long as the time it allows for the stop is longer than the slowest handler.
The listener closes
Shutdown also closes the listener, so a client that connects after the stop is refused instead of waiting. The program below makes one request, shuts the server down, and makes a second one; the second fails at the connection and the client reports the error.
package main
import (
"context"
"fmt"
"net"
"net/http"
"time"
)
func main() {
mux := http.NewServeMux()
mux.HandleFunc("GET /hello", func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintln(w, "hello")
})
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
panic(err)
}
srv := &http.Server{Handler: mux}
go srv.Serve(ln)
url := "http://" + ln.Addr().String() + "/hello"
resp, err := http.Get(url)
if err != nil {
panic(err)
}
resp.Body.Close()
ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
if err := srv.Shutdown(ctx); err != nil {
panic(err)
}
fmt.Println("after shutdown:")
client := &http.Client{Timeout: 100 * time.Millisecond}
_, err = client.Get(url)
fmt.Println("second request failed:", err != nil)
}
go build -o app main.go && ./app
after shutdown:
second request failed: true
The second call could not connect because nothing listens on the port any more. A load balancer that removes the instance before the signal arrives is what makes this window harmless; a service that stops at the same instant it is removed from the pool drops the requests in between.
signal.NotifyContext
signal.NotifyContext returns a context that is cancelled when the process receives one of the signals, which is the bridge between the operating system and the context the rest of the program already uses. The program below has a 150 millisecond fallback so it also ends when no signal arrives, and the run shown sent a real SIGINT to it.
package main
import (
"context"
"fmt"
"os"
"os/signal"
"syscall"
"time"
)
func main() {
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
fmt.Println("waiting for a signal or 150ms")
select {
case <-ctx.Done():
fmt.Println("signal received:", ctx.Err())
case <-time.After(150 * time.Millisecond):
fmt.Println("no signal, stopping on the timer")
}
}
go build -o app main.go && (./app & pid=$!; sleep 0.05; kill -INT $pid; wait $pid)
waiting for a signal or 150ms
signal received: context canceled
The context reports the cancellation and not the signal number, which is what the shutdown code wants. The timer branch is the output of the same binary run without a signal, and it is there so the program is runnable outside a terminal; a real service drops it and blocks on the context alone.
A shutdown that runs out of time
The context passed to Shutdown is the budget for the stop. A handler that takes a hundred milliseconds against a ten millisecond budget makes Shutdown return an error and leaves the handler running; the process is about to exit, so the connection is closed under it. A service sets the budget from its platform: a Kubernetes pod gets about thirty seconds between SIGTERM and SIGKILL by default.
package main
import (
"context"
"fmt"
"net"
"net/http"
"time"
)
func main() {
mux := http.NewServeMux()
mux.HandleFunc("GET /slow", func(w http.ResponseWriter, r *http.Request) {
time.Sleep(100 * time.Millisecond)
fmt.Fprintln(w, "slow")
})
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
panic(err)
}
srv := &http.Server{Handler: mux}
go srv.Serve(ln)
go func() {
resp, err := http.Get("http://" + ln.Addr().String() + "/slow")
if err == nil {
resp.Body.Close()
}
}()
time.Sleep(10 * time.Millisecond)
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Millisecond)
defer cancel()
err = srv.Shutdown(ctx)
fmt.Println("shutdown error:", err)
}
go build -o app main.go && ./app
shutdown error: context deadline exceeded
The error is the shutdown's context expiring, not the handler failing. A service logs it and exits anyway, because a process that does not stop is worse than a request that did not finish; the log line is the signal that the budget or the handler needs a look.
The order of a stop
A service stops in one order. The signal cancels the root context. The server shuts down and stops taking work. The background workers see the same context and return. The database pool closes. The log flushes. Every step has a deadline, and the whole thing fits inside the platform's grace period, or the platform kills the process in the middle of a step.
The programs above are the pieces: NotifyContext, Shutdown, the context that reaches the handlers through r.Context(). The capstone service wires them and its deploy lesson runs the stop against a container. The next lesson opens the database that makes the stop worth doing carefully.
Exercises
- Make the handler sleep two hundred milliseconds and the shutdown budget fifty. What does Shutdown return, and what does the client see?
- Send SIGTERM instead of SIGINT. Does the program take the same branch, and why must a service handle both?
- Call Shutdown twice in a row. What does the second call do, and what does that say about the function?
Questions
A new set of questions in a new order every time. Choose one answer for each, then check.
-
What does Server.Shutdown do?
Answer
Closes the listener, waits for active handlers, returns when done or the context ends
It is the graceful stop.
-
What happens to a request that arrives after Shutdown starts?
Answer
It cannot connect, because the listener is closed
The pool must remove the instance first.
-
What does signal.NotifyContext return?
Answer
A context cancelled when the signal arrives
It bridges the OS signal to the context.
-
Which signals should a service handle?
Answer
SIGINT and SIGTERM
SIGKILL cannot be handled at all.
-
What does the shutdown context bound?
Answer
How long the stop may take
The platform kills the process after its grace period.
-
What does Shutdown return when the budget expires?
Answer
context.DeadlineExceeded
The handler is still running.
-
Why does the slow handler not lose its response?
Answer
Shutdown waits for it before returning
That is the point of a graceful stop.
-
What is the stop order in a service?
Answer
Signal, server, workers, database, log flush
Work stops before the resources it uses.
-
What should a service log when the budget expires?
Answer
The shutdown error and the request that was in flight
The budget or the handler needs a look.
-
Why is a process that does not stop worse than a lost request?
Answer
The platform kills it anyway and may cut other work
A bounded stop is part of the design.
The code on this page was run with go 1.26.10 on Alpine Linux 3.24.3 (x86_64) in a Docker container.
Article author: Arthur Isaev