The launch checklist and the maintenance plan

The domain

The name is the part of the site that cannot be rebuilt from a backup. Check it first, and check it once a quarter rather than once.

• The registrant data is correct and the email address still works.
• Auto-renew is on, and the card on file does not expire this year.
• The registrar account has a unique password and two-factor authentication.
• The transfer lock is on: WHOIS shows clientTransferProhibited.
• The name servers are the ones you chose, and DNSSEC is either on and working, or off.
• The expiry date is in your calendar, one month before the real date.
• The site has a second name, www, and it redirects to the primary one.

The details are in lesson 1.

Hosting and site

• The domain resolves to the right address over IPv4 and IPv6: dig +short example.com, dig +short AAAA example.com.
• https://example.com answers 200, and http:// redirects to it once.
• The certificate covers both names, is valid, and renews by itself.
• The site can save a post, send a form and show an upload: the three things that touch the database, mail and disk.
• The CMS and every plugin are updated, and the admin account has two-factor authentication.
• No .git, .env or database dump is reachable under the web root.
• The database is reachable only from the server, and its user is not root.
• Files and database have a backup from this week, on another machine.
• A 404 page exists and looks like the site, not like a server default.

Lessons 3 and 4 build this list on shared hosting; lesson 5 does the same on your own server.

The server

• SSH accepts keys only; root cannot log in; fail2ban is running.
• ufw allows the ports you need and nothing else: sudo ufw status.
• Automatic security updates are on, and the server has been rebooted since the last kernel update.
• Only the services you know listen on a public address: sudo ss -tlnp.
• Every site has its own system user and its own PHP-FPM pool.
• Configuration files with secrets are 600, site files are 640 or 750.
• HSTS and the security headers are present, and the version strings are hidden.
• Rate limits are set on the web server.
• The provider account and the panel account have two-factor authentication.

This is the checklist of lesson 8, compressed. Run it again after every major change, not only at launch.

Mail

• MX records point at the mail server, and the mail server answers on port 25.
• The SPF record lists every sender, and there is exactly one SPF record.
• DKIM signs outgoing mail, and the public key is published.
• DMARC exists, reports arrive, and the policy is at least quarantine.
• The PTR record of the address matches the name the server announces.
• Outgoing port 25 is open, or a relay is configured.
• The server is not an open relay, and port 587 requires authentication.
• Mail to and from a large provider passes SPF, DKIM and DMARC.
• The mailbox has a strong password, and the mail client uses TLS.

The commands and the record values are in lesson 9. A site can go live without mail; a business cannot.

The maintenance schedule

A server does not need attention every day, but it needs a rhythm. This one fits a small site and takes about ten minutes a day, half an hour a week, and an hour a month.

The commands are ordinary ones from this course: cron runs the backup, sudo apt upgrade patches the system, df -h checks the disk, and journalctl reads the services. The habit matters more than the tool: a calendar reminder that nobody ignores is the cheapest monitoring there is.

When the site is down

Work from the outside in, and stop at the first layer that answers wrongly. Most answers come from three tools: dig for the name, curl for the page and df for the disk. The order is the same every time, which is why it is worth memorising.

1. Is it the site or the visitor? Ask someone else, or use a different network, before you change anything. A local DNS cache or a provider outage looks exactly like a broken server from one chair.

2. Does the name resolve?

dig +short example.com

No answer, or the wrong address, means DNS: the record changed, the domain expired, or the name servers are down. Compare with a public resolver and read the delegation with dig +trace.

3. Does the port answer?

nc -vz example.com 443

Connection to example.com 443 port [tcp/https] succeeded!

A timeout means the firewall, the provider network or a stopped web server. Test from the server itself: if localhost answers and the internet does not, the problem is the path.

4. Does the site answer over HTTP?

curl -I https://example.com

HTTP/2 502 server: nginx

A 502 means Nginx is alive and PHP-FPM is not, or the socket path is wrong. A 500 means PHP ran and the application failed; read its error log. A 403 is a permission or an index problem. A 404 with the right server is a site configuration problem. The codes themselves are in HTTP status codes.

5. Is the disk full? A full disk breaks the database, the logs and the sessions at once, and the error messages point everywhere except at the disk.

df -h /

df -i /

Filesystem Size Used Avail Use% Mounted on /dev/vda1 39G 38G 0G 100% /

A hundred percent used, or a hundred percent of inodes, is the answer. Delete old logs and old backups, then find what grew: du -h /var/log/* | sort -h | tail. Do not delete the database files to make space; move a backup off the server instead.

6. Is the database running?

systemctl status mariadb --no-pager

sudo journalctl -u mariadb -n 20 --no-pager

A database that stops after a disk-full event usually starts again once space is free. A database that stops for another reason has a message in the journal that names the table or the file.

7. What changed? The last deploy, the last plugin update, the last certificate renewal, the last firewall rule. In practice the answer is almost always in the last hour, and the fastest fix is to undo it before you understand it. Then read the logs and understand it.

8. When nothing works, restore. A site that cannot be fixed in half an hour is a site that goes back from the last good backup, while you investigate on a copy. That is why the backup has a restore test in the schedule: at the worst moment, you do not want to learn the restore procedure.

Write down what you found, even when the answer was simple. The next outage will be different, and your notes will be the fastest way to the second line of the diagnosis.

Quiz

Article author: Arthur Isaev