Mail on your own domain

Hosted mail or your own server

Mail is the hardest part of running a website, and the part most people should not run themselves. A mailbox is easy to create; getting a message into a stranger inbox is the hard problem, because every large provider decides by reputation, not by your good intentions.

Hosted mail serviceYour own mail server
SetupChange MX records and create mailboxesInstall, configure and maintain two services
DeliverabilityHandled by a provider with a reputationYours to build from zero, one IP at a time
CostPer user, per monthThe server you already pay for
ControlLimited to what the provider offersEvery setting, every log line
MaintenanceThe provider patches itYou patch Postfix, Dovecot and the filters
Good forA business, a real mailbox, a shared teamLearning, many addresses, a mail gateway you understand

A small site with one mailbox is better served by a hosted service or by the mail of the hosting provider from lesson 4. Build your own when you want to learn how mail works, when you need many addresses, or when you already run a server and want the logs in your own hands. The rest of this lesson does exactly that, on the same Ubuntu machine.

MX, SPF, DKIM, DMARC

Four records decide whether your mail is delivered, and each answers a different question.

With a hosted service, the provider gives you the exact values and you paste them into the DNS panel. With your own server, they look like this:

example.com. 3600 IN MX 10 mail.example.com. mail.example.com. 3600 IN A 203.0.113.10 example.com. 3600 IN TXT "v=spf1 mx -all" mail._domainkey.example.com. 3600 IN TXT "v=DKIM1; k=rsa; p=MIIBIjANBg..." _dmarc.example.com. 3600 IN TXT "v=DMARC1; p=none; rua=mailto:dmarc@example.com"

The SPF record means: the servers in the MX records may send, and everything else is a failure. There must be exactly one SPF record per name, and the record may cause at most ten DNS lookups. The DKIM value is long and is copied exactly, without line breaks. DMARC passes when SPF or DKIM passes and is aligned with the domain in the From header; it fails only when neither gives an aligned pass. SPF is checked against the envelope sender, which a relay or a mailing tool may rewrite, so an SPF pass is not automatically aligned; DKIM signs the From domain itself. DMARC starts at p=none, which only reports; after a month of clean reports, raise it to quarantine and then to reject.

Reverse DNS

Forward DNS turns a name into an address. Reverse DNS does the opposite, and mail servers check it: a message from an IP address whose reverse name is missing, or does not match the name in the greeting, is treated as suspicious. You cannot create it yourself; the owner of the address must set it.

In the panel of the VPS provider, find the reverse DNS or PTR setting for your IPv4 address and set it to mail.example.com. The IPv6 address needs its own PTR record. Then check from outside:

dig +short -x 203.0.113.10

mail.example.com.

The name in the answer must match the name your mail server announces in its greeting, which is myhostname in Postfix. A mismatch is the single most common reason a new mail server lands in spam.

Port 25 is often closed

Many VPS providers block outgoing port 25 by default, because it is the port spam uses. Mail still arrives, since incoming port 25 is usually open, but nothing you send leaves the machine. Test it before you configure anything:

sudo apt install netcat-openbsd

nc -vz alt1.gmail-smtp-in.l.google.com 25

Connection to alt1.gmail-smtp-in.l.google.com 25 port [tcp/smtp] succeeded!

A timeout instead of a success means the provider blocks the port. Ask support to open it, which many providers do for a verified account, or send through a relay: a smarthost that accepts your mail on port 587 with authentication and delivers it for you. A relay is also the practical answer when your IP has a poor reputation and you do not want to wait weeks for it to improve.

Postfix and Dovecot

Postfix moves mail between servers; Dovecot lets a mail client read the mailbox over IMAP. The firewall from lesson 8 allows only SSH and the web ports, so open the mail ports first, or the services will run and stay unreachable:

sudo ufw allow 25/tcp

sudo ufw allow 587/tcp

sudo ufw allow 993/tcp

Then install both services, and answer Internet Site with example.com as the system mail name when the installer asks:

sudo apt install postfix dovecot-core dovecot-imapd

The main configuration holds the identity of the server, the domains it accepts mail for, and the TLS certificate. Ubuntu's main.cf already contains most of these keys, so editing the file by hand means finding each old line and removing it. postconf -e replaces the value in place, one setting per command:

sudo postconf -e 'myhostname = mail.example.com'

sudo postconf -e 'mydomain = example.com'

sudo postconf -e 'myorigin = $mydomain'

sudo postconf -e 'inet_interfaces = all'

sudo postconf -e 'inet_protocols = ipv4'

sudo postconf -e 'mydestination = $myhostname, $mydomain, localhost.$mydomain, localhost'

sudo postconf -e 'home_mailbox = Maildir/'

sudo postconf -e 'mynetworks = 127.0.0.0/8'

sudo postconf -e 'relay_domains ='

sudo postconf -e 'smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination'

sudo postconf -e 'smtpd_tls_cert_file = /etc/letsencrypt/live/example.com/fullchain.pem'

sudo postconf -e 'smtpd_tls_key_file = /etc/letsencrypt/live/example.com/privkey.pem'

sudo postconf -e 'smtpd_tls_security_level = may'

sudo postconf -e 'smtpd_sasl_type = dovecot'

sudo postconf -e 'smtpd_sasl_path = private/auth'

sudo postconf -e 'smtpd_sasl_auth_enable = yes'

sudo postconf -e 'smtpd_tls_auth_only = yes'

sudo systemctl reload postfix

mydestination lists the domains this server considers its own; relay_domains is empty, so it forwards nothing for anyone else. home_mailbox makes Postfix write each message into the Maildir folder of the user, which is where Dovecot reads it. The relay restrictions allow mail from the machine itself and from authenticated users, and refuse everything else for outside destinations. That single line is what makes an open relay impossible.

smtpd_tls_auth_only = yes is the other half of the lock: with authentication enabled globally, it makes Postfix refuse a password until TLS is up. Without it, port 25 would accept a password in clear text before the encrypted session starts.

The submission port, 587, is where mail clients send. The client connects in clear text, switches to TLS with STARTTLS, and only then may authenticate. Port 465 is the other arrangement, called implicit TLS: there the TLS session starts with the connection, from the first byte.

sudo vi /etc/postfix/master.cf

submission inet n - y - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_relay_restrictions=permit_sasl_authenticated,reject

Dovecot needs a mailbox location, TLS, and a socket that Postfix uses to check passwords. Three files hold them:

sudo vi /etc/dovecot/conf.d/10-mail.conf

mail_location = maildir:~/Maildir

sudo vi /etc/dovecot/conf.d/10-ssl.conf

ssl = required ssl_cert = </etc/letsencrypt/live/example.com/fullchain.pem ssl_key = </etc/letsencrypt/live/example.com/privkey.pem

sudo vi /etc/dovecot/conf.d/10-master.conf

service auth { unix_listener /var/spool/postfix/private/auth { mode = 0660 user = postfix group = postfix } }

The block goes inside the existing service auth { } section, where it replaces the commented-out listener of the same name that the package already put there.

Restart both and create the first mailbox as a system user, which is the simplest setup for a handful of addresses:

sudo systemctl restart postfix dovecot

sudo adduser you

The account logs in to IMAP as you, with its system password, on port 993 with TLS, and sends on port 587 with STARTTLS. A larger setup stores users in a database, which is the same idea with more moving parts.

DKIM signing

DKIM signs every outgoing message with a private key, and the public key sits in a TXT record so receivers can verify the signature. Generate a 2048-bit key for the selector mail:

sudo apt install opendkim opendkim-tools

sudo mkdir -p /etc/dkimkeys

sudo opendkim-genkey -b 2048 -v -d example.com -s mail -D /etc/dkimkeys

opendkim-genkey: generating private key opendkim-genkey: private key written to mail.private opendkim-genkey: extracting public key opendkim-genkey: DNS TXT record written to mail.txt

opendkim-genkey is silent unless -v is given, which is why the command above has it. The command runs as root, so the private key belongs to root, and the OpenDKIM service, which runs as opendkim, cannot read it and would sign nothing. Give the key to that user:

sudo chown opendkim:opendkim /etc/dkimkeys/mail.private

The file /etc/dkimkeys/mail.txt holds the DNS record; publish its content as a TXT record at mail._domainkey.example.com. The file splits the long key over several quoted strings for readability; join them into one value, without spaces or line breaks.

Then tell OpenDKIM which key signs for which domain. Ubuntu's /etc/opendkim.conf already has an active Socket line, local:/run/opendkim/opendkim.sock: replace that line with the inet socket below, do not add a second one:

sudo vi /etc/opendkim.conf

Domain example.com Selector mail KeyFile /etc/dkimkeys/mail.private Socket inet:12301@localhost

Connect Postfix to the milter, then restart both services:

sudo postconf -e 'milter_default_action = accept'

sudo postconf -e 'smtpd_milters = inet:localhost:12301'

sudo postconf -e 'non_smtpd_milters = inet:localhost:12301'

sudo systemctl restart opendkim postfix

dig +short TXT mail._domainkey.example.com

"v=DKIM1; k=rsa; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA..."

A signature that verifies at the receiver is the goal, not a record that exists. Send a message to a mailbox at a large provider and read its headers, as the testing section shows.

TLS for mail, no open relay

The certificate for mail.example.com comes from certbot like any other. Add the name to the server block of the site, so that the HTTP check can reach it:

sudo vi /etc/nginx/sites-available/example.com

server_name example.com www.example.com mail.example.com;

sudo nginx -t

sudo systemctl reload nginx

Then issue one certificate for all three names:

sudo certbot --nginx -d example.com -d www.example.com -d mail.example.com

Postfix and Dovecot now read the files under /etc/letsencrypt/live/example.com/: certbot names the folder after the first domain in the command, and the certificate itself covers all three. The nginx plugin of certbot reloads nginx only, so Postfix and Dovecot need a deploy hook of their own; every renewal runs it:

sudo vi /etc/letsencrypt/renewal-hooks/deploy/reload-mail.sh

#!/bin/sh systemctl reload postfix dovecot

sudo chmod 755 /etc/letsencrypt/renewal-hooks/deploy/reload-mail.sh

Now prove that the server does not relay. From another machine, connect to port 25 and try to send a message to an address that is not yours. The server must refuse with a relay error:

swaks --to someone@example.net --server 203.0.113.10:25

<** 554 5.7.1 <someone@example.net>: Relay access denied

A refusal is the correct answer. If the server accepts the message, the relay restrictions are wrong, and you have an open relay: stop and fix it before the address is used to send spam. The same test against port 587 without a password must also fail, with an authentication error.

Sending from the website

A contact form should not use the mail() function of PHP through the local queue if you can avoid it: a message sent that way has no authentication and is easy to mark as spam. Send through the submission port with authentication instead. msmtp is a small SMTP client that PHP can call in place of sendmail:

sudo apt install msmtp

Install msmtp alone, not the msmtp-mta package: that one replaces Postfix as the local mail transport agent, and this server needs Postfix.

sudo mkdir -p /etc/msmtp

sudo vi /etc/msmtp/example.conf

defaults auth on tls on tls_trust_file /etc/ssl/certs/ca-certificates.crt account example host mail.example.com port 587 user you password a-long-random-password from you@example.com account default : example

sudo chown root:example /etc/msmtp/example.conf

sudo chmod 640 /etc/msmtp/example.conf

The login is you, the system user from the mailbox step, not the full address: Dovecot authenticates system users by user name. The sender address stays you@example.com.

The file belongs to root and to the group of the site, so the PHP process can read the password and other users cannot. Then point the pool of the site at it. sendmail_path is a system setting, so it goes into the pool with php_admin_value:

sudo vi /etc/php/8.3/fpm/pool.d/example.com.conf

php_admin_value[sendmail_path] = /usr/bin/msmtp -t --file=/etc/msmtp/example.conf

sudo systemctl reload php8.3-fpm

A form can now send through your own server, with TLS and a real sender address. Never put the SMTP password inside the web root; a leaked .env file is one of the threats from lesson 7.

Testing with swaks and headers

Bash and a mail client are not enough to debug mail. swaks is a single command that speaks SMTP and prints the whole conversation:

sudo apt install swaks

swaks --to you@gmail.com --from you@example.com --server mail.example.com:587 --auth LOGIN --auth-user you --auth-password "your-password" --tls

<- 250 2.0.0 Ok: queued as 4Xy2Z -> QUIT <- 221 2.0.0 Bye === Connection closed with remote host.

A queued message is only half the proof. At the receiving mailbox, open the full headers and look for the authentication results:

Authentication-Results: mx.google.com; spf=pass (google.com: domain of you@example.com designates 203.0.113.10 as permitted sender) smtp.mailfrom=you@example.com; dkim=pass header.i=@example.com header.s=mail header.b=AbCdEfGh; dmarc=pass (p=NONE sp=NONE) header.from=example.com

Three passes mean the records are right. spf=fail means an address that is not in the record sends your mail; dkim=fail means the selector, the key or the signature is wrong. dmarc=fail appears only when neither SPF nor DKIM gives an aligned pass: both failed, or both passed for a domain other than the one in the From header, which is what a relay that rewrites the envelope sender produces. Read the first failure, not the last.

If the message lands in spam with three passes, the IP reputation is the usual cause. Check the address against the public blocklists at check.spamhaus.org and at mxtoolbox.com. A listed address must be cleaned up and then requested for removal; while it is listed, most receivers will refuse or quarantine your mail.

MTA-STS and reports

Two optional records make mail more trustworthy, and both are worth adding once the basics pass.

MTA-STS publishes a policy that says mail to your domain must use TLS. It needs a TXT record and a small HTTPS page:

_mta-sts.example.com. 3600 IN TXT "v=STSv1; id=20261106T000000"

The policy itself is served at https://mta-sts.example.com/.well-known/mta-sts.txt:

version: STSv1 mode: testing mx: mail.example.com max_age: 86400

Start with mode: testing and read the reports; switch to enforce when every sender you care about still delivers. TLS-RPT is the report channel for TLS failures:

_smtp._tls.example.com. 3600 IN TXT "v=TLSRPTv1; rua=mailto:tlsrpt@example.com"

Both records are easy to add and easy to forget. Put them in the notes with the MX and SPF values, and check them again after any change to the mail server.

Quiz

1 What does the MX record of a domain say?

  1. Which servers may send mail for the domain
  2. Where mail for the domain is delivered
  3. Which key signs outgoing messages
  4. What to do when authentication fails
Answer

b) MX names the destination mail servers, with a priority. SPF is the list of allowed senders, DKIM the signature, DMARC the policy.

2 Your SPF record is v=spf1 mx -all. What does -all mean?

  1. Every sender passes
  2. Any server not listed in the MX records is a hard failure
  3. The record is a test only
  4. Mail is forwarded to all
Answer

b) The minus sign is a hard fail. Use ~all during a migration, when some senders are not in the list yet.

3 Why does a new mail server need a matching PTR record?

  1. Receivers check that the address belongs to the name the server announces
  2. It speeds up IMAP
  3. It signs DKIM
  4. It encrypts port 25
Answer

a) A missing or mismatched reverse name is a strong spam signal. The PTR is set by the address owner, which is the VPS provider, not by your DNS panel.

4 nc -vz alt1.gmail-smtp-in.l.google.com 25 times out. What does it mean?

  1. Gmail is down
  2. Outgoing port 25 is blocked, by the provider or the network
  3. The certificate expired
  4. Postfix is not installed
Answer

b) A timeout on outbound 25 is the classic block. Ask the provider to open it, or send through a relay on port 587.

5 Which Postfix setting stops the server from being an open relay?

  1. mydestination listing every domain
  2. smtpd_relay_restrictions allowing only authenticated users and local mail
  3. Starting Postfix on port 25
  4. Adding a DKIM key
Answer

b) The relay restrictions decide who may send mail to outside domains. The test from another machine must answer Relay access denied.

6 A message arrives with spf=pass, dkim=fail and dmarc=fail. Where do you look first?

  1. The SPF record
  2. The DKIM selector, key and DNS record
  3. The PTR record
  4. The MTA-STS policy
Answer

b) SPF passed, but its pass was not aligned with the From domain, so DMARC still failed: the envelope sender belongs to another domain, for example a relay or a mailing tool, while the From header says your domain. DKIM signs the From domain itself, so it is the check to fix: compare the selector in the signature with the TXT record and the key on disk.

7 Why send website mail through authenticated SMTP instead of the local mail() queue?

  1. It is faster
  2. The message carries authentication and a real sender, so receivers trust it more
  3. It needs no DNS records
  4. It encrypts the website
Answer

b) An authenticated submission on port 587 with TLS is what the receiving side expects from a real mail domain. It also keeps the SMTP password out of the web application code.

Article author: Arthur Isaev