DANE and MTA-STS: Securing the Mail Transport Layer

Your email travels encrypted. Most of the time. SMTP supports STARTTLS, which upgrades a plaintext connection to TLS before the message transfers. The problem is that "supports" is doing all the work there. Without additional verification, a network attacker can strip the STARTTLS negotiation and force a plaintext connection. Your MTA happily continues, because opportunistic TLS means "encrypt if possible, otherwise continue anyway."

DANE and MTA-STS both address this. They use different mechanisms, have different requirements, and have different levels of adoption. You need to understand both.

The Core Problem: Opportunistic TLS Isn't Enough

When a sending MTA connects to your mail server:

It establishes a TCP connection on port 25.
The receiving server advertises STARTTLS in its EHLO response.
The sender upgrades to TLS.
The message transfers.

Step 3 is the problem. Nothing in this flow authenticates the TLS certificate. A man-in-the-middle can:

Strip the STARTTLS advertisement (downgrade attack), forcing plaintext
Present a forged certificate the sender accepts because it doesn't verify

The Kaminsky cache poisoning attack (2008) showed how easy it was to forge DNS responses without cryptographic verification. DNSSEC was the fix at the DNS layer. DANE extends that fix to TLS certificate binding: TLSA records protected by DNSSEC bind the expected certificate directly to DNS, so an attacker who can't forge signed DNS records can't swap in a fraudulent certificate. MTA-STS takes a different approach, publishing an enforcement policy over HTTPS that tells senders: require TLS, verify the certificate, and fail if it doesn't match.

DANE: DNS-Based Authentication of Named Entities

DANE uses TLSA records to publish the expected TLS certificate (or certificate authority) directly in DNS. If the zone is signed with DNSSEC, the TLSA record is authenticated and tamper-evident.

TLSA Record Structure

_25._tcp.mail.example.com.   IN   TLSA   3 1 1   <certificate-hash>

Breaking this down:

_25._tcp.mail.example.com. — the service port (25), transport protocol (tcp), and the MX hostname
3 1 1 — three parameters: usage, selector, matching type
The final field is the hash of the certificate data

Usage field:

0 — PKIX-TA: CA constraint (certificate must chain to this CA)
1 — PKIX-EE: End entity constraint (must match this cert, must chain to trusted CA)
2 — DANE-TA: Trust anchor (this is a trusted CA, no PKIX required)
3 — DANE-EE: End entity only (must match this cert, no PKIX chain check)

For mail servers, 3 1 1 is the most common choice: DANE-EE, selector is SubjectPublicKeyInfo, matching type is SHA-256. This pins the public key hash, which survives certificate renewal as long as you keep the same key pair.

Selector field:

0 — Full certificate
1 — SubjectPublicKeyInfo only (the public key)

Selector 1 is better for renewal scenarios — you pin the key, not the whole cert.

Matching type:

0 — Full content (not hashed)
1 — SHA-256
2 — SHA-512

Generating the TLSA Record

# Generate from an existing certificate file
openssl x509 -in /etc/ssl/mail.example.com.crt -noout -pubkey | \
  openssl pkey -pubin -outform DER | \
  openssl dgst -sha256 -binary | \
  xxd -p -c 32

# Or use tlsa-rdata directly if available
# Or use the online generator at ssl-tools.net

The hash output goes into your TLSA record.

Publishing the TLSA Record

Your MX hostname is mail.example.com. The TLSA record goes at:

_25._tcp.mail.example.com.   300   IN   TLSA   3 1 1   abc123...

For DNSSEC to protect this, your zone must be signed. If it isn't, DANE provides no security guarantee. TLSA records without DNSSEC can be spoofed just like any other DNS record.

Who Uses DANE?

Primarily European mail operators, financial institutions, and government mail systems in NL, DE, SE, DK. Many large German mail providers (Posteo, mailbox.org) have full DANE support. Google and Microsoft's consumer mail services do not currently validate DANE. If you're running your own mail infrastructure and want the strongest possible transport security, DANE is the right answer — but verify your SMTP neighbors support it before depending on it.

Check if a domain publishes TLSA records:

dig TLSA _25._tcp.mail.example.com

MTA-STS: Mail Transfer Agent Strict Transport Security

MTA-STS is the HTTPS-based alternative. No DNSSEC required. Broader adoption. The tradeoff: the policy is cached, so there's a window where an attacker could serve a stale policy. DANE with DNSSEC is theoretically stronger; MTA-STS is more practical for most deployments.

How MTA-STS Works

The sending MTA checks for a _mta-sts TXT record on your domain.
If found, it fetches the policy file from https://mta-sts.yourdomain.com/.well-known/mta-sts.txt.
The policy specifies which mail servers are allowed and what TLS behavior to enforce.
The sender enforces the policy for future connections.

The DNS Record

_mta-sts.example.com.   IN   TXT   "v=STSv1; id=20240301120000;"

The id field is a version identifier. Change it whenever you update the policy file. The sending MTA compares this id to its cached version — if they differ, it re-fetches the policy. Use a timestamp (YYYYMMDDHHmmss) as a convention for easy ordering.

The Policy File

Host a text file at https://mta-sts.example.com/.well-known/mta-sts.txt.

The subdomain mta-sts.example.com needs a valid TLS certificate (the policy delivery itself is authenticated via HTTPS).

Example policy file:

version: STSv1
mode: enforce
mx: mail.example.com
mx: *.example.com
max_age: 86400

mode options:

testing — report failures, don't enforce. Use this first.
enforce — reject connections that don't comply.
none — disable the policy (used to signal decommissioning).

mx: lists the allowed mail server hostnames. The connecting MTA verifies the server's certificate matches one of these patterns.

max_age: how long (in seconds) the policy should be cached. 86400 = one day. For production, 604800 (one week) is common.

Deployment Sequence

Set up mta-sts.example.com with a valid TLS certificate.
Deploy the policy file at /.well-known/mta-sts.txt.
Set mode to testing.
Publish the _mta-sts TXT record.
Configure TLS-RPT to receive reports (see below).
Monitor reports for failures.
Switch mode to enforce.
Update the id in the TXT record.

TLS-RPT: Transport Layer Security Reporting

TLS-RPT is the reporting mechanism for both DANE and MTA-STS failures. Sending servers that encounter TLS issues will send JSON reports to the address you specify.

_smtp._tls.example.com.   IN   TXT   "v=TLSRPTv1; rua=mailto:tls-reports@example.com"

Or send to an HTTPS endpoint:

_smtp._tls.example.com.   IN   TXT   "v=TLSRPTv1; rua=https://reporting.example.com/tls"

The reports tell you which senders had TLS failures connecting to your mail servers, what errors they got, and how many connections were affected. Without TLS-RPT, you're deploying MTA-STS or DANE blind.

Verifying Your Setup

# Check the _mta-sts DNS record
dig TXT _mta-sts.example.com +short

# Check the TLS-RPT record
dig TXT _smtp._tls.example.com +short

# Fetch the policy file manually
curl https://mta-sts.example.com/.well-known/mta-sts.txt

# Check DANE (TLSA) record
dig TLSA _25._tcp.mail.example.com

Key Takeaways

Opportunistic STARTTLS doesn't verify the certificate. Downgrade attacks are real.
DANE uses TLSA records + DNSSEC to bind certificates to DNS. Strong security, requires signed zones, limited to operators who support it.
MTA-STS uses a DNS TXT record + HTTPS policy file. No DNSSEC required, broader support.
Start MTA-STS in testing mode, use TLS-RPT to monitor, then switch to enforce.
TLS-RPT (_smtp._tls TXT record) should be deployed alongside both DANE and MTA-STS.
DANE record format: _25._tcp.<mx-hostname>. IN TLSA 3 1 1 <sha256-of-pubkey>

Up Next

Lesson 03 covers SPF: the TXT record that tells the internet which IP addresses are allowed to send email as your domain — and why the one-character difference between ~all and -all matters for deliverability.