RFC 5321: SMTP Protocol Deep Dive

SMTP Architecture

Simple Mail Transfer Protocol (RFC 5321, 2008 — updates RFC 2821 from 2001) defines how email is transmitted between servers. The architecture involves three agent types:

MUA (Mail User Agent) — the email client (Outlook, Thunderbird, mobile mail app). Users write and send email here.
MSA (Mail Submission Agent) — accepts email from MUAs on port 587 (or 465 for SMTPS). Applies policy, adds headers, forwards to MTA.
MTA (Mail Transfer Agent) — routes email between servers on port 25. MTAs are the backbone of email delivery (Postfix, Exim, sendmail).
MDA (Mail Delivery Agent) — final delivery to mailbox (Dovecot).

MUA → MSA (port 587) → MTA → MTA → MDA → Mailbox
      (submission)     (relay)        (delivery)

The SMTP Session

An SMTP session is a TCP connection with a structured dialog. The server speaks first, advertising capabilities. The client drives the transaction.

Server: 220 mail.example.com ESMTP Postfix ready
Client: EHLO sender.example.net
Server: 250-mail.example.com Hello
        250-SIZE 52428800
        250-STARTTLS
        250-AUTH LOGIN PLAIN XOAUTH2
        250 SMTPUTF8

After EHLO, the server lists its ESMTP extensions — capabilities beyond the base RFC 5321 protocol. The client may then use these extensions in subsequent commands.

Mail Transaction Commands

EHLO (Extended HELO)

EHLO (RFC 1869) replaced HELO as the standard greeting for ESMTP. Always use EHLO — it advertises your domain and triggers extension listing. If the server returns 500 (unknown command), fall back to HELO.

MAIL FROM

Declares the envelope sender (the return path for bounces):

MAIL FROM:<[email protected]> SIZE=12345
250 Ok

This is the envelope From, distinct from the From: header visible in the email body. SPF checks are performed against the envelope From domain, not the header From.

RCPT TO

Declares each recipient (one command per recipient):

RCPT TO:<[email protected]>
250 Ok
RCPT TO:<[email protected]>
550 5.1.1 User unknown

A 550 at this stage is a hard bounce — the recipient doesn't exist. A 450 is a soft bounce — try again later. Critically, RCPT TO responses are per-recipient, so a message can be accepted for some recipients and rejected for others in the same transaction.

DATA

Initiates message body transmission. The server responds 354 Start input. The client sends the message (RFC 5322 headers + body) terminated by a line containing only a period (.):

DATA
354 End data with <CR><LF>.<CR><LF>
From: [email protected]
To: [email protected]
Subject: Hello

Message body here.
.
250 Ok: queued as 12345

Reply Code System

SMTP reply codes are three-digit numbers with structured semantics:

First digit	Class
2xx	Positive Completion — command succeeded
3xx	Positive Intermediate — continue (used for `DATA`)
4xx	Transient Negative — temporary failure, retry later
5xx	Permanent Negative — hard failure, do not retry

The second and third digits add detail. 250 Ok, 421 Service not available, 550 Mailbox unavailable, 552 Storage exceeded.

RFC 3463 added Enhanced Status Codes (X.Y.Z suffixes) for more precise machine-readable diagnostics:

550 5.1.1 The email account that you tried to reach does not exist.
^   ^ enhanced status code
|   5 = Permanent failure
|     1 = Addressing status
|       1 = Bad destination mailbox address

Extensions (ESMTP)

Extension	EHLO keyword	Function
Message size limit	`SIZE`	Reject oversized messages early
UTF-8 mailboxes	`SMTPUTF8`	Support international addresses
Pipelining	`PIPELINING`	Send multiple commands without waiting
8-bit MIME	`8BITMIME`	Send 8-bit content without encoding
Binary MIME	`BINARYMIME`	Skip line-length limits
STARTTLS	`STARTTLS`	Upgrade to TLS in-session
Auth	`AUTH`	SMTP authentication

Relay and Routing

Email routing uses DNS MX records. When delivering to [email protected], the sending MTA queries:

dig MX example.com
# example.com. 300 IN MX 10 mail1.example.com.
# example.com. 300 IN MX 20 mail2.example.com.  (backup)

Lower priority number = higher preference. The MTA tries MX records in priority order, falling back to higher numbers on failure.

Security: STARTTLS and Auth

STARTTLS (RFC 3207) upgrades a plain TCP SMTP connection to TLS:

Client: STARTTLS
Server: 220 Ready to start TLS
(TLS handshake...)
Client: EHLO sender.example.net  # Re-EHLO after TLS

After STARTTLS, the server re-advertises AUTH options — credentials are only transmitted inside the TLS tunnel. SMTP AUTH supports mechanisms like LOGIN, PLAIN, and XOAUTH2 for Gmail/Microsoft 365.

STARTTLS does not authenticate the server — always verify the TLS certificate. MTA-STS policies mandate certificate validation for server-to-server SMTP, preventing downgrade attacks.