Summary

  • RFC 3516 let an IMAP server remove a MIME content-transfer-encoding and return the decoded body section through BINARY, including NUL-bearing data in literal8. The result was a decoded view, not the serialized message that entered the mailbox.
  • Decoded sizes and partial offsets belonged to that view. Storage form, FETCH BODY, CRLF handling, encoded headers and cryptographic operations retained separate byte boundaries.

Base64 solved an old transport problem by making arbitrary data travel through systems that could not carry arbitrary octets. It also made a new retrieval problem: a client on a constrained link might download the larger encoded representation only to reverse the encoding locally. For slow radio and streaming media, RFC 3516 observed, that overhead could matter.

Published in April 2003, the IMAP4 Binary Content Extension moved the decoding step. A server advertising BINARY could accept FETCH BINARY and remove the selected body section's MIME content-transfer-encoding before transmission. The extension saved bandwidth and client work. It did not make the returned bytes the original message.

That distinction starts with MIME. Every IMAP body section has a content-transfer-encoding, explicitly or by an implicit 7bit label. The CTE tells a decoder both which transformation to reverse and the domain of the decoded data. Base64 and quoted-printable perform visible transformations. Other CTE values may use an identity transformation while still declaring a domain that base IMAP cannot carry.

RFC 3516 therefore described server processing as two logical steps. First, perform the CTE-related decoding. Second, determine the domain of the decoded result. These are not interchangeable decisions. Knowing that an algorithm produced octets does not yet tell the server which IMAP protocol element can represent them.

The extension introduced literal8 for the wider domain. Its syntax begins with a tilde and a counted literal, then carries arbitrary octets. Unlike the base literal, it can carry a NUL. When decoded data are only 8-bit and contain no NUL, the server should use an ordinary string instead. That choice lets the client infer the narrower domain without scanning the entire stream for zero octets.

The framing is exact, but its authority is bounded. A literal8 count proves how many octets follow in that IMAP element. It does not reveal whether the server stored those octets directly, reconstructed them from base64, normalized a textual representation, or generated them from another compliant internal form.

BINARY.PEEK added another boundary. Like its BODY.PEEK counterpart, it requests the data without implicitly setting the message's \\Seen flag. The difference concerns mailbox state, not byte provenance. A response can be correct and leave \\Seen untouched while still being a server-generated decoded view.

BINARY.SIZE reported the decoded section size: the number of octets expected from the corresponding FETCH BINARY. It did not report the size of the encoded body in the message, the mailbox storage allocation, the complete message, or a downstream rendered file. RFC 3516 warned that calculating the value could be expensive because some servers would have to decode the section merely to count it.

Partial retrieval made the coordinate system especially important. A partial FETCH BINARY uses offsets into decoded data. An offset into the base64 text or a FETCH BODY result does not identify the same place. A client that resumes a transfer using a counter from the wrong representation may receive a syntactically valid fragment from the wrong logical position.

The protocol refused to guess when the decoder lacked authority. If a server did not know a section's CTE, both BINARY and BINARY.SIZE had to fail with a NO response containing UNKNOWN-CTE. That response did not prove the message corrupt. It proved that this server could not perform the requested transformation under the named rule.

RFC 4466 later revised the framework for IMAP response codes, and RFC 9051 incorporated the binary mechanisms into IMAP4rev2. Those changes affect how modern implementations frame capabilities and errors. They do not collapse encoded message, decoded section and stored representation into one byte sequence.

Headers occupied a different layer. RFC 2047 encoded words allow non-ASCII text inside certain message header fields. Their encoding is not the content-transfer-encoding applied to MIME bodies. RFC 3516 therefore prohibited a server from converting encoded header text in response to BINARY FETCH or APPEND. “Decode the body section” was not authority to rewrite every encoded-looking sequence in the message.

Text added a controlled transformation of its own. Servers had to transmit line-oriented textual sections using IMAP CRLF line termination regardless of the underlying storage form. That requirement supported protocol consistency, but it also meant that a returned textual view could not be treated casually as forensic proof of the server's local newline bytes.

The document made the storage boundary explicit. A server could store binary message content without a transfer encoding. Nevertheless, its BODYSTRUCTURE response had to describe the message as though binary sections used a CTE acceptable to base IMAP, and FETCH BODY had to return the format promised by that structure. The public protocol view was a contract, not a memory dump.

APPEND approached the boundary from the opposite direction. A client could submit NUL-bearing data with literal8. A server lacking binary storage support had to reject the request with UNKNOWN-CTE. A supporting server could alter the appended data's CTE, but the transformation was forbidden from losing payload data.

“No loss of data” still did not mean “same serialized message.” Base64, quoted-printable and identity forms can encode equivalent payloads in different octet sequences. Header folding, line termination and CTE labels are part of the serialized representation. A payload-preserving change can therefore invalidate a digest or signature calculated over that representation.

RFC 3516 said so directly: gratuitous encoding changes would render most cryptographic operations performed on the message useless. The warning is not a contradiction. One promise concerns recovery of content; the other depends on exact bytes and covered headers. An operator who records only “lossless conversion” has not recorded the input to the cryptographic check.

Nor did the extension remove decoding responsibility from clients. It was an optimization for particular conditions, not a new universal storage format. Supporting clients were still advised to implement basic CTE decoding. Capability discovery and a successful response improved efficiency; they did not justify an architecture unable to handle the ordinary encoded form.

Heng Lu's distinction between symbolic and operational reality helps organize the evidence. BINARY in CAPABILITY is a symbolic claim of supported behavior. BINARY.SIZE is a promise about one decoded response. The counted literal is an observable sequence on one IMAP connection. The mailbox's stored bytes, the RFC 5322 message, the MIME payload, the displayed media and the input to a signature verifier occupy related but different layers.

A useful receipt chain therefore begins with the message identifier and immutable retrieval context. It records BODYSTRUCTURE, selected section, original CTE, command form, \\Seen behavior, response code, decoded-domain choice, literal length, partial offset, complete returned bytes and their hash. If original-message identity matters, it separately preserves a raw message retrieval and its hash. If a signature matters, it records the exact canonicalization and byte range the verifier consumed.

Without those labels, two honest statements can appear to conflict. The server can truthfully say it returned the decoded body, while an investigator truthfully says the downloaded file does not hash to the stored message. RFC 3516 did not ask either side to be wrong. It defined a transformation between two evidence surfaces.

The extension's historical achievement was practical: binary content no longer had to pay the full transfer-encoding cost on every IMAP hop. Its deeper lesson was restraint. The server decoded the body. The client obtained useful octets. The original message still needed its own retrieval, its own hash and its own proof.

Sources