Summary

  • With STRU R and MODE S, FTP placed two-byte control codes inside the data stream: an all-ones escape followed by 1 meant EOR, 2 meant EOF and 3 meant both together.
  • A literal all-ones data byte had to be repeated. The receiver therefore needed negotiated context and parser state to distinguish doubled data from a structural boundary.
  • The design separated portable record meaning from host storage and TCP delivery chunks. Later host requirements reduced mandatory record support, but still required honest, preferably invertible treatment when it was accepted.

The pair that decoded to one

Imagine a receiver has consumed ordinary data and now sees a byte whose bits are all one. It cannot yet deliver that byte to the file. It must retain a small piece of state and inspect what follows.

If the next byte is also all ones, the pair represents one literal data byte. If the next value is 1, the pair closes a record. Value 2 closes the file. Value 3 says that the same point is both the final record boundary and the end of the file.

This compact grammar, preserved by RFC 765 and RFC 959, made one octet do two jobs without leaving an ambiguous literal value behind. The first all-ones byte was not intrinsically control. Its meaning was completed by its neighbor and by the transfer parameters already negotiated on the control connection.

The doubling rule is the revealing detail. An in-band sentinel claims a value from the data alphabet. To give that value back to the file, the sender quotes it with another copy. The receiver reverses the operation. Raw transfer length and reconstructed data length can therefore differ even when nothing has been compressed or lost.

A stream was not a record service

FTP described structure and transmission mode as separate choices. STRU F treated a file as a continuous sequence of data bytes. STRU R said that the file consisted of sequential records. MODE S selected a largely unframed byte stream for transfer.

The combination mattered. A stream connection supplied order and delivery, but it did not say where one source record ended. FTP inserted that missing statement itself. For record-structured transfers, every EOR was explicit, including the final EOR.

The standards explain why a local convention could not simply ride along unchanged. A record-oriented host might store a count field or some other private marker. Another host would not know what that field meant. Before transfer, the sender translated its local record notation into FTP's negotiated representation; after transfer, the receiver translated back into whatever form its own system could use.

That was an interoperability contract, not a promise that host file systems were alike. The shared part was the boundary declaration. Local storage remained local.

Four continuations after the escape

The STREAM grammar was deliberately small:

  • escape plus 1: end of record;
  • escape plus 2: end of file;
  • escape plus 3: end of record and end of file at the same position;
  • escape plus another escape: one literal all-ones data byte.

The combined code avoided inventing an empty record merely to say that the last record and file ended together. The literal form ensured that every possible transfer byte could still appear in the reconstructed data.

An implementation could not safely scan for the sentinel and immediately declare a boundary. Nor could it preserve both bytes of a doubled literal pair. Correct decoding required one step of look-ahead, exact unquoting and a distinction between transfer bytes and file bytes.

It also required state across local reads. RFC 9293 defines TCP as an ordered byte-stream service, not an application-record service. The escape may be the final byte returned by one receive operation while its discriminator arrives in the next. That split changes neither the FTP code nor the record map. Treating receive calls or TCP segments as records would add boundaries the sender never declared.

The same stream byte was data under another structure

The sharpest evidence of negotiated meaning appears in FTP's rule for file structure. With STRU F in STREAM mode, every byte is data and EOF is indicated by closing the data connection. The all-ones byte has no escape role there.

With STRU R, the parser activates the two-byte grammar. A capture of the data connection without the matching STRU and MODE state is consequently incomplete. The octets alone do not settle whether a doubled pair reconstructs one byte or remains two, or whether a particular pair ends a record.

Connection closure also changes role. Under file structure it supplies the ordinary EOF signal. Under record structure, EOR markers remain explicit and the STREAM representation can state EOF inside the stream. A monitor that records only “connection closed normally” can miss whether the final structural declaration arrived.

Other modes moved the boundary out of the payload grammar

FTP did not bind record meaning to this one escape scheme. In BLOCK mode, each data block carried a count and descriptor flags. One descriptor bit said the block ended at EOR; another said it ended at EOF; both could be set together. Other bits identified suspect data or a restart marker.

COMPRESSED mode likewise used descriptor forms for EOR and EOF while adding encodings for replicated data and filler. The boundary semantics survived, but their on-wire representation depended on MODE.

This is why “same payload” is too weak a statement for a structured transfer. Two captures can contain the same reconstructed data bytes while using different boundary encodings. Conversely, the same raw octets can have different interpretations under different parameter tuples. Reliable evidence joins control-plane negotiation, data-stream bytes and decoded structure.

A line ending was not automatically a record ending

FTP also kept end-of-line separate from end-of-record. ASCII text without record structure could use CRLF to divide printing lines; EBCDIC text could use NL. A record-structured file carried explicit EOR markers instead.

Visible text may make those boundaries coincide often enough to tempt a shortcut. The shortcut becomes destructive as soon as a record contains an embedded line ending, spans several lines or has no textual representation at all. Replacing EOR with newline guesses changes the source structure while producing a file that still looks readable.

RFC 959 required ASCII and EBCDIC text implementations to accept record structure. It also emphasized useful, invertible transformations between record-oriented and file-oriented hosts. When an identical pair of systems moved non-record text, using a binary representation could avoid needless transformation. Each rule reflects the same discipline: do not infer structure from presentation when the protocol can state it.

The later minimum became narrower

RFC 1123 adjusted the implementation boundary. Record structure was required only for hosts whose file systems supported it. A host without native records could still accept STRU R and record the byte stream literally.

That option needs careful wording. Literal preservation can protect the encoded evidence for later retrieval, but it does not prove that local applications received a native record map. A system can preserve transfer syntax, reconstruct records, flatten records, or reject the request. Those outcomes should not share one undifferentiated success claim.

The host requirements retained the preference for transformations that were useful and invertible where possible. If retrieval under the same structure cannot reproduce the original records, the receiver should not imply full semantic preservation merely because all raw bytes reached storage.

A registry entry was not a parser test

RFC 5797 keeps STRU and MODE among FTP's mandatory base commands. The IANA FTP Commands and Extensions registry preserves their parameter-setting role and RFC 959 reference.

That registry coordinates vocabulary. It does not prove that a server accepts the R argument, supports every mode combination, retains escape state across reads, handles every legal control pair, reconstructs records, or preserves them through a storage round trip.

The distinction mirrors the byte itself. A known name creates a place for meaning. Only the surrounding state and observed behavior establish which meaning was realized.

Evidence must include the decoder

A defensible transfer record therefore contains more than a file hash. It retains TYPE, STRU, MODE, the raw stream hash, the decoded data hash, raw and decoded lengths, every EOR and EOF position, the final parser state, connection-close status and the target reconstruction result.

Canaries should place a literal doubled sentinel before and after each legal control pair, split the pair across receive calls and compare STREAM and BLOCK encodings of the same record map. A dangling escape at close, an unknown discriminator, a missing final EOR or a changed parameter set without parser reset is evidence of a failed interpretation even if the connection itself completed cleanly.

The byte had to appear twice because the protocol refused to sacrifice it. The record boundary lived in the stream, but only as a reversible agreement between sender and receiver.

Sources