Summary

  • FTP's STRU P represented discontinuous files as independent indexed pages; Page Index described logical placement, not transmission order.
  • A missing page-table entry was simply not sent, while a present page containing zeros remained data. RFC 959 explicitly warned that a hole was not the same as a page of zeros.
  • Page structure arose chiefly from TOPS-20 and NLS needs. RFC 1123 later discouraged general implementation but required the standard format, rather than a private substitute, when random-access or holey files genuinely needed it.

The page that never crossed the data connection

Suppose a receiver sees one page header, then another whose logical index skips a position. A packet-oriented diagnosis begins by looking for loss. A byte-stream reconstruction may fill the gap with zeros. Both can be wrong.

RFC 959 defined page structure for discontinuous files, also called random-access or “holey” files. Its Page Index field was the logical page number within the file, explicitly not the sequence number of transmission. A jump in index could therefore describe the source file rather than damage in transit.

The appendix makes the distinction unusually blunt. Empty pages in the TOPS-20 page map were not transmitted, and “a hole is NOT the same as a page of zeros.” One is structural absence. The other is an occupied page whose values happen to be zero. Replacing the first with the second changes the file even if an application later reads similar values from both on one system.

This is the article's central historical fact: FTP once needed a wire representation for something that was meaningful precisely because no data page existed there.

Three structures, three claims about a file

FTP separated representation type, file structure and transmission mode. They interacted, but they answered different questions. TYPE described logical representation. STRU described internal organization. MODE described how transfer bytes were framed on the data connection.

The protocol defined three structures. File structure treated the file as one continuous sequence of data bytes and served as the default. Record structure treated it as sequential records. Page structure treated it as independent indexed pages.

That menu came from heterogeneous storage, not taxonomic enthusiasm. RFC 959 contrasted an IBM mainframe that naturally stored source code in fixed-length records with a TOPS-20 host that commonly presented characters as a stream divided into lines. If unlike systems were to exchange useful files, each side needed to know what assumptions the other had made.

Transformations were allowed, but reversibility mattered. A file-oriented host receiving record structure could convert records into a locally useful form, yet it had to be able to reverse that conversion when the file was retrieved again with record structure. Later, RFC 1123 retained the same principle: transformations between record and file structure should be invertible as far as practical while remaining useful on the target host.

Bytes alone could not state which transformation had been applied. The active parameter tuple was part of the evidence.

An index was placement, not arrival order

Every page in STRU P carried a header. The minimum header contained four logical fields: Header Length, Page Index, Data Length and Page Type. Each field occupied one logical byte, whose bit width came from TYPE rather than from an assumption about the receiver's machine word.

The Page Index gave a destination in the file map. It did not say “this was the seventh unit sent.” Pages could be interpreted by their declared placement even when the order on the connection differed. Arrival order and file order were separate facts.

Data Length could be zero. Page Type then prevented that number from carrying the whole meaning. A Last Page used a four-field header and zero data to close a paged transfer. A Simple Page represented ordinary page data. A Descriptor Page carried descriptive information for the file as a whole. An Access Controlled Page added a field for page-level control information.

So “zero length” was not a universal synonym for “nothing.” A zero-data terminal header had protocol meaning. A descriptor page could concern the whole file. An omitted index could represent a hole. Interpretation depended on the type and map position, not a single count.

Why TOPS-20 needed the distinction

The page-structure appendix in RFC 765 and RFC 959 states its provenance. The need arose principally from efficient file transfer between TOPS-20 systems, particularly files used by NLS.

A TOPS-20 disk file consisted of a pathname, a page table, a possibly empty set of pages and attributes. The page table could have up to 2^18 entries. An entry could be empty or point to a page; occupied entries could carry page-specific access bits. The file's attributes included timestamps, byte size, an end-of-file pointer, counters and backup information.

Neither the page map nor the ending had to resemble a compact byte array. Empty table entries could lie between occupied ones. The end-of-file pointer was a number and did not have to point to the last datum. NLS files used both of these unusual cases.

For that environment, the standards documented TYPE L 36, STRU P, MODE S. The 36-bit logical type matched a TOPS-20 word. The page header carried map index, data length, page kind and, for data pages, access bits. A separate descriptor page could carry file-wide information.

This was not a claim that every host should become TOPS-20. It was an attempt to keep one host's meaningful structure from being flattened merely because another host organized storage differently.

A hole, a zero page and shortened data

The appendix distinguishes three cases that a careless normalizer can merge.

First, an empty page-map entry is a hole and is omitted from transfer. Second, an occupied page may contain zero-valued words; it remains a page. Third, trailing zeros inside an occupied TOPS-20 page could be discarded by reducing Data Length below the normal 512 words.

All three can reduce the number of nonzero values on the wire. They do not make the same statement. The hole says there was no page at that map position. The explicit page says storage existed there. The shortened page says a present page had a smaller transmitted data extent under the documented rule.

This is why a checksum of concatenated page payloads cannot by itself prove faithful reconstruction. Evidence must include indexes, types, lengths, descriptor fields, the end marker and the transformation policy. Otherwise different file maps can yield deceptively similar transmitted content.

The standards do not say that every receiving filesystem exposes holes identically or that an omitted slot always reads as zeros. Nor do they guarantee that page-level access bits can be enforced by an unlike target. Those are local storage and policy questions. FTP supplied the distinctions; an implementation still had to map them honestly.

Not recommended, but not yours to reinvent

By 1989, RFC 1123 made a revealing judgment. Implementation of page structure was “not recommended” in general. The mechanism was specialized and broad interoperability depended on a smaller required core.

But the document did not license private reinvention. If a host genuinely needed FTP for random-access or holey files, it had to use the defined page-structure format rather than create a private FTP representation.

That is a sharper rule than simple deprecation. General implementations could decline the feature. Specialized implementations could retain it. What they could not do was reuse FTP while assigning incompatible private meaning to the same problem.

The minimum host requirements reinforced the boundary. File structure was required. Record structure was required only for hosts whose filesystems supported it, though another host could accept STRU R by storing the stream literally. Page support remained optional. The command vocabulary and each argument's availability were not the same promise.

A registry row is not page support

RFC 5797 later placed STRU among FTP's mandatory base commands. The IANA FTP Commands and Extensions registry records it as File Structure, a parameter-setting command with mandatory conformance and RFC 959 as its reference.

That row coordinates the verb and its role. It does not prove that a particular server accepts P, that a client can reconstruct a TOPS-20 page map, that page controls will survive, or that anyone deploys the facility today. A mandatory command can still have optional or system-dependent parameter capabilities.

The distinction mirrors the content of the file. Presence of a name in a registry is not presence of an implementation, just as absence of a transmitted page was not presence of a zero page.

What an empty berth still tells us

Modern data systems repeatedly compress “not present,” “zero,” “unknown,” “redacted” and “not applicable” into one convenient value. The resulting file may parse and the checksum may match the normalized output, while the original structure has disappeared.

FTP's page structure was specialized, tied to machines and applications that no longer define ordinary Internet file transfer. Its design discipline remains current. Name absence explicitly. Carry placement separately from arrival order. Preserve metadata needed to reconstruct meaning. State which transformations are reversible. Do not infer content from silence unless the format grants that inference.

The missing page was useful information. Filling it was not recovery. It was a different file.

Sources