Summary

  • RFC 2070 separated a concrete HTML resource's external byte encoding from HTML's fixed abstract UCS document character set; MIME and HTTP charset named the decoder-side mapping.
  • Numeric character references resolved in the fixed document character set, so they kept the same character identity regardless of the external encoding that carried the markup.
  • A successful decode was only one receipt. SGML parsing, language and bidi processing, font availability, glyph output and reader interpretation remained later states with different owners.

Take the Cyrillic capital letter И. A resource might carry it as a literal character in one supported encoding. Another might spell it as И. A third might store that reference in a multibyte UCS representation, producing a very different sequence of octets from an ASCII-compatible file. If the receiver selected the right decoder and the markup was valid, all three routes could reach the same character number in the HTML document.

That was the decisive boundary in RFC 2070. The bytes were properties of a concrete resource. The character number belonged to an abstract HTML document. Confusing the two would make numeric references change meaning whenever a server changed the wire encoding. The specification instead made the reference stable and put variability at the decoder boundary.

SGML needed a character world before it could parse a document

HTML was still specified as an SGML application. SGML's document character set was not a list of bytes permitted in one file. It combined a repertoire of characters with numbers that identified them for the document type and its instances. The DTD, markup and data characters were interpreted within that one numbered space.

RFC 1866's HTML 2.0 baseline included Latin-1 and aligned its positions with ISO 10646, while pointing toward a larger future repertoire. RFC 2070 completed that move for its HTML 2.x internationalization profile by selecting the Universal Character Set of ISO 10646:1993, as amended. At publication, the RFC described that set as code-by-code identical with Unicode 1.1.

The change admitted data characters outside ISO-8859-1 and expanded the range available to numeric references. It did not declare every integer legal. The SGML declaration left positions 128 through 159 unused, so a reference such as ’ remained illegal in that model even if later software commonly assigned it a typographic appearance.

This matters because “Unicode page” can conceal two different statements. One statement concerns the abstract characters HTML can name. The other concerns the external coding used to serialize a particular resource. RFC 2070 fixed the first without requiring every resource to use one byte encoding for the second.

charset chose the road into HTML; it did not redraw the destination

The RFC deferred external encoding to the transport and storage context. Over HTTP, the charset parameter on the response's Content-Type identified the external character encoding. In electronic mail, the MIME Content-Type parameter did the same and supplied its own default rules. The document recorded no standardized signal for FTP or distributed file systems at that time.

MIME terminology was potentially misleading. Its charset did not merely name a repertoire. It named a method for mapping octet sequences to character sequences, possibly with more than one octet sequence leading to the same character result. RFC 2045 required a named MIME charset to define that mapping fully; it did not require every abstract character to be encodable in the reverse direction.

RFC 2070 expressed the division with a reference pipeline:

resource → decoder → entity manager → SGML parser → application → display.

The decoder translated the external representation into the document character set. The entity manager, parser and application then dealt in document characters for character semantics. The display side could transform those characters again into a device representation. An implementation did not have to contain literal modules with those names; it had to behave as if the boundary existed to an outside observer.

That qualification prevents a modern reconstruction from becoming false archaeology. The diagram is a contract for observable behavior, not a census of 1997 browser internals.

A number reference was stable precisely because it came after decoding

RFC 2070 called the invariant of numeric references the most important consequence of its model. They were resolved against the fixed document character set and therefore referred to the same characters whatever external encoding carried the document.

The order is essential. A receiver first needs enough evidence to turn bytes into the markup characters &, #, digits and ;. Only then can the SGML machinery recognize a numeric reference and resolve its number. The reference does not repair an incorrectly decoded stream. It becomes stable once the stream has crossed the decoder boundary correctly.

Nor does a stable character number imply byte identity in the opposite direction. RFC 2070 warned form designers that an untouched default value might return in different valid octets from those in the source document. Composite and precomposed sequences could also vary while representing equivalent text elements. Copying, submitting or saving text created new encoding events, not a promise to reproduce the original file.

The encoding label itself had an evidence hierarchy

RFC 2070 was candid about deployment limits in 1997: servers often omitted an appropriate charset, while some browsers mishandled a Content-Type carrying one. That observation is historical; it is not a current browser census.

The specification preferred the charset received from the source of the document. An early META HTTP-EQUIV declaration came next, and a link's advisory CHARSET attribute came after that. The ordering exposed a bootstrapping problem. A META declaration could only be read if the bytes before and through the element were already decoded well enough for the parser to find it. RFC 2070 therefore called the technique not foolproof and limited its practical value to encodings whose ASCII-valued octets behaved suitably at the beginning.

The hierarchy separated authority from availability. A nearby hint might be available sooner to the user interface, but that did not give it the same authority as the response metadata. Conversely, an authoritative label remained an assertion. It did not cryptographically prove that the body obeyed the named mapping or that a decoder implemented it faithfully.

A decoded character could still fail on the screen

Making UCS the document character set enlarged what HTML could identify. It could not manufacture fonts. RFC 2070 explicitly anticipated characters that a system could parse but not display and declined to prescribe one mandatory fallback. A visible missing-glyph marker or hexadecimal identifier was presentation policy, not a new character identity.

Language and direction added further state. LANG could affect glyph disambiguation, quotation marks, hyphenation, ligatures, spacing and speech. DIR, BDO and the Unicode bidi algorithm could change the visible order needed for semantically legible text. These mechanisms operated after the receiver had character identities. They did not redefine what the external bytes meant.

The chain therefore had at least four distinct failure reports: the declared or inferred encoding could be wrong; the decoder could produce the wrong characters; the parser could build the wrong structure or reject the markup; the renderer could lack a font or apply language and direction rules badly. A screenshot cannot reveal which earlier state failed. A clean byte hash cannot prove any later one succeeded.

The boundary survived the institution that first published it

RFC 2854 obsoleted RFC 2070 and the earlier IETF HTML documents when control of HTML specifications moved to W3C. That succession is a governance and document-history fact, not evidence that the decoder boundary stopped working. The later text/html registration still described charset as the encoding used to represent an HTML document as bytes. HTML 4.01 likewise said the document character set alone was insufficient to interpret an exchanged byte sequence.

Lu Heng's later distinction between a minimum shared specification, local future decisions and running code offers a useful retrospective lens. The fixed character-number space was a small common invariant. A receiver still made local implementation decisions about supported decoders, error handling, fonts and presentation. Publication of the invariant did not execute those decisions.

That is the enduring lesson of RFC 2070's diagram. A resource was not yet a document merely because its bytes had arrived. A document character was not yet a glyph merely because its number had been resolved. Each transition needed its own evidence, and no stage acquired authority by borrowing the receipt from the one before it.

Sources