Summary

  • RFC 2372 advised creating a prepared-recovery record before sending PREPARED and a commit-recovery record before sending COMMIT from Prepared state, then retaining each record until named resolving evidence arrived.
  • That ordering made a protocol statement depend on durable local memory, but it did not prove that bytes reached stable storage, survived a crash, replayed under the correct transaction identity or produced a result the application could know.

The most dangerous word in a distributed transaction is often not COMMIT. It is PREPARED.

COMMIT sounds final, so engineers instinctively ask what it means. PREPARED is easier to hear as an intermediate acknowledgement: a participant has done some work, the coordinator will decide, and the process will continue. Yet once a subordinate sends PREPARED, it has crossed a one-way boundary. It may no longer discard the transaction merely because a socket closes or a process restarts. It has promised to preserve enough state to obey a future decision.

RFC 2372, published in July 1998, made that promise visible by describing what local recovery records should exist before particular TIP messages left a transaction manager. The document was Informational, not an Internet standard, and it called its protocol-to-log guidance advisory rather than definitive. That modest status matters. It did not report a deployed product, a storage engine, a crash test or a successful recovery.

But its ordering rules reveal a durable principle in Internet infrastructure: before software announces a state whose consequences survive failure, the evidence needed to reconstruct that state must already be under custody.

PREPARED changed who could still choose

TIP used a presumed-abort form of two-phase commit. Before a participant prepared, failure could resolve toward abort. After the participant had prepared, unilateral retreat was no longer safe. The superior might already have collected every vote and decided to commit. The subordinate therefore had to remember the transaction across the very failures that made the network conversation disappear.

RFC 2372’s guidance was explicit about order. A subordinate was to create a prepared-recovery record before sending PREPARED. It was to retain that record until it received ABORT, COMMIT or QUERIEDNOTFOUND. It was not to send COMMITTED or NOTRECONNECTED while that prepared record still existed. The record represented an unresolved obligation, not merely a cached message.

The superior carried a related burden. It first received PREPARED. Before sending COMMIT while the transaction was in Prepared state, it was to create a commit-recovery record. That record remained until COMMITTED or NOTRECONNECTED returned. The superior’s durable evidence therefore had to precede the outward instruction whose replay might later be necessary.

This sequence is stronger than “log important events.” It describes custody transitions. PREPARED transfers a participant from local freedom to externally constrained duty. COMMIT transfers a superior from deliberation to a decision it may have to repeat. Deleting either record is not housekeeping. It is a claim that specified evidence has closed the duty.

A connection was not the memory

TIP separated the application conversation from transaction coordination. Business requests could travel on an application channel such as HTTP while transaction managers used a different channel. That two-pipe design made the transaction protocol reusable, but it also prevented the connection from serving as the full record of what had happened.

After a failure, a superior could issue RECONNECT using the subordinate’s transaction identifier, recovering that identifier from its transaction log if necessary. If the subordinate still held Prepared state, it answered RECONNECTED. If it had already received COMMIT, answered COMMITTED and forgotten the transaction, it answered NOTRECONNECTED. The negative response was not necessarily failure; in the specified sequence it could be evidence that the subordinate had already completed and discarded its state.

A subordinate could ask the opposite question with QUERY. QUERIEDEXISTS meant that the superior still knew the transaction and would later reconnect. QUERIEDNOTFOUND let the subordinate abort because the superior might have sent ABORT, failed to receive ABORTED and then forgotten the presumed-aborted transaction. The same apparent absence therefore had meaning only inside a role, state and message history.

That is why a restored TCP connection was never enough. Transport could reconnect endpoints. It could not recreate the correct transaction identifier, decide which side held the unresolved duty or prove whether a missing record meant “safely finished,” “presumed aborted” or “lost.” Recovery depended on local durable identity joined to remote protocol evidence.

The deletion receipts were asymmetric

The prepared record and the commit record did not share one universal closing signal. The subordinate retained its prepared evidence until ABORT, COMMIT or QUERIEDNOTFOUND. The superior retained commit evidence until COMMITTED or NOTRECONNECTED. Those lists encode different knowledge positions.

For the subordinate, an authoritative decision or a recovery query proving that the superior no longer knew the transaction could release the prepared obligation. For the superior, the participant’s acknowledgement—or the specific inability to reconnect after the participant had already forgotten completed state—could close the need to repeat COMMIT. The protocol did not license deletion because a timer expired, a process came back, disk space was scarce or an operator believed the peer was probably done.

Timeouts still had a place. RFC 2372 anticipated local transaction timeouts to limit unavailable resources and help resolve deadlock. A timeout policy could prevent locks from persisting forever. It did not turn elapsed time into proof of the correct outcome. Resource release, protocol resolution and recovered knowledge remained separate facts.

Written order did not prove durable order

The phrase “create a recovery record” looks precise until it meets a storage stack. Did create mean append to a userspace buffer, write into the kernel page cache, flush a journal, persist to media, replicate to another failure domain or receive a storage acknowledgement? Could the record be torn? Could its transaction identifier be recovered without the volatile index that mapped it? Was the outbound PREPARED actually ordered after durability, or could an asynchronous sender outrun the log?

RFC 2372 did not answer those implementation questions, nor did it pretend to. It supplied requirements and supplemental information. It did not name a database, publish a conformance suite, inject a power failure or measure a reconstruction. Its local record guidance is therefore an obligation to be implemented and tested, not evidence that any implementation met it.

Heng Lu’s running-code lens is useful here. Text can establish the sequence that correct software must preserve. Operational proof begins later: observe the durable write, interrupt the process at the adversarial instant, restart it, rebuild the transaction identity, reconcile with the peer and verify the local resource outcome. The RFC gives that test a shape. It does not give the test a passing result.

The same distinction applies to deletion. An implementation might remove the record only after receiving the named message and still be wrong if the removal became durable before the resource manager’s corresponding state transition. Conversely, a system might retain stale records safely but indefinitely, creating resource and operator burdens. Correctness lives in the relationship between protocol state, log state and resource state, not in the presence of a file bearing a plausible name.

Delegation moved responsibility; it did not erase it

RFC 2372 distinguished a lightweight client from a fuller server system. A client could delegate transaction coordination to a server and need no transaction log of its own. That arrangement reduced the client’s durable machinery, but the recovery burden did not vanish. It moved to the system that accepted the role.

This asymmetry is easily misread. “No client log required” does not mean “no durable recovery evidence required.” It means the client is not the custodian of that evidence for the delegated protocol state. An audit must follow the obligation to the server: which record it wrote, under which identity, before which statement, and after which receipt it deleted the record.

Delegation also leaves the caller’s knowledge unresolved. A client application could invoke commit, lose the final response and later discover that the transaction had nevertheless completed. RFC 2372 left the application responsible for ascertaining the outcome and suggested that an implementation-specific user log could help. A transaction manager could successfully recover its protocol state while the person or service that initiated the work still lacked a business-level answer.

That boundary keeps this history distinct from the broad lesson of RFC 2371. The protocol definition showed that a TIP COMMIT response was not automatically a business receipt. RFC 2372’s sharper contribution was to expose the local evidentiary chain beneath recovery: the participant’s prepared record, the superior’s commit record, the transaction identity recovered after failure and the particular remote signal that allowed each record to disappear.

Two pipes demanded application discipline

The separation between application traffic and TIP commands created another ordering problem. The transaction manager could not necessarily see whether the application still had requests in flight. RFC 2372 therefore required application discipline: do not call commit while associated application requests remain outstanding, and do not positively answer a partner’s transactional request before the local transaction manager has registered the transaction.

No transaction log can repair an application that violated that serialization. A perfectly durable commit record might preserve a decision about an incomplete set of business operations. A perfectly reconstructed TIP state might be joined to an application reply that escaped too early. The layers must be checked together without treating them as the same layer.

This is the reality-layer problem in concrete form. There was an application request, a local resource action, a TIP state, a log record, a storage acknowledgement, a network message, a recovered identifier, a peer response, a final local outcome and a client-visible conclusion. Each could be true while the next remained unknown. Collapsing them into a single word such as “committed” created confidence faster than evidence.

Security authenticated a peer, not the whole transaction

RFC 2372 kept application authentication and authorization outside TIP. Its own security concern was narrower: an unauthorized party could issue commands or impersonate a trusted transaction manager. TLS could authenticate endpoints and encrypt TIP commands, though its use was negotiated.

Even a properly authenticated TLS channel would not prove that the peer had durably written its recovery record, that the application request was authorized, that the resource manager applied the correct operation or that a later business system recognized the outcome. Endpoint identity protected one edge of the evidence chain. It did not substitute for local durability or application authority.

That separation is especially important during recovery. A reconnecting peer needs both a trusted channel and the correct transaction identity. One without the other is inadequate. A securely authenticated request for the wrong transaction is still wrong; a correct identifier accepted from an impostor is dangerous. Security evidence, state evidence and business authorization remain complementary.

A minimum specification left future implementation local

RFC 2372 did not prescribe a universal log format, storage engine, transaction API or operational console. That restraint made sense. Interoperability required shared commands, roles, identities and recovery meanings. It did not require every participant to use the same database.

Through Heng Lu’s minimum-initial-specification lens, this was a localized decision boundary. The RFC defined enough common behavior for two systems to reconcile after a broken conversation. Each implementation retained the freedom—and the burden—to choose how it would make the record durable, bind it to resource state, expose ambiguity to applications and test the failure path.

The architecture’s freedom is therefore inseparable from evidence. A vendor or operator cannot answer a recovery question merely by pointing at the specification. The useful questions are empirical: Was the record durable before the message? Does it survive power loss? Can the identifier be reconstructed without volatile indexes? Which exact receipt permits deletion? Is deletion ordered with local completion? Can the application learn the final result after losing its response?

Those questions remain contemporary because distributed systems still speak before they fail. A queue says accepted; a storage service says replicated; an orchestrator says ready; a payment service says authorized. Each word creates expectations about what will survive. RFC 2372’s lesson is not that a log makes the word true. It is that a word with post-failure consequences needs an inspectable, durable and correctly ordered evidence object behind it.

Sources