Summary

  • NFS version 2 made modifying operations return after stable storage. That simplified recovery for a stateless server, but put persistence latency into every write.
  • NFSv3 split requested and achieved commitment into UNSTABLE, DATA_SYNC, and FILE_SYNC; a successful reply also carried the actual byte count and an opaque write verifier.
  • The client retained unstable data until COMMIT or a sufficiently strong result. If the verifier changed, it had to treat old unstable ranges as possibly lost and retransmit them.

The successful call that left the buffer occupied

A client writes a range of a remote file and receives NFS3_OK. Read alone, the status sounds final. But the same result may say committed = UNSTABLE. The RPC succeeded and some bytes were accepted; their survival across loss of server memory remains unresolved.

The contrast begins with RFC 1094. NFS version 2 treated modifying operations as synchronous. When a write returned, the client could assume the request's data was on stable storage and discard its copy. That rule supported the ideal of a stateless server: after failure, clients retried operations instead of reconstructing a long session. It also placed the latency of durable media in every write response.

RFC 1813 identified that path as a write-throughput bottleneck. One of NFSv3's principal changes was support for safe asynchronous writes. “Safe” did not mean that an early reply had become durable. It meant that the protocol preserved the copy, state, and evidence needed either to finish persistence or to repeat the write.

Three levels kept OK from carrying four meanings

The stable field in a WRITE request could ask for UNSTABLE, DATA_SYNC, or FILE_SYNC. FILE_SYNC required the data and all file-system metadata to reach stable storage before the reply. DATA_SYNC required the data and enough metadata to retrieve it. With UNSTABLE, the server could persist all, some, or none before answering.

The server did not merely echo the request. Its committed result reported the level actually achieved. A server with fast non-volatile storage could return FILE_SYNC after a weaker request and let the client retire state early. It could not deliver less than the requested level. The returned value, rather than the request's preference or the outer success code, governed the next obligation.

The reply also carried count. A server could perform a short write and report the actual number of bytes. The client then had to write the remainder. Before asking whether data was durable, it first had to establish which range had actually been accepted.

The design thus separated propositions that dashboards often collapse: a call received a response; a counted range was written; that range reached a stated commitment level; the client may now forget it. Those propositions can all become true, but not at the same instant.

The verifier named an epoch, not a file

The final result field was verf, the write verifier. This opaque cookie had to remain consistent during a relevant server instance and change between instances where uncommitted data might have been lost. A reboot was the common case, but the semantic boundary was the loss of volatile custody, not the physical act of power cycling.

The verifier was not a content digest. Different files and ranges could share it during one instance. Equality did not prove that a file was unchanged, that every requested byte was written, or that another client had not modified it. Its authority was narrower: a later operation could determine whether it still belonged to the incarnation that had held the earlier unstable bytes.

That left memory at the client. RFC 1813 described three buffer states: dirty; done but needs to be committed; and done. The middle category is the cost of the faster response made visible. Transmission may be complete while durability custody remains open.

COMMIT closed the debt only after comparison

COMMIT forced previously unstable modified data to stable storage. It could cover a range; offset zero and count zero meant from the beginning of the file to its end. Calling it a remote, range-capable fsync is useful, but bounded. It did not certify an application's choice of content, a backup, or a replica.

A successful COMMIT returned another verifier. If it matched the WRITE replies, the client could join acceptance and persistence to the same server instance. If it differed, the old instance's volatile custody no longer had evidentiary value. The client had to treat ranges returned under the old verifier as UNSTABLE as possibly lost and write them again.

The word “possibly” matters. Some bytes may have reached disk before the failure. A changed verifier did not measure total loss; it removed the client's ability to distinguish what survived. Rewriting known offsets was a recoverable action under uncertainty, not a claim of precise observation.

The reverse limit mattered too. An unchanged verifier was not a file receipt. The COMMIT status, actual counts, covered ranges, and coordination with other writers still mattered. Server-incarnation continuity was one dimension of proof.

Stable storage had a boundary

RFC 1813 described stable storage as persistent across repeated power failures, certain hardware failures, and repeated software crashes including reboot cycles. It immediately excluded failure of the stable-storage module itself. Durability was not a synonym for backup, replication, logical consistency, or immortality.

The specification also declined to promise strict cache consistency between clients and servers or among clients. A committed range could coexist with a stale view elsewhere. Persistence answers whether bytes survive a failure class; consistency asks which state participants should treat as current.

The mechanism outlived its original generation. RFC 3530 carried it into NFSv4, and RFC 7530 retained the distinction between requested and returned commitment. RFC 8881 states the recovery rule explicitly: when the write verifier changes, a client must assume data returned as UNSTABLE4 under the old value was lost and recover it. A more stateful protocol still needed this narrow incarnation boundary.

Performance moved work instead of deleting it

NFSv3 shortened the normal path by allowing servers to batch persistence. The work reappeared as retained client memory, range accounting, later COMMITs, and replay traffic after an instance change. A large unstable backlog could become a recovery burst precisely when the storage service was returning from disruption.

Asynchrony was therefore a contingent liability with an owner. Systems that sized only steady-state throughput could borrow capacity from their own failure path. The protocol did not hide that debt: it named the intermediate state and supplied the evidence that determined when it could be retired.