Summary

  • RFC 9561 gives pNFS SCSI layouts an NVMe vocabulary for namespace identity, reservation keys, client fencing and cache flushing.
  • A namespace identifier names a target; a preempt command requests a state transition; a completed flush supports a bounded stable-storage claim. None is the application's final result.
  • Operators need a joined receipt chain across host identity, every controller path, reservation state, post-fence I/O, volatile-cache state, LAYOUTCOMMIT and later readback.

The metadata server returns from LAYOUTCOMMIT. On a dashboard, the operation is green. Yet three different questions remain: did the old client lose access on every path, did the relevant bytes leave volatile cache, and did the application later see the intended file state? RFC 9561 supplies the interoperable commands. It does not collapse those questions into one answer.

RFC 9561, published in April 2024 on the IETF Standards Track, maps the pNFS SCSI layout defined by RFC 8154 onto NVMe. It is explicit about scope: the document does not amend the layout. It maps SCSI concepts to NVMe constructs and works across controller transports such as PCIe, RDMA, TCP and Fibre Channel.

The identifier reaches a namespace, not an outcome

An eligible NVMe namespace must expose EUI64 or NGUID. The pNFS volume designator carries one of those binary values; its XDR length distinguishes the eight-octet EUI64 from the sixteen-octet NGUID. This is useful precision. It lets two implementations refer to the same namespace representation.

But the identifier does not attest current contents, controller-path agreement or application use. A host can reach a namespace through multiple controllers. NVMe reservation keys apply across the controllers associated with the same Host Identifier. The resulting host-to-namespace association is wider than a single path, which is exactly why an inventory of Host Identifiers and controller associations matters.

Fencing has a command receipt and a storage result

Before returning the volume, the metadata server registers its key and acquires an Exclusive Access – Registrants Only reservation. To fence a non-responsive client, it sends Reservation Acquire with Preempt or Preempt and Abort, naming both its own key and the target client's key. A fenced client can identify the condition through Reservation Conflict.

The command is not the entire proof. RFC 8154 states the intended result more strongly: all I/O issued by the fenced client must be rejected. That claim needs before-and-after reservation state and observed I/O behavior on every relevant controller path. A preempt completion without the target path inventory can become a neat receipt for an incomplete operation.

RFC 9561 also prescribes recovery. After a non-retryable NVMe error, the client must commit affected layouts through the metadata server, return outstanding layouts, forget the device ID and unregister its key. The specification defines responsibility; it does not remotely certify that a particular client completed the sequence.

A successful commit contains a conditional flush edge

NVMe controllers may use a volatile write cache. When the VWC capability and WCE setting make that cache active, the metadata server must issue NVMe Flush so the cache reaches stable storage before LAYOUTCOMMIT returns. The mapping equates that operation with SCSI SYNCHRONIZE CACHE.

This makes a successful return meaningful, but bounded. The evidence still needs the correct namespace, cache state, command ordering, flush completion and failure context. Stable storage at the device does not automatically prove replication, later readability or an application transaction. It says something narrower and valuable about the storage boundary the command reached.

The security boundary is similarly layered. NFSv4 and RPC protect interaction with the metadata server, not automatically the lower-layer NVMe I/O. NVMe/TCP can use TLS; PCIe may offer essentially no added protocol security. NFS commonly authorizes users and files, while NVMe authorizes initiators and volumes. The client must preserve the correspondence. One green control plane cannot impersonate the other.

Heng Lu's reality-layer discipline applies cleanly: identifier, configuration, executed command, device state and application observation are all real. Trouble begins when one is promoted into the next without evidence.