Summary

  • NVM Express, Inc. is the non-profit industry consortium behind the NVMe specification family. It does not produce SSDs, controllers, switches or storage systems.
  • NVMe replaced serial assumptions from the hard-disk era with multiple in-memory submission and completion queues, matching parallel flash and multicore CPUs.
  • The same model of controller, subsystem and namespace works locally over PCIe or remotely over NVMe over Fabrics; TCP and RDMA bring different operating profiles.
  • NVMe 2.x is modular. The 2.4 set, published on 4 August 2026, coordinates the base specification, PCIe/RDMA/TCP transports, NVMe-MI, boot and several specialised command sets.
  • The protocol makes capacity more composable, but it defines neither file system, endurance, RAID, erasure coding, network design, operational safety nor application consistency.

A storage command can leave the server without changing its underlying language

An application reads from a local PCIe namespace or from a remote subsystem over a fabric. Media, controller and path change, yet the command model remains largely the same.

This continuity enables disaggregation. Capacity can be pooled and assigned to hosts on demand, rather than building every device into the consuming server.

Added to this are discovery, networking, multipath, authentication, reconnect and new failure domains. NVMe turns storage into a programmable service, not a topology-free resource.

Flash needed a protocol for parallelism rather than mechanical latency

Older interfaces reflected hard drives and narrow queues. Flash can process many operations in parallel, while servers have numerous CPU cores.

NVMe uses pairs of submission and completion queues in memory. The host writes commands, rings doorbells and receives completions via interrupt or polling. Multiple queues reduce shared locks and can be mapped to cores.

Performance still depends on firmware, PCIe topology, NUMA, queue depth, interrupts and workload. The protocol name guarantees no universal latency.

The consortium separates protocol governance from product competition

Work began in the late 2000s; NVMe 1.0 appeared in 2011, and NVM Express, Inc. was founded in 2014. Members come from CPU, media, controller, networking, cloud and systems markets.

As of the reference date, Amber Huffman of Google was President, Curtis Ballard of AMD was Treasurer and David Allen of Microchip was Secretary. The public board listed thirteen promoter representatives.

The consortium defines contracts and compliance. Members continue to compete on silicon, firmware, systems, support and performance. Membership does not prove a complete implementation.

NVMe 2.0 split a growing document into a modular family

Multiple transports and data models made a single document unwieldy. The 2.0 restructuring in 2021 separated the base, command sets and transports.

TCP can evolve without rewriting every command; ZNS can grow without changing every standard controller. The trade-off is a more complex version matrix.

The 2.4 set of 4 August 2026 included Base 2.4, PCIe 1.4, RDMA 1.3, TCP 1.3, NVMe-MI 2.2, Boot 1.4, and the NVM, Key-Value, Zoned Namespace, Computational Programs and Simple Log Memory command sets. “NVMe 2.4” is not a single document.

Submission and completion queues bring work closer to CPU cores

The host writes commands into submission queues; the controller reports results in completion queues. Processes or cores can use their own pairs and reduce contention.

Doorbells announce new work. Polling can reduce latency and consume CPU; interrupt coalescing saves cycles and can increase wait time. Queue depth affects utilisation and tail latency.

A deeper queue is not automatically better. It can hide saturation and increase latency. The right setting follows the workload.

Controllers, subsystems and namespaces separate logical endpoints from the media

A controller provides queues and commands. A subsystem can contain multiple controllers. Namespaces represent logical capacity and can be reachable over multiple paths.

Host identity therefore does not necessarily match a physical SSD. An array can pool many media and present stable namespaces.

NVMe determines neither data placement nor replication or protection. RAID, erasure coding, thin provisioning and consistency sit in the surrounding system.

Administrative commands define the environment for I/O

The admin queue manages identification, queue creation, features, logs, firmware, namespaces and security. It creates the context for normal reads and writes.

An admin command can delete a namespace, activate incorrect firmware or change a power state. Permissions, audit and change control are critical.

Separating admin from I/O improves the architecture, but it does not make the control plane less critical. On a fabric, both can depend on the same network.

PCIe keeps the local path close to memory and hardware

NVMe over PCIe uses shared memory structures and controller registers on the local bus. This is the direct path to SSDs and cards in the server.

Topology remains important. A device can sit behind a PCIe switch, on another NUMA socket or on shared lanes. CPU, memory and device must be placed sensibly.

Drive count is not enough as inventory. Latency can arise in the host before the media is even reached.

NVMe over Fabrics turns a local relationship into a network service

NVMe-oF 1.0 and NVMe-MI 1.0 appeared on 9 June 2016. NVMe-oF transports capsules and data to a remote subsystem and creates queues across the fabric.

Capacity can be pooled and separated from compute. Hosts can be replaced without moving the data.

In return, NIC, switch, route, discovery, controller, authentication and multipath enter the data path. The network becomes part of the data integrity system.

Discovery controllers simplify attachment and create a critical dependency

Hosts query discovery controllers for subsystems, addresses and services. Manual configuration of every target becomes unnecessary.

Incorrect information or an outage can block new connections or point to the wrong target. Redundancy, caching, validation and identity protection are required.

Discovery and data availability are not the same. Existing sessions can keep running while new hosts find nothing.

NVMe/TCP brought storage fabrics onto ordinary IP networks

The TCP transport, standardised in 2019, maps commands and data onto TCP connections. Operators can use routed Ethernet, IP tools and firewalls without building an RDMA fabric.

Accessibility comes at the cost of overhead. The TCP stack, copies, interrupts and CPU affect tail latency. Kernel, offload, transfer size and tuning matter.

The transport adds framing and digests and can use TLS. TLS support does not mean TLS is enabled or operated correctly.

NVMe/RDMA targets low latency and demands fabric discipline

RDMA uses queue pairs, registered memory and NIC offload to move data with less CPU involvement. This is attractive for HPC and AI.

RDMA is not a single network. RoCE, iWARP and other bindings have different requirements for congestion, loss, PFC, ECN and memory registration.

A fast benchmark does not prove simple operation. Fabric problems appear as storage timeouts and require networking and systems expertise.

Multipath makes redundancy a host decision

A namespace can be reachable through multiple controllers and paths. The host decides load balancing and failover. Asymmetric Namespace Access marks optimised, non-optimised or unavailable paths.

Two links can share a switch, controller, power supply or route. Independence must be tested with real failures.

A wrong policy keeps a bad path too long or sends I/O over a slow route. Redundancy is observed behaviour, not port count.

Reservations coordinate shared access but do not replace consensus

NVMe reservations let hosts register and reserve a namespace so that unauthorised writers are blocked. They help in cluster and failover scenarios.

They do not replace cluster consensus or application consistency. A failed host can leave state behind; fencing must stop old nodes from writing.

Poor recovery can turn protection into outage or corruption.

Authentication and TLS become important once storage leaves PCIe

A local device benefited from an implicit physical boundary. On the network, initiator and target must prove identities and protect channels.

NVMe defines authentication; TCP can use TLS. The outcome depends on keys, certificates, rotation, algorithms and policy. Support is not the same as secure use.

Discovery, admin and data planes must be considered together. A genuine identity can still hold too many rights.

NVMe-MI creates a management plane separate from application I/O

The NVMe Management Interface allows inventory, health queries and certain actions outside the primary I/O path.

This supports maintenance and recovery, but adds a privileged interface. Management tools can read sensitive information or change devices at scale.

NVMe-MI can be integrated with Redfish. The integration does not remove responsibility boundaries between standards and vendors.

Zoned Namespaces makes media constraints visible to software

ZNS divides capacity into sequentially writable zones. Visibility to the host can reduce internal garbage collection and improve endurance or predictability.

The benefit requires zone-aware file systems, databases or storage layers. A normal block application does not benefit automatically.

ZNS illustrates the trade-off: more media insight improves efficiency and raises the portability requirements on software.

Key Value, Simple Log Memory and Computational Programs extend the namespace

Specialised command sets enable key/value access, simple log structures or programs close to storage. They aim to reduce translation and data movement.

Use depends on controllers, drivers, libraries and applications. A specification does not make the feature universal.

The more options there are, the more important capability discovery and fallback become. Modularity creates flexibility and a new market matrix.

Compliance provides evidence but does not certify end-to-end performance

Programmes and workshops test concrete host-controller interactions and uncover deviations that the text alone does not show.

A listing does not measure latency, endurance, recovery or security in every topology. Optional features and driver-firmware matrices change.

Buyers should treat compliance as a floor and test their own workloads, failures and upgrades.

Disaggregation separates capacity from the host and redistributes responsibility

Local storage linked device, server and systems team. A remote pool serves many consumers and is allocated by software.

Networking, storage, platform, security and application teams now share incidents. Fabric degradation can look like a database failure; firmware problems like network loss.

Economics depend on utilisation and operational capability, not only on price per terabyte.

AI turns storage latency into compute cost

Training loads datasets, writes checkpoints and moves state at scale. When thousands of accelerators wait, storage wait time becomes expensive.

TCP, RDMA, multipath and shared namespaces offer different architectures. Congestion, recovery, metadata and queue behaviour belong in the selection.

The goal is not only peak throughput but predictable queue and recovery latency under synchronised load.

NVMe 2.4 shows breadth and version pressure

The set coordinates the base, transports, boot, management interface and command sets. NVMe is a stack, not merely an SSD connection.

A host can support Base 2.4 without any command set; a TCP controller can have different features from an RDMA product. Support statements must be specific.

Drivers, OS, firmware, transport and management tools form a matrix that is practically part of the protocol.

NVMe made storage composable, not simple

The shared contract lowers one switching-cost barrier: commands and namespaces can survive a move from PCIe to fabric or from vendor to vendor. Data migration, security, observability, networking and support remain.

The benefit is the separation of protocol and product. The risk is hiding very different architectures under the same name.

Storage becomes programmable and distributed, but remains an integrity system that must be understood under failures.