Summary

  • NVM Express, Inc. is the non-profit consortium that governs the NVMe family; it does not manufacture SSDs, controllers, switches or storage systems.
  • NVMe replaced assumptions inherited from mechanical disks with multiple in-memory submission and completion queues, better aligned with parallel flash and multicore CPUs.
  • The same controller, subsystem and namespace model can operate locally over PCIe or remotely through NVMe over Fabrics, using TCP and RDMA transports with different operational profiles.
  • NVMe 2.x is modular. The 2.4 specification set, published on 4 August 2026, coordinates Base, PCIe/RDMA/TCP transports, NVMe-MI, Boot and several specialised command sets.
  • The protocol makes access to capacity more composable. It does not define the filesystem, durability, RAID, erasure coding, network design, operational security or application consistency.

A storage command can leave the server without changing its basic language

A read can be directed to a local NVMe namespace over PCIe or to a remote subsystem through a fabric. The medium, controller and path change, but the command model remains the same.

That continuity enables disaggregation. Capacity can be pooled and assigned to hosts on demand instead of installing every device in the chassis that consumes it.

Disaggregation adds dependencies: discovery, networking, multipath, authentication, reconnection and new failure domains. NVMe turns storage into a programmable service, not a topology-free resource.

Flash needed a protocol designed for parallelism

Historical interfaces reflected mechanical latency and narrow queues. Flash can handle many operations in parallel, and modern servers have numerous cores.

NVMe uses pairs of in-memory submission and completion queues. The host places commands, updates doorbells and receives completions through interrupts or polling. Multiple queues reduce shared locks and can be associated with specific cores.

The outcome depends on firmware, PCIe, NUMA, queue depth, interrupts and workload. The protocol name does not guarantee a universal level of performance.

The consortium separates protocol governance from product competition

Work began in the late 2000s; NVMe 1.0 appeared in 2011 and NVM Express, Inc. was incorporated in 2014. Entities include CPU, media, controller, networking, systems and cloud companies.

At the cut-off date, Amber Huffman of Google was President; Curtis Ballard of AMD was Treasurer; and David Allen of Microchip was Secretary. The public board included thirteen promoter representatives.

The consortium defines contracts and compliance programmes. Members compete in silicon, firmware, systems, support and performance. Consortium membership does not prove that a product implements every feature.

NVMe 2.0 turned a growing document into a modular family

With several transports and new data models, a single specification became difficult to maintain. The 2.0 restructuring in 2021 separated Base, command sets and transports.

This allows TCP to evolve without rewriting every command, and ZNS to advance without changing every conventional controller. The trade-off is a more complex version matrix.

NVMe 2.4, published on 4 August 2026, coordinated Base 2.4, PCIe 1.4, RDMA 1.3, TCP 1.3, NVMe-MI 2.2, Boot 1.4 and the NVM, Key Value, Zoned Namespace, Computational Programs and Simple Log Memory command sets. It is not a single file.

Submission and completion queues bring work closer to the cores

The host writes commands to a submission queue; the controller returns status in the completion queue. Different cores or processes can use separate pairs and reduce contention.

Doorbells signal new work. Polling can reduce latency at the cost of CPU time; interrupt coalescing can save cycles while adding delay. Queue depth affects utilisation and tail latency.

A deeper queue is not always better. It can conceal saturation and increase waiting time. Configuration must follow the workload.

Controllers, subsystems and namespaces separate the logical endpoint from the physical medium

A controller exposes queues and commands. A subsystem can group several controllers. Namespaces represent logical capacity and can be presented through multiple paths.

The identity visible to the host no longer necessarily corresponds to an SSD. An array can aggregate many media devices and provide stable namespaces.

NVMe does not determine data placement, replication or protection. RAID, erasure coding, thin provisioning and consistency belong to the higher-level system.

Administrative commands control the I/O environment

The admin queue manages identification, queue creation, features, logs, firmware, namespaces and security. It is the layer that enables routine work.

An administrative command can erase a namespace, activate the wrong firmware or change the power state. It requires permissions, auditing and change control.

Separating admin and I/O improves the architecture, but does not reduce the importance of the control plane. In fabrics, both may depend on the same network.

PCIe keeps the local path close to memory and hardware

NVMe over PCIe uses shared memory and controller registers on the local bus. It is the most direct way to connect SSDs or cards within a server.

Topology still matters. A device may sit behind a PCIe switch, on another NUMA socket or share lanes. CPU, memory and device placement must be properly aligned.

For this reason, an inventory of the ‘number of drives’ is not enough. Some latency may originate within the server before reaching the medium.

NVMe over Fabrics turns a local relationship into a network service

NVMe-oF 1.0 and NVMe-MI 1.0 were published on 9 June 2016. NVMe-oF carries capsules and data to a remote subsystem, where the host creates queues and sees namespaces over the network.

The architecture enables shared pools and separation between compute and storage. Hosts can change without moving data.

It also adds the NIC, switch, routing, discovery, controller, authentication and multipath to the data path. The network fabric becomes part of the integrity system.

Discovery controllers support dynamism and create a critical dependency

A host queries a discovery controller to learn about subsystems, addresses and services. This avoids configuring every target manually.

Incorrect information or an unavailable service can prevent new connections or direct the host to the wrong location. Redundancy, caches, validation and identity protection are required.

Discovery is not the same as data-path availability. Existing connections may continue while new hosts are unable to discover anything.

NVMe/TCP brought the fabric to ordinary IP networks

NVMe/TCP, standardised in 2019, maps commands and data over TCP. Operators can use routable Ethernet, IP tools, firewalls and established practices without building an RDMA environment.

That accessibility adds overhead. TCP, copies, interrupts and CPU use can affect tail latency. The kernel, offloads, transfer size and tuning are decisive.

The transport includes specific framing and digests, and can use TLS. TLS capability does not mean it is enabled or operated properly.

NVMe/RDMA seeks low latency with greater fabric discipline

RDMA uses queue pairs, registered memory and NIC capabilities to move data with less CPU involvement. It is attractive for HPC and AI.

RDMA is not a single network. RoCE, iWARP and other bindings have different requirements for congestion, loss, PFC, ECN and memory.

A fast benchmark does not demonstrate simple operation. A network problem can appear as a storage timeout and require combined expertise.

Multipath turns redundancy into a host decision

A namespace can be visible through multiple controllers and paths. The host selects load balancing or failover. Asymmetric Namespace Access identifies routes as optimised, non-optimised or unavailable.

Two links may share a switch, controller, power source or route. Independence must be tested through real failures.

An incorrect policy can send I/O over a slow path or take too long to abandon a dead one. Redundancy is observed behaviour, not a port count.

Reservations coordinate shared access without replacing consensus

NVMe reservations allow hosts to register and reserve a namespace to prevent unauthorised writers. This is useful in clusters and failover configurations.

They do not replace cluster consensus or application coherence. A failed host can leave state that must be cleared, and fencing must prevent an old node from continuing to write.

An incorrectly recovered reservation can cause unavailability or corruption.

Authentication and TLS matter when storage leaves PCIe

A local device inherited a degree of physical boundary. On a network, the initiator and target must authenticate identities and protect channels.

NVMe defines authentication mechanisms, and TCP can use TLS. The outcome depends on keys, certificates, renewal, algorithms and policy. Support does not equate to secure configuration.

Discovery, admin and data planes must be analysed together. An authenticated identity may still have excessive permissions.

NVMe-MI creates a management plane separate from I/O

NVMe Management Interface enables operators to inventory subsystems, read health information and perform operations even when the application path is not active.

This supports maintenance and recovery, but adds another privileged interface. A management platform can read sensitive information or change devices at scale.

NVMe-MI integrates with Redfish and other systems. Integration does not merge the responsibilities of each standard.

Zoned Namespaces exposes media constraints to software

ZNS divides capacity into sequentially written zones. By making them visible to the host, it can reduce internal garbage collection and improve endurance or predictability.

The benefit requires a filesystem, database or storage layer that understands zones. An application designed for a conventional block device does not gain it automatically.

ZNS demonstrates the trade-off: greater knowledge of the medium can improve efficiency, but raises the cost of software portability.

Key Value, Simple Log Memory and Computational Programs broaden the concept of a namespace

Specialised command sets support keys and values, simple logs or program execution close to storage. They seek to reduce translation and data movement.

Their use depends on controllers, drivers, libraries and applications. Inclusion in the specification does not make them universal.

As the number of options grows, capability discovery and a fallback become more important. Modularity creates flexibility and a new commercial matrix.

Compliance provides evidence, not end-to-end performance certification

Interoperability programmes and workshops test specific behaviours between host and controller. They identify discrepancies that the text does not reveal.

A listing does not measure latency, endurance, recovery or security across an entire topology. Options and driver-firmware matrices change.

Buyers should treat compliance as a baseline and test their own workloads, failures and upgrades.

Disaggregation separates capacity from the server and redistributes responsibility

Local storage associated the device, host and systems team. A remote pool can serve many consumers and change allocations through software.

Network, storage, platform, security and application teams now share incidents. Fabric degradation can look like a database problem; firmware can look like network loss.

The economics depend on utilisation and operational capability, not only on price per terabyte.

AI turns storage latency into compute cost

Training and inference load datasets, write checkpoints and move state at scale. If thousands of accelerators are waiting, wasted storage time becomes a significant cost.

TCP, RDMA, multipath and shared namespaces provide options, but must be evaluated with congestion, recovery, metadata and queue behaviour.

The goal is not only peak throughput, but predictable queue latency and recovery during synchronised workloads.

NVMe 2.4 demonstrates breadth and versioning pressure

The specification set coordinates Base, transports, Boot, Management Interface and command sets. NVMe is now a stack, not an SSD connector.

A host may support Base 2.4 without every command set; a TCP controller may differ from an RDMA controller. Support claims require detail.

Driver, OS, firmware, transport and management tool form a matrix that, in practice, is part of the protocol.

NVMe made storage composable without making it simple

The contract reduces one switching cost: commands and namespaces can survive a move from PCIe to a fabric or from one supplier to another. Data migration, security, observability, networking and support remain.

The advantage is the separation of protocol and product. The risk is that very different architectures are concealed beneath the same NVMe brand.

Storage becomes programmable and distributable, but remains an integrity system that must be understood during failure.