Solid-state drives need fast controller communication, and NVMe storage provides an open interface protocol designed to link non-volatile flash memory to host processors over PCI Express (PCIe) connections [8]. Older storage buses slowed flash media by routing data through controllers built for mechanical platters [5]. NVM Express changes that path by pairing controller logic with multi-lane bus links, letting host memory push requests straight into hardware queues [8]. Modern NVMe storage architectures remove legacy software delays so solid-state drives read and write blocks at full media speeds [5].
The Origins of NVMe Storage
Flash memory reached mass markets between 2005 and 2010, but early solid-state drives were held back by slow hard disk buses while high-end drives relied on proprietary PCI Express expansion slots, hardware RAID controllers, or SAS bridges. Custom driver requirements forced hardware vendors to write individual software for every operating system. Standardizing this communication path let computers manage solid-state media through a single shared driver, similar to USB mass storage devices [8].
Early design work on a shared standard began when Intel organized the NVMHCI working group in 2007. The group completed the NVMHCI 1.0 specification in April 2008, defining host protocols that talked to memory chips from the Open NAND Flash Interface Working Group. Technical development on NVM Express started in late 2009. Amber Huffman led the effort. Amber Huffman of Intel chaired the NVM Express Workgroup, uniting more than 90 technology companies including Cisco, Dell, EMC, and Samsung to create an open industry specification that eliminated per-device driver development across computing platforms [8].
The consortium published the version 1.0 specification in March 2011, followed by version 1.1 in October 2012 with multi-path support and namespace sharing. Integrated Device Technology released the IDT 89HF16P04AG3 and IDT 89HF32P08AG3 chipsets in August 2012. Samsung announced the Samsung XS1715 enterprise drive in July 2013, reporting read speeds of 3 GB/s, while LSI SandForce delivered the LSI SandForce SF3700 controller family later that year. Intel followed in June 2014 with its data center family, including the Intel SSD DC P3700, Intel DC P3600, and Intel DC P3500 series. NVM Express, Inc. incorporated in March 2014 under a board comprising Cisco, Dell, EMC, HGST, Intel, Micron, Microsoft, NetApp, Oracle, PMC, Samsung, SanDisk, and Seagate [8].
Why AHCI Created Bottlenecks for Flash
Computers long relied on the Advanced Host Controller Interface (AHCI) to move data between processors and storage drives, but StorageReview writer Juan Mulford said AHCI was built for spinning mechanical disks where physical heads took milliseconds to seek data tracks [5]. Physical heads took milliseconds to seek data tracks [8]. Because of this mechanical delay, the interface could not support thousands of rapid electronic operations in parallel [5].
SATA III reached its limit. SATA III connections capped transfer rates at 600 MB/s, which choked available throughput for fast solid-state media while the AHCI standard managed only a single queue holding 32 commands. When busy enterprise software or multi-core processors fired hundreds of read requests at the same time, the AHCI queue backed up completely. Host processors sat idle while waiting for completions [5].
Software stacks handling AHCI wasted processor time on locks and hardware interrupts [5]. Every input and output action generated an interrupt request. Interrupt processing forced the host CPU to pause active work and switch execution contexts [6]. Solid-state drives could not show their real throughput while trapped behind controller rules built for rotating magnetic platters. System designers bypassed the old storage controller pipeline by connecting solid-state memory straight to the PCI Express bus with dedicated lanes, which eliminated SATA bus caps and allowed memory chips to communicate with host software using microsecond response times [5].
Submission and Completion Queues
NVMe storage replaces the single AHCI queue with a parallel architecture that supports 65K queues. Controllers can stack up to 65K commands within every queue. Parallel processing delivers high capacity for concurrent workloads. Modern multi-core processors process multiple data operations at once instead of queuing work behind a single hardware bottleneck [5].
Multi-core processors take advantage of this layout by allocating dedicated submission and completion queue pairs to individual CPU cores. A CPU core writes entries into its own queue without waiting for locks on shared memory buffers [5]. When an application reads data, the operating system driver places a command into a host submission queue. The driver rings a controller doorbell register. The NVMe storage controller fetches the command over PCIe, executes the read on NAND flash cells, and places a status entry into the matching completion queue without burdening other processor cores [8].
The completion workflow operates with the same efficiency within system memory. After reading the requested flash cells, the storage controller writes a completion status entry and sends an interrupt. In version 1.2, NVM Express added the Host Memory Buffer feature to improve performance on entry-level hardware, allowing drives built without onboard DRAM chips to allocate a portion of the host computer’s main memory to cache flash translation tables. Dynamic memory caching allows budget SSDs to deliver fast response times without expensive dedicated memory chips on the drive card [8].
Physical Form Factors for Flash Drives
First-generation drives arrived as PCI Express add-in cards that plugged into motherboard expansion slots, but while full-height and half-height cards suited enterprise servers, personal computers needed smaller hardware. The industry introduced the M.2 form factor, initially called Next Generation Form Factor, to bring four PCIe lanes to a compact module that plugs flat against circuit boards in laptops and desktops [8].
For hot-swappable enterprise storage racks, the consortium established the 2.5-inch U.2 form factor, formerly known as SFF-8639. A U.2 connector routes four PCI Express lanes through a sturdy backplane socket. Technicians can replace faulty drives while servers keep running. Enterprise servers can mount up to 48 U.2 drives in a single chassis. The newer U.3 standard, detailed in SFF-TA-1001, builds on U.2 by letting a tri-mode backplane receptacle automatically support NVMe, SATA, or SAS drives through automatic controller detection [8].
NVMe specifications also power removable consumer memory cards and massive scientific computing clusters. In September 2016, the CompactFlash Association announced CFexpress, a removable card standard that uses NVMe protocols over PCI Express lanes to accelerate digital cameras. At the high-performance computing tier, supercomputers including Fugaku Supercomputer, Summit Supercomputer, and Sierra Supercomputer use NVMe solid-state hardware as burst buffer storage. High-speed flash tiers absorb huge volumes of scientific data during simulation runs, and these installations save states quickly before offloading records to long-term storage pools [8]. NVMe solid-state drives can deliver write speeds higher than 3,000 MB/s, making them five times faster than SATA SSDs and thirty times faster than spinning hard disks [5].
Restructuring the NVMe Storage Standard
As solid-state drives took on new roles in enterprise servers, maintaining a single specification document grew complicated, prompting the NVM Express consortium to refactor the entire standard with the release of version 2.0 in 2021. NVM Express president Amber Huffman said rearchitecting the specification was necessary to meet the evolving demands of future storage technologies, explaining that pulling out transport and command specifications makes it simpler for developers to build specialized hardware [3].
Technical Work Group chair Peter Onufryk said adding distinct media required different command sets and namespaces. Version 2.0 introduced the Zoned Namespaces (ZNS) specification, which lets host software and SSDs collaborate on data placement by writing sequentially to physical flash zones. ZNS lowers the drive write amplification factor by grouping data based on usage and access frequency. Gary Hilson reported in EE Times that Samsung Electronics enterprise SSDs using ZNS technology can achieve up to four times the lifespan of standard flash drives. ZNS technology provides durable, energy-efficient storage for cloud data centers [3].
The refactored standard added Endurance Group Management so administrators can organize flash media into distinct pools, configuring NAND flash blocks into SLC or TLC mode. “What Endurance Group Management does is it allows you to create multiple pools of NVM and then create namespaces out of those pools,” said Onufryk. Version 2.0 also added the Key Value command set, which retrieves data using a key rather than a block address. Key-value indexing bypasses intermediate translation tables. Mechanical hard disk drives gained support under the refactored specification as well. “When we created NVMe, we were really targeting NVM,” said Onufryk, saying the consortium never imagined users would connect spinning disks to NVMe [3].
How NVMe over Fabrics Moves Data
Enterprise storage networks needed to share high-speed flash pools without losing low-latency benefits, leading NVM Express Inc. to release version 1.0 of the NVMe over Fabrics (NVMe-oF) specification in June 2016 [4]. NVMe-oF extends NVMe command sets across network fabrics, replacing local PCIe memory mapping with message-based network encapsulation while keeping latency in microseconds. Remote storage targets appear as namespaces, functioning like SCSI logical unit numbers with higher concurrency [5].
Data centers deploy NVMe-oF across three primary transport types: Remote Direct Memory Access, Fibre Channel, and TCP/IP. RDMA transports such as InfiniBand and RoCE move data directly between host memories without involving operating system kernels or host processor caches. RoCE v1 operates at Ethernet layer 2, while RoCE v2 routes packets across IP subnets using UDP. Linux Kernel 4.5 and Mellanox OFED 2.3 added RoCE v2 support [5]. TechTarget analysts Paul Kirvan and Carol Sliwa reported that the INCITS T11 committee finalized the FC-NVMe standard in August 2017. Networking vendors like Broadcom, Cisco, and Marvell supply Fibre Channel switches and host bus adapters supporting FC-NVMe, running NVMe and legacy SCSI traffic together on Fibre Channel fabrics [4].
Enterprise virtualization platforms adopted fabric storage to supply scalable capacity to virtual machines. VMware introduced NVMe-oF connectivity in vSphere 7.0, allowing VMware ESXi hypervisors to discover remote flash namespaces over Fibre Channel and RoCE v2 networks. Instead of using the older Native Multipathing Plugin, ESXi relies on the High-Performance Plugin to route traffic to NVMe targets, which appear as active namespaces with the same 65K parallel queues available on local drives [5]. The University of New Hampshire’s Interoperability Lab provides formal testing through UNH-IOL to verify that multi-vendor NVMe storage gear complies with industry specifications [1].
User Space Processing and Kernel Bypass
As solid-state drive speeds accelerated, operating system kernels began to limit maximum I/O performance because traditional kernel drivers create hardware interrupts for every completed storage operation. Interrupts forced host processors to pause active work and switch execution contexts. When drives process millions of operations every second, interrupt servicing consumes excessive CPU cycles. Kernel bypass solves this bottleneck [6].
To remove software delays, the Storage Performance Development Kit created an open-source reference NVMe-oF target running in user space under a BSD license. Benjamin Walker announced the SPDK target alongside community Linux kernel host and target implementations in June 2016. SPDK uses a polling-based asynchronous model rather than interrupts, checking queues constantly to handle completions. Built on the OpenFabrics Enterprise Distribution layer, SPDK supports InfiniBand, iWARP, and RoCE fabrics, enabling fast media like 3D XPoint to run at full speed [6].
Adoption of NVMe storage also grew in client computing platforms and boot firmware. Flash Memory Summit 2014 data projected client NVMe SSDs from manufacturers such as Samsung and SK Hynix would account for over 60 percent of the solid-state market. Firmware developers Jason B. Akers of Intel and Aaron Durbin of Google added an EFI NVMe driver port to the Chromium OS depthcharge project to allow Chromebook devices to boot from NVMe SSDs. Kconfig settings let developers validate storage platforms early. Their work showed that compact firmware code can initialize NVMe controllers before the operating system kernel starts [7].
Recent Additions to NVMe Storage
The NVM Express specification continues to expand with new releases that address enterprise multi-tenancy and data integrity. Version 1.4 introduced IO Determinism to isolate workloads into separate sets so that virtual machines achieve predictable latency. Diagnostic event logs and Verify commands assist engineers with troubleshooting hardware faults [8].
Security and out-of-band management received important enhancements in version 2.1. Key Per I/O allows host software to apply individual encryption keys on a per-operation basis. Live Migration maintains service uptime when moving active workloads between physical hosts. Furthermore, NVM Express standardized UEFI network boot capabilities and improved out-of-band management through the NVMe Management Interface, letting administrators inspect drive diagnostics outside regular data channels [8].
The consortium ratified its version 2.4 specifications on August 4, 2026 [2]. Cryptographic protocols incorporate post-quantum algorithms to protect stored enterprise records against future computational attacks. Version 2.4 also provides PCIe Exported NVM Subsystem Migration, extending virtualization controls to locally attached PCIe SSDs. Flexible Data Placement enables host software to mark hot and cold data ranges to maximize physical flash efficiency. Controller-based rate limiting allows storage administrators to restrict bandwidth and IOPS across individual namespaces, while integrated voltage monitoring tracks hardware health in server racks [8]. These continued architectural updates confirm that NVMe storage provides the enduring technical foundation for high-speed computer memory and enterprise data storage [1].
- WEBSITE NVM Express. (2015, October 8). NVM Express – scalable, efficient, and industry standard. [Article Link]
- WEBSITE NVM Express. (2020, January 10). Specifications – NVM Express. [Article Link]
- ONLINE NEWS Hilson, G. (2021, June 30). NVMe Gets Refactored. EE Times. [Article Link]
- ONLINE NEWS Kirvan, P., & Sliwa, C. (2024, July 26). What is NVMe over Fibre Channel (NVMe over FC)? Definition and Guide. TechTarget. [Article Link]
- ONLINE NEWS Mulford, J. (2020, July 27). What is NVMe-oF? StorageReview.com. [Article Link]
- WEBSITE Walker, B. (2016, June 7). Announcing the SPDK NVMf Target. SPDK. [Article Link]
- WEBSITE Akers, J. B., & Durbin, A. (2015, January 22). Add simple NVMe storage driver. Chromium OS Depthcharge Project. [Article Link]
- WEBSITE Wikipedia. NVM Express. [Article Link]
