
Intro: Why AI Storage Architecture Is Changing
Modern AI systems are no longer limited primarily by GPU arithmetic. Training and inference workloads increasingly depend on how quickly large volumes of data can move from persistent storage to accelerators. Datasets contain billions of small files, model checkpoints can reach terabytes, and retrieval-augmented generation systems continuously fetch embeddings, documents, and metadata.
This is forcing a redesign of AI storage architecture. Instead of treating storage as a passive repository behind a file system, infrastructure teams are examining whether storage devices, GPUs, DPUs, and network adapters should participate directly in data processing.
A GPU may process data at extraordinary speed, yet remain underutilized while the host system resolves file paths, checks permissions, translates files into blocks, copies buffers, and manages network requests. The problem resembles a high-speed factory supplied by a narrow conveyor belt: increasing the factory’s machinery does little if materials arrive slowly.
Traditional data paths often look like this:
- An application requests a file.
- The operating system and file system resolve metadata.
- The block layer translates the request.
- A storage driver submits commands to an SSD.
- Data is copied through host memory.
- The CPU prepares buffers for the GPU.
- The GPU finally begins processing.
Every step consumes CPU cycles, memory bandwidth, or latency budget. At scale, these costs multiply across thousands of drives and accelerators.
AI infrastructure teams must now optimize the complete path between data and computation. Storage capacity and peak sequential throughput remain important, but they are no longer sufficient metrics. Teams must also evaluate metadata operations, random-read behavior, CPU utilization, PCIe topology, network congestion, and the cost of moving data between address spaces.
The emerging direction is clear: make the data path more direct and more intelligent. GPU direct storage can reduce host-memory copies. NVMe over Fabrics can make remote flash appear like a high-performance storage resource. File-semantic SSDs and computational storage aim to move file processing itself into the drive.
The result will not be one universal architecture. Local NVMe, GPU direct storage, disaggregated NVMe-oF, and computational storage will coexist, selected according to workload characteristics.
Background: How AI Storage Architecture Works Today
AI storage architecture is the combination of storage media, controllers, file systems, operating-system layers, networks, accelerators, and software used to deliver data to machine-learning workloads. It covers both physical infrastructure and the interfaces that determine how data is addressed and transferred.
GPUs perform parallel computation, but CPUs still coordinate file systems, scheduling, security, memory management, and I/O. SSDs provide persistent storage and increasingly high bandwidth through PCIe. Networks connect compute nodes to shared or disaggregated storage.
In a conventional server, the CPU acts as a traffic controller between the SSD and GPU. That arrangement is flexible, but it can become inefficient when many accelerators request data simultaneously. The CPU may spend more time managing data movement than performing application logic.
File systems provide essential abstractions: filenames, directories, permissions, consistency, and allocation. Block layers expose storage as addressable sectors. These abstractions are valuable, but each translation introduces work.
For large sequential reads, that overhead may be relatively small. For AI workloads involving millions of small files or random accesses, path lookup and metadata processing can become a major bottleneck. In this context, the file system and block layer can resemble two translators standing between a customer and a warehouse: each adds coordination before the requested item reaches its destination.
GPU direct storage technologies, including NVIDIA’s GPUDirect Storage and the cuFile software interface, allow storage data to move more directly into GPU memory. The objective is to bypass unnecessary copies through host memory and reduce CPU involvement.
This does not eliminate every software layer. Applications still need compatible libraries, file systems, drivers, and storage devices. However, it can reduce the number of memory copies and allow the GPU’s high-bandwidth memory subsystem to receive data more efficiently.
GPU direct storage primarily improves the movement of data after the application has identified what it needs. The host may still perform file lookup, permissions checks, file-to-block translation, queue management, and other control-plane operations.
Vendor measurements have suggested that file-system and block-layer activity can consume a substantial share of CPU cycles in random-read workloads, but such results depend heavily on configuration and require independent validation. GPU direct storage is therefore best understood as a shorter data path, not a complete removal of the host from storage architecture.
NVMe over Fabrics, or NVMe-oF, extends the NVMe command model across networks such as Ethernet, Fibre Channel, or InfiniBand. It allows compute nodes to access remote NVMe devices with lower software overhead than traditional network file-storage protocols.
For AI clusters, NVMe-oF can support disaggregated storage pools. GPUs and servers do not need to be physically attached to every SSD. Instead, high-speed network adapters and storage controllers connect compute resources to shared flash.
A file-semantic design could place NFSv3 requests inside standard NVMe commands, allowing remote clients, DPUs, and accelerators to request files without relying on a conventional storage-server CPU for every operation.
| Approach | Primary benefit | Main limitation |
|---|---|---|
| Local NVMe | Lowest path latency | Limited sharing |
| GPU direct storage | Fewer host copies | Host metadata remains |
| NVMe-oF | Disaggregated access | Network latency and complexity |
| Computational storage | Processing near data | Immature ecosystem |
Local NVMe remains attractive for latency-sensitive workloads. GPU direct storage improves accelerator utilization, while NVMe-oF adds flexibility and pooling. Computational storage goes further by executing selected operations inside or near the drive.
Trend: Storage Intelligence Moves Into the Drive
A file-semantic SSD attempts to make files directly addressable by the storage device. Instead of receiving only block addresses, the SSD firmware can process file-oriented requests, including lookup and access validation.
In a file-semantic design, firmware may handle path resolution, permission checks, and file-to-flash translation. The host no longer needs to perform every step before issuing a block request.
The approach effectively moves part of the file system into the SSD controller. It is similar to moving a warehouse’s inventory system onto the loading dock: requests can be matched to physical goods without repeatedly consulting a distant central office.
One proposed design carries NFSv3 operations inside standard NVMe commands. NFSv3 is relatively small, stateless, and broadly supported. Its stateless structure means each request contains the information needed for processing, reducing the need for the SSD to maintain complex session state.
This could allow local applications, GPUs, DPUs, and remote clients to access files through the same command model. However, embedding a file protocol into NVMe does not automatically make it a standard. Interoperability will depend on vendor adoption, specifications, drivers, and security behavior.
Computational storage for AI places selected processing near the data. SSD controllers may filter, transform, decompress, index, or organize data before transferring it to a CPU or GPU. DPUs and smart NICs can coordinate remote access, security, and data movement.
For example, an AI pipeline could request only records matching a filter rather than transferring an entire dataset to the host. Inference systems could also retrieve and pre-process embeddings near the storage layer.
Conventional architectures often maintain separate mappings: the file system maps files to blocks, while the SSD’s flash translation layer maps logical blocks to flash cells. A file-semantic SSD can combine these layers into a unified file-oriented mapping.
This may reduce duplicated metadata and repeated garbage-collection work. Some vendor simulations have reported significant write-amplification reductions compared with conventional host-side file systems, but simulated results should not be treated as production measurements.
Moving lookup, mapping, and filtering into firmware can free host CPUs for orchestration and model-serving tasks.
A unified mapping layer may avoid redundant data movement and cleanup. Lower write amplification could improve endurance and sustained performance.
Typical SSDs devote substantial DRAM to block mapping. A file-semantic design could use more of that memory for frequently accessed file metadata, potentially improving small-file workloads.
If drives process file requests directly, remote access may scale with storage resources rather than the CPU count of a central file server. Preliminary vendor testing has reported approximately 460,000 operations per second remaining relatively stable as server CPU resources declined, while a conventional Linux NFS server degraded more sharply. These results require broader independent testing.
Reducing server CPUs, memory copies, and duplicated processing could lower power and infrastructure costs. Projected savings, including claims of materially lower power and cost for multi-drive systems, should be evaluated against complete system designs rather than isolated drive metrics.
Insight: Comparing Emerging AI Storage Architectures
File-semantic SSDs may be especially useful when workloads involve many small files, metadata-heavy access patterns, and remote clients. By resolving file operations closer to flash, they could reduce CPU contention before data reaches the GPU.
GPU direct storage is more mature and better suited to environments already optimized for large files, checkpoints, and streaming datasets. Organizations can deploy it without redesigning the storage device’s command model.
The two approaches are not mutually exclusive. A future system could combine file-semantic lookup with direct GPU transfer.
NVMe-oF generally offers a more direct protocol path than traditional network file storage, particularly for high-throughput block workloads. File-semantic NVMe commands could add file access while retaining efficient transport.
Traditional NFS is widely understood and integrated with enterprise tools. NVMe-oF may deliver better performance but requires specialized networking, multipathing, monitoring, and operational expertise. File-semantic storage adds another compatibility layer that must be proven across operating systems and applications.
A result near 460,000 operations per second suggests that drive-side processing can reduce dependence on storage-server CPU resources. It does not prove that every workload will achieve the same performance. Operation size, access distribution, metadata locality, queue depth, and network behavior all matter.
A Gen6-class drive tested in a PCIe 5.0 platform can be constrained by the host link. Sequential reads near 14.1 GB/s may reflect the platform ceiling rather than the drive’s full capability. PCIe 6.0 testing will be important for determining whether the firmware can sustain higher bandwidth.
Simulations, whitepaper models, and preliminary demonstrations are useful indicators, but production decisions require reproducible tests. Independent validation should include end-to-end AI pipelines, not only synthetic operations.
A proprietary file-semantic command set risks vendor lock-in. Standardization must address naming, permissions, consistency, caching, error handling, encryption, and multi-tenant isolation.
Earlier research, including device-embedded file systems and key-value NVMe commands, demonstrates that storage semantics can move below the host file system. The recurring challenge is ecosystem adoption. A technically sound interface still needs operating-system support, application integration, observability tools, and long-term compatibility.
Forecast: The Next Phase of AI Storage Architecture
PCIe 6.0 SSDs and 800G network adapters could make remote storage fast enough for increasingly demanding AI pipelines. DPUs may handle transport, security, virtualization, and data routing while GPUs remain focused on computation.
Future reference architectures may combine high-density Gen6 SSDs, NVMe-oF fabrics, DPU-enabled just-a-bunch-of-flash systems, and direct accelerator access. Storage will become a composable resource allocated dynamically to training and inference jobs.
Compression, filtering, deduplication, vector retrieval, and format conversion are likely candidates for near-data processing. The most successful functions will be those that reduce data movement without imposing excessive firmware complexity.
The long-term goal is a common data plane where GPUs, DPUs, NICs, and remote clients can access storage with minimal host mediation. This could make storage behave less like a passive disk system and more like a distributed data service.
Partner samples and Gen6 benchmarks will reveal whether file-semantic controllers can sustain their claimed benefits at full interface speed. Testing should include random reads, mixed workloads, metadata-heavy access, and failure recovery.
Adoption will accelerate when vendors provide stable APIs, Kubernetes integrations, observability, security controls, and migration paths from NFS and POSIX file systems.
Applications may depend on subtle POSIX behavior. Differences in locking, caching, or consistency could prevent direct adoption.
Moving access checks into firmware creates new trust boundaries. Secure boot, encryption, identity propagation, audit logs, and tenant isolation must be first-class features.
Synthetic operations-per-second numbers do not capture training checkpoints, streaming reads, retrieval workloads, failures, or noisy-neighbor effects. Transparent, independently reproduced benchmarks will determine credibility.
Call to Action: Plan Your AI Storage Architecture Upgrade
Profile CPU cycles, system calls, metadata latency, memory copies, queue depth, and GPU idle time. Establish a baseline before changing architecture.
Large datasets, checkpoint loading, and streaming reads are strong candidates. Small-file workloads may require additional metadata optimization or file-semantic storage.
Specify measurable targets for operations per second, tail latency, bandwidth, CPU utilization, and concurrent clients.
Include switches, DPUs, network adapters, software licensing, cooling, maintenance, and staff expertise. A faster drive is not automatically a lower-cost system.
Use real datasets, checkpoint patterns, retrieval indexes, and production-like concurrency. Synthetic benchmarks should supplement—not replace—application tests.
Require repeatable measurements across vendors and platforms. Validate failure handling, permissions, upgrades, observability, and security before committing to a large deployment.
Conclusion: Building a More Direct AI Storage Architecture
GPU direct storage reduces unnecessary host copies, while NVMe-oF enables flexible access to remote flash. Together, they create a foundation for disaggregated AI infrastructure with shorter and more scalable data paths.
File-semantic SSDs extend the idea by moving lookup, mapping, and access operations into the storage device. If standardized and independently validated, they could reduce CPU overhead for metadata-heavy AI workloads and make remote file access more efficient.
Organizations should begin with measurement rather than assumptions. Profile current bottlenecks, test GPU direct storage where workloads fit, evaluate NVMe-oF for disaggregation, and track emerging file-semantic and computational-storage products.
The future of AI storage architecture will not be defined by SSD capacity alone. It will be defined by how directly data can move from persistent media to the accelerators that transform it.