NVIDIA Unveils AI Storage Architecture at FMS, Enabling Direct GPU-to-Storage Access

As AI workloads continue to grow in scale, the role of storage infrastructure is undergoing a fundamental shift. At this week’s Future of Memory and Storage (FMS) conference, NVIDIA showcased new storage advancements and made the case that the next leap in AI depends as much on storage infrastructure as on computing power.

AI Agents Push Storage Demands to New Heights

NVIDIA highlighted that AI agents are consuming massive amounts of data at an unprecedented rate. More notably, GPUs can now initiate storage requests directly, generating thousands of concurrent operations. This places immense pressure on storage systems, which must continuously encrypt, compress, verify, and reconstruct data. These critical data services can become bottlenecks when thousands of agents access storage simultaneously.

According to benchmarks published in an NVIDIA technical blog, the NVIDIA Vera CPU, part of NVIDIA Vera BlueField-4 STX, delivers up to 3.21 times higher throughput than an x86 CPU in a two-stage compression and encryption pipeline. This means storage platforms built with Vera can absorb the flood of AI data more efficiently, delivering greater throughput with significantly less compute infrastructure.

From Passive Storage to Active Data Path

For four decades, storage has been treated as a passive place to hold data. The traditional tradeoff was between memory (fast but expensive) and storage (capacious but slow). NVIDIA argues that accelerated computing is fundamentally changing this equation.

On today’s GPUs, paired with AI storage solutions from NVIDIA and its partners, the same access-time tradeoff has shifted from minutes to microseconds. Storage is no longer a passive data repository; it becomes an active participant in the data processing path.

NVIDIA emphasized that closing the gap between AI’s needs and memory shortages requires deep co-design across the entire ecosystem, from memory and storage manufacturers to the software built on top of them.

cuFile APIs Go Open Source, Enabling GPU-Storage Interoperability

At FMS, NVIDIA announced it is open-sourcing its cuFile application programming interfaces (APIs) and the vertical storage software stack underneath them. cuFile is an open-source component of NVIDIA GPUDirect Storage, enabling GPUs, not just CPUs, to read from and write to storage directly.

By leveraging hundreds of thousands of GPU threads, fast high-bandwidth memory, and other techniques, cuFile can securely access data from storage in microseconds. This represents how the industry is unifying around a security-first storage stack based on Linux best practices, providing interoperability between GPUs and data.

NVIDIA noted that fast, secure access to data and storage is foundational to powering preventive and detective cybersecurity measures. Making cuFile openly available helps drive standardization of security practices across the industry.

Source: NVIDIA Blog