Hot Chips 2026: Analysis Suggests High Bandwidth Flash Is a High-Capacity Memory, Not an HBM Replacement
GPU IP company OXMIQ Labs said at Hot Chips 2026 that High Bandwidth Flash (HBF) cannot replace High Bandwidth Memory (HBM) across the vast majority of workloads. For some applications, HBF could emerge as a specialized memory tier for huge but relatively cold datasets. For others, it can do more harm than good.
- OXMIQ Labs said High Bandwidth Flash cannot replace HBM across most workloads
- HBF offers roughly 8 to 16 times more capacity than HBM at about the same cost
- Modeling shows an HBF-only config multiplies rack capacity by 14 but cuts aggregate bandwidth to about 60% of HBM
- Strongest use cases include MoE expert weights, fitting very large models, and long-context inference
What HBF is
SanDisk unveiled its High-Bandwidth Flash concept in early 2025, promising AI accelerators terabytes of relatively inexpensive memory and a reduced need for traditional HBM. The emerging HBF specification includes three performance grades. Grade 1 uses an 8-Hi 256GB NAND stack with an 8 GT/s UCIe interface and 384 GB/s bandwidth. Grade 2 uses a 512GB NAND stack with a 16 GT/s UCIe interface and 1.536 TB/s bandwidth. Grade 3 reaches 3.072 TB/s using 32 GT/s UCIe 2.0 while keeping 512GB of capacity.
Because HBF relies on 3D NAND, it supports read block sizes between 64 bytes and 4 kilobytes, 4KB writes and 4KB page sizes. OXMIQ said HBF Grade 3 can compete against HBM4E, but HBF’s main feature is not raw performance; it is 8 to 16 times more capacity than HBM at roughly the same cost.
The trade-off between capacity and bandwidth
OXMIQ modeled a 72-GPU rack running the 1-trillion-parameter Kimi-K2 model at FP4. At cost and power parity, an HBM-only configuration provides 20.7TB of memory and 1,584 TB/s of aggregate bandwidth. Replacing HBM with HBF increases rack capacity 14 times to 294.9TB but cuts aggregate bandwidth to 922 TB/s. A hybrid configuration provides 89.3TB and between 279 TB/s and 1,418 TB/s depending on workload.
In an HBF-only configuration each GPU can hold its own Kimi-K2 instance, running 72 model instances per rack, whereas an HBM-only configuration needs eight GPUs per model instance and runs nine instances per rack. This makes HBF attractive when memory capacity determines the number of GPUs required. But as users and token-generation rates climb, HBF’s lower bandwidth becomes the bottleneck, and the HBM-based rack can deliver lower cost per token thanks to substantially higher memory bandwidth. OXMIQ’s conclusion: “HBM for the rack, HBF for the box.”
Good fits and the hardest part
OXMIQ said mixture-of-experts (MoE) models appear especially suitable for HBF. Its Kimi-K3 example has 1.56TB of weights, of which 1.45TB, or 93%, are MoE expert weights. Since only selected experts activate for each token, this pool is largely write-once and infrequently read, so OXMIQ proposes keeping these experts in HBF while frequently accessed weights stay in HBM. More local capacity could also reduce communication between accelerators and expert-parallel network traffic. Long-context inference with large KV caches is another potential use case.
The biggest obstacle is software. HBF requires large transfers (64KB reads, 1MB writes) moved via DMA, along with management of its limited write endurance. Current inference software is not ready: OXMIQ says vLLM would need dedicated HBF support, and accelerator vendors such as AMD and NVIDIA would need hardware and driver mechanisms to move data between HBF and HBM. This requires joint effort across HBF hardware vendors, AI accelerator vendors and inference-framework developers. For now HBF remains a nascent technology whose value depends heavily on particular workload characteristics.
Source: Tom’s Hardware
Cover image generated by AI for illustrative purposes only.
