Sandisk and SK hynix have unveiled the first technical standard for High Bandwidth Flash (HBF), a technology developed to address the growing memory bottleneck in AI hardware. Published through the Open Compute Project, the work aims to move NAND flash beyond conventional SSD connections and place it much closer to the processor. According to the initial definitions, HBF packages could use 8- or 16-layer NAND structures to provide capacities of up to 512 GB and bandwidth of approximately 3 TB per second in the highest performance tier.
These figures clearly distinguish HBF from an ordinary enterprise SSD. While NVMe SSDs communicate with the processor through PCI Express and a controller, HBF is designed as broadly connected memory stacks positioned around the processor package. The standard’s adoption of the open UCIe chiplet interconnect for links to the host processor or accelerator is important because it prevents the technology from being tied to a single GPU manufacturer. In theory, CPUs, GPUs and AI accelerators from different vendors could use the same fundamental interface.
Why is HBF being developed now?
In large language models, inference has become as serious a capacity challenge as training. When a model generates a response for a user, it stores not only its weights but also the KV cache, which represents previous words and conversational context. As the context window, number of simultaneous users and model size increase, the accelerator’s expensive HBM capacity can become insufficient. Reading the data again from a remote SSD increases latency and may leave the GPU’s compute units waiting.
HBF aims to create a new tier between these two extremes. Because it is based on NAND, it can retain data during a power outage and reach much higher capacities than HBM. However, the access latency, write endurance and operating characteristics of NAND cells differ from those of DRAM-based HBM. A peak bandwidth of 3 TB per second therefore does not mean that HBF will respond as quickly as HBM in every workload.
The initial standard defines three performance tiers, with reported speeds ranging from approximately 0.4 TB/s to 3 TB/s. A top-tier 512 GB HBF stack could offer several times the capacity of current HBM packages. This could make it possible to keep a larger portion of model weights or large KV caches close to the accelerator. Less frequently used data could be stored in HBF, sections needed for immediate computation in HBM, and large datasets in SSD and hard-drive tiers.
It will not replace HBM immediately
Describing the new standard as “cheap HBM” would be misleading. HBM is a DRAM technology developed for matrix computations that require low latency and a continuous, intensive flow of data. HBF instead emphasizes high capacity, persistence and NAND’s economical density. The most likely use will be in hierarchical systems where the two types of memory complement each other. The operating system, accelerator software and AI runtime will have to decide which data should be held in each tier.
Write endurance is another critical issue. While weights are mostly read during AI inference, the KV cache can change continuously. Manufacturers will need to implement wear leveling, error correction, bad-block management and thermal-control mechanisms without excessively increasing package size or latency. The stated bandwidth figures must also be independently verified with real models, under sustained loads and across different read-write patterns.
Why does an open standard matter?
Developing HBF under the umbrella of the Open Compute Project reduces the risk of the interface becoming a proprietary feature controlled by one vendor. Interest in the working group from organizations such as Google and Tenstorrent indicates that the standard is more than a presentation by memory manufacturers, although the ecosystem is still at an early stage. Without controller designs, packaging capacity, server boards, software support and reliability testing, the technical specification alone does not create a marketable product.
Sandisk’s previously announced roadmap targeted the first HBF samples for the second half of 2026 and samples of the first AI inference devices using HBF for early 2027. The new announcement did not provide a firm mass-production date for this schedule. It should therefore be viewed less as the launch of a product available for purchase today and more as a technical milestone that allows manufacturers to begin designing products.
If HBF succeeds, its impact may not be limited to running larger models. Some inference tasks that are divided across multiple accelerators because they do not fit within HBM capacity could be handled with fewer chips, reducing the need for data movement and recomputation. It is not yet certain, however, that total energy consumption would fall. Additional capacity could encourage the use of larger models and longer contexts, offsetting the resulting efficiency gains.
Ultimately, HBF introduces a technically compelling third option between the “fast but limited HBM” and “spacious but distant SSD” in AI servers. Its 512 GB capacity, 3 TB/s ceiling and UCIe connection are noteworthy, but product samples, endurance measurements, latency results and software support will determine its true value. For now, the most important development is that NAND flash is preparing to move beyond its role as a storage device and become a direct part of the accelerator package.