AI's Memory Stack: Bernstein Says Demand Is Spreading Beyond HBM

AI's Memory Bottleneck Is No Longer Just an HBM Story

Bernstein analysts are telling investors that the artificial-intelligence memory boom is no longer confined to high-bandwidth memory. While training large models still leans heavily on HBM, the analysts say demand is spreading into conventional DRAM, NAND flash and storage as AI workloads become more varied.

The distinction matters because different stages of AI use make different demands on hardware. Training is compute-intensive, with memory bandwidth limiting model size and computing speed. Inference is more complex: the initial prefill stage is largely compute-bound, but the later decode stage is memory-bound. Models store earlier tokens in a key-value cache, and Bernstein says those KV caches could eventually need more memory than the model weights themselves in large deployments — a constraint on how many users an AI service can support.

Against that backdrop, Bernstein highlights a broader set of equipment winners. The analysts rate Samsung Electronics, SK hynix, Micron, SanDisk, Seagate and Western Digital Outperform, while Kioxia is rated Underperform. Emerging approaches such as CXL memory, Nvidia's Storage Next and CMX context storage are being developed to balance performance, capacity and cost as AI systems demand larger memory pools.

How the Workload Shift Splits Samsung, SK hynix, Micron and Storage Rivals

Why inference turns DRAM into a capacity problem

The article's most consequential analytical point is that inference is not one workload. The prefill phase is compute-bound and favors GPUs and HBM, but the decode phase is memory-bound because every token generated depends on the model's KV cache. Bernstein's argument is that KV-cache requirements grow with context length and the number of concurrent users, so in large deployments the cache may outgrow the model weights. That would make conventional DRAM capacity a direct driver of how many users an AI service can support — not simply a support component for accelerators.

This interpretation is ours, but it follows from the mechanics the analysts describe: a service that reaches memory limits has to reduce context, queue users or spend more on memory infrastructure. That links memory supply to the unit economics of AI services.

The ratings split from Samsung to Kioxia

Bernstein's specific calls are broad but not uniform. Samsung, SK hynix and Micron give investors exposure to HBM plus conventional DRAM and, in Samsung's case, NAND. SanDisk, Seagate and Western Digital sit further down the stack in storage. Kioxia is the outlier: it is rated Underperform, though the article does not state the reason. Readers should treat that as Bernstein's relative positioning rather than a diagnosis supplied by the report.

Where CXL, Storage Next and high-bandwidth flash could reset the stack

The note identifies a wave of technologies intended to ease the tension among performance, capacity and cost: CXL memory, Nvidia's Storage Next and CMX context storage. In addition, memory manufacturers are working on high-bandwidth flash, which would combine HBM-like bandwidth with NAND's larger capacity and lower cost. Bernstein says the technical hurdles for high-bandwidth flash remain high, so the technology is not a near-term substitute for existing memory types; it is a longer-term attempt to redraw the cost curves of the AI memory stack.

What Bernstein's Six Outperform Names Mean for Investors Watching the Memory Stack

• Separate training from inference exposure. Bernstein's framework says training remains HBM-heavy, while the decode stage is memory-bound because KV caches scale with context length and concurrent users. Companies or products exposed to DRAM capacity may benefit differently from those tied mainly to HBM.
• Use the named ratings as the analyst's coverage map. Bernstein rates Samsung Electronics, SK hynix, Micron, SanDisk, Seagate and Western Digital Outperform, with Kioxia Underperform. The note itself gives no explanation for the Kioxia call, so that rating should not be read as more than a relative view.
• Assess the named emerging architectures as indicators of demand direction. CXL memory, Nvidia's Storage Next and CMX context storage are the specific technologies Bernstein says are being developed to balance performance, capacity and cost; their adoption would signal which part of the memory stack captures incremental AI budgets.
• Use RAG database build-outs as a demand signal. Bernstein says RAG databases require substantial SSD or HDD capacity alongside system DRAM, with DRAM playing a larger role when those databases are searched. Enterprise RAG adoption is therefore a direct demand signal for the storage and conventional-memory names.
• Treat agentic AI as a demand multiplier for traditional servers. Because autonomous systems retain intermediate results and pass outputs between agents, Bernstein says they can rapidly increase KV-cache demand and raise requirements for CPUs and conventional memory in traditional servers, not only in GPU-accelerated boxes.