SK Hynix Advances PIM, HBF, and SALT-KV as HBM Strategy Extends to Inference Workloads
nashnova research
SK Hynix unveiled three new technology tracks for inference — PIM, HBF, and SALT-KV — at its AI infrastructure summit, marking a formal expansion of its memory strategy beyond GPU-training support into the latency-capacity-cost trilemma of inference workloads.
What problem does PIM actually solve?
The decoding phase of AI inference shuttles data back and forth between memory and accelerator chips. The shuttling itself is the bottleneck.
PIM — processing-in-memory, running simple operations inside or right next to the memory die — flips the model: data stays put; compute moves to it, slashing data-movement overhead.
This means → compared with SRAM, the go-to for latency-sensitive inference, PIM delivers roughly 300× the capacity in the same chip area while maintaining high internal bandwidth.
In plain terms = SRAM is a tiny fast desk; PIM is a warehouse retrofitted with its own assembly line — goods get processed without ever leaving the building.
What are the new architecture and "rack-scale PIM"?
SK Hynix proposes physically separating the memory die from the compute die, then reconnecting them vertically via hybrid bonding — a process that joins two chips face-to-face.
This means → compute logic no longer eats into memory-die area, saving space while keeping processing closer to data.
The longer-term vision scales PIM from a single chip to the full server rack, tracking the industry's move toward rack-scale computing.
Why is HBF called a "new memory tier"?
Today's memory stack has a gap: HBM is fast but expensive and capacity-limited; SSDs are cheap and large but slow. Nothing sits cleanly in between.
HBF — high-bandwidth flash — combines 3D NAND's high density with TSV (through-silicon via, vertical hole-based interconnects between chip layers), aiming to deliver near-HBM bandwidth at near-flash cost.
This means → for long-context inference and high-volume serving — workloads that are capacity-first and cost-sensitive — HBF offers a price-performance option that did not previously exist.
What does SALT-KV solve at the software layer?
As conversations grow longer and multi-step reasoning becomes common, the KV cache — temporary data a model uses to remember context — keeps ballooning. Parking all of it in the most expensive HBM tier is unsustainable.
SALT-KV is a software scheduling framework: it segments cache data, scores each segment on reuse probability and storage cost, then routes it to HBM, DRAM, or SSD accordingly.
In plain terms = it is a hot-cold sorting pass on cache data — frequently used blocks stay in the fast lane; the rest move to cheaper storage — cutting overall cost without interrupting service.
What is the system-level logic behind all three?
Earlier this month at SEMICON Taiwan, SK Hynix labeled data movement the next major computing bottleneck.
Starting with HBM4, the base die can be built on a logic process node, creating room to add functionality near the DRAM stack instead of routing every operation back to the accelerator.
This reflects a deeper signal: memory makers are shifting from component suppliers to system co-design partners. Whether PIM, HBF, and SALT-KV reach volume production on schedule and clear customer qualification will be the key test of that strategic pivot's commercial viability.
市场有风险，内容仅供研究参考，不构成投资建议。