SK Hynix Unveils Memory-Centric AI Rack Architecture MARS
nashnova research
SK hynix launched MARS at the OCP Global Summit, using a CXL bus to turn DRAM from a fixed resource tied to each server into a rack-wide shared memory pool — targeting the core bottleneck in today's AI infrastructure: memory capacity can't keep up, and scaling it costs too much.
Where exactly is AI server memory hitting a wall?
High-bandwidth memory (HBM) on GPUs is capped by physical space and cost. Adding memory means buying an entire new server — along with GPUs the operator may not need.
This means → operators end up over-provisioning expensive GPU clusters just to get more memory, spending on compute they never use.
Shuttling KV cache — the data a large language model uses to remember context — across servers via RDMA networking introduces redundant data copies and unpredictable tail-latency spikes.
What does MARS actually change?
The core idea: fully decouple memory allocation, scaling, and reclamation from CPU/GPU compute.
In plain terms = memory used to be a fixed water tank welded onto each machine. MARS turns it into a shared pipe — whichever GPU is thirsty draws straight from the pipe, no dedicated tank required.
Multiple GPUs and NPUs — neural-network processors — can share model weights and context KV cache from the same pool. In a pipeline workload, one accelerator writes its output to shared memory and the next reads it directly, skipping the network hop entirely.
What do the performance numbers show?
Multi-turn agentic AI: offloading growing KV cache to the MARS pool cut time-to-first-token (TTFT) by 3.5–5.2× versus recomputation, virtually eliminating tail-latency spikes.
Voice-agent pipeline (jointly tested with AI partner Pickles): routing intermediate outputs through shared CXL memory instead of RDMA cut time-to-first-audio from 0.67 s to 0.46 s and end-to-end latency from 8.08 s to 5.73 s.
Trillion-scale graph neural network training: graph sampling sped up 17.57× over the RDMA baseline — the shared pool offers constant-latency direct access, removing frequent RDMA synchronization waits.
How big is the software stack and partner network?
SK hynix is co-developing a 16-tool MARS software stack with industry, government, and academic partners, covering memory allocators, inter-node locking, KV cache tiering, in-memory storage, and a Graph RAG engine.
Partners include Los Alamos National Laboratory, Stanford, Carnegie Mellon, UC Berkeley, and more than ten other top institutions.
This reflects a play beyond hardware: SK hynix is trying to turn MARS into an industry standard through a software ecosystem — hardware is easy to replicate, ecosystems are not.
What is HBF, and why add another memory tier?
High-bandwidth flash (HBF) sits between HBM and enterprise SSDs: roughly half HBM's bandwidth, 10–14× its capacity, and 3.7× better energy efficiency per bit than conventional enterprise storage.
In plain terms = HBM is a small, expensive bowl of fast memory; an SSD is a cheap, slow barrel. HBF is a mid-size bucket — speed close to the bowl, capacity close to the barrel.
In demos with Llama 3 70B and DeepSeek R1 671B, HBF supported context windows of up to 19 million tokens per user. The OCP HBF spec v0.7 is published; v1.0 is planned for next year, with HBF Cube hardware samples expected around 2028.
How should investors read this?
If MARS delivers, the most direct impact is that scaling data-center memory no longer requires proportionally scaling GPUs — reshaping AI infrastructure procurement logic and capex structure.
This means → memory makers (SK hynix itself) could gain more leverage in the AI supply chain, while the case for selling pure GPU compute gets diluted.
Two validation checkpoints to watch: whether MARS can replicate these gains in live data-center deployments, and whether HBF ships on schedule around 2028. Until then, this remains a technology roadmap, not a deployed commercial reality.
市场有风险，内容仅供研究参考，不构成投资建议。