Memory, not compute: Majestic Labs bets LPDDR6 pooling can crack Nvidia's AI wall

Why a Tel Aviv Startup Says AI's Real Bottleneck Is Memory, Not GPUs

Majestic Labs, a Tel Aviv startup founded in 2023 by former Google and Meta engineers, has unveiled an AI inference server built without a single GPU. Its Prometheus system pairs Arm cores with RISC-V vector and tensor engines — units the company calls Ignite AI Processing Units — and links up to 12 of them to a pooled memory space of between 8TB and 128TB of LPDDR6, the same low-cost memory found in smartphones.

The bet behind it is simple: serving AI models is increasingly limited by how much fast memory a chip can actually reach, not by raw compute. And high-bandwidth memory (HBM), the scarce and costly type GPUs depend on, is the wrong tool for the job. Majestic's answer is to pool cheap memory through custom aggregation chiplets connected by copper cables up to a metre long, creating one coherent block far larger than a GPU box can address.

The comparison it draws is stark. An Nvidia DGX B300 carries 2.3TB of high-bandwidth memory; Majestic claims Prometheus offers more than 50 times that fast memory, at 1.7 times the interconnect bandwidth, with one rack matching 25 Nvidia Vera Rubin racks for fast memory at a fraction of the power. On the software side, the server is built to open standards and supports PyTorch, vLLM and OpenAI's Triton, with models trained for GPUs meant to run unchanged.

Every figure, however, is Majestic's own, ahead of independent testing — and nothing has shipped. The company has roughly 40 staff across Tel Aviv and Los Angeles and raised $100 million late last year. It says hardware arrives next year, alongside a growing list of startups attacking Nvidia from unconventional angles: optical chips, edge-inference silicon, open-networking gear. The memory-wall pitch is the newest of them, and the least proven.

Can a 128TB Memory Pool Really Take on Nvidia? The Claims, the Physics and the Field

The memory-wall argument has real weight — in principle

The claim that inference is memory-bound holds up structurally. Serving a large model is often gated by how quickly weights and the growing key-value cache can be moved into compute, not by how many FLOPs the chips can perform. HBM is scarce and expensive; LPDDR6 is cheap and abundant. Pooling it at tens of terabytes is a plausible way to cut the cost per byte of fast memory sharply — which is why the pitch deserves attention even though the headline numbers (50x memory, 1.7x bandwidth, 25 Vera Rubin racks) are self-reported vendor figures with no independent verification.

The hard part is coherence at 64,000 dies

The design's weakest point is physical. A 128TB pool built from 2GB LPDDR6 dies would need roughly 64,000 of them, implying more than a hundred aggregation chiplets in a single server, all kept coherent over copper cables up to a metre long. That is a lot of parts to coordinate, and it is the step from architectural argument to working product. The smaller 8TB configuration is a far more credible place to start proving it.

A new flank in the attack on Nvidia — aimed at the software moat

Majestic joins optical-chip startups, edge-inference silicon and open-networking vendors in probing different weaknesses of the incumbent. Its particular angle targets Nvidia where the stakes are highest: the software ecosystem. By supporting PyTorch, vLLM and Triton and promising that GPU-trained models run without changes, Majestic is trying to make switching cost nearly zero. If independent tests confirm that drop-in compatibility, the memory advantage becomes usable in practice. If they do not, the hardware spec counts for little.

Execution risk is the quiet variable

Roughly 40 staff and a $100 million raise is a modest base from which to take on Nvidia's engineering scale, and the company's claim of orders from large enterprises, neoclouds and hyperscalers is as unverified as its benchmarks. The next 12 months — shipping hardware and passing independent tests — will determine whether Prometheus is an architectural curiosity or the proof that the AI bottleneck has indeed moved to memory.

What to Watch Before Prometheus Ships: Benchmarks, Drop-In Compatibility and Early Orders

For enterprises, infrastructure buyers and investors watching the AI hardware race, Prometheus is a claim to test, not a product to plan around — yet. The useful moves are specific:

• Treat every headline number as vendor data until third-party benchmarks exist: the 50x memory, 1.7x bandwidth and 25-rack equivalence figures are Majestic's own, and nothing has shipped.
• Test the drop-in promise, not just the specs. Majestic says GPU-trained models run unchanged via PyTorch, vLLM and OpenAI's Triton; if that holds on your own workloads, switching costs collapse, and if it does not, the memory advantage is irrelevant.
• Watch for the first independent signals: the company claims orders from enterprises, neoclouds and hyperscalers, and a named early adopter would carry far more weight than a spec sheet.
• Track the coherence problem against the timeline: a 128TB pool implies roughly 64,000 LPDDR6 dies and more than 100 chiplets in one server, so how Majestic validates that configuration — or starts with the smaller 8TB build — is the tell on execution risk.
• For Nvidia-watchers, read this as one more pressure line — optical chips, edge silicon and open networking included — even though a successful Prometheus would squeeze low-end inference economics before it threatens DGX-class training systems.