NVIDIA's Vera Rubin Platform Promises to Cut AI Costs by 90%. Here's What That Actually Means.

Six chips, one rack, 3.6 exaflops

The Vera Rubin platform isn't a single chip. It's six co-designed components built to operate as an integrated system: the Vera CPU, Rubin GPU, NVLink 6 Switch, ConnectX-9 SuperNIC, BlueField-4 DPU, and Spectrum-6 Ethernet Switch. NVIDIA calls it "extreme co-design" — every chip in the stack was engineered to work with the others rather than assembled from off-the-shelf parts. [1][2] The flagship configuration is the NVL72 rack: 72 Rubin GPUs and 36 Vera CPUs connected in a single all-to-all NVLink 6 fabric. The numbers are staggering. Each Rubin GPU delivers 50 PFLOPS of NVFP4 inference — a 5x improvement over Blackwell. The rack collectively delivers 3.6 exaFLOPS of inference performance and 2.5 exaFLOPS for training, with 20.7 TB of HBM4 memory and 1.6 PB/s of bandwidth. [1][2][3] The Vera CPU replaces the Grace CPU from Blackwell. Its 88 Olympus cores use Armv9.2 architecture and connect to Rubin GPUs through NVLink-C2C at 1.8 TB/s coherent bandwidth. NVIDIA positions it specifically as the coordination engine for agentic workloads — AI systems that don't just generate text but take actions, call tools, and manage complex multi-step workflows. [2][3] For mixture-of-experts models — the architecture behind DeepSeek and many current frontier models — NVIDIA claims you need one-quarter as many GPUs to train compared to Blackwell. That's not a marginal improvement. That's a fundamental shift in the economics of frontier model development. [1][2]

The Groq acquisition changes the inference game

Perhaps the most consequential announcement wasn't the Rubin GPU itself but what NVIDIA did with Groq. Following its acquisition of Groq, NVIDIA introduced the LPX rack — a new system housing 256 Groq 3 LPUs (Language Processing Units), each containing approximately 500 MB of stacked SRAM. The rack provides roughly 128 GB of aggregate on-chip SRAM and 640 TB/s of scale-up bandwidth. [2][3] The architecture splits inference work between two fundamentally different processors. The Rubin GPU handles prefill and attention operations — the memory-intensive part of inference where the model processes your input and builds context. The Groq LPU handles decode — the sequential, latency-sensitive part where the model generates tokens one at a time. [2] Think of it this way: the GPU is the factory that understands your question, and the LPU is the factory that delivers the answer at maximum speed. Each is optimized for a fundamentally different compute pattern. The contrast is stark. A Rubin GPU provides 288 GB of HBM4 at 22 TB/s bandwidth — lots of memory, fast access. The Groq LPU trades capacity for raw bandwidth: 500 MB of SRAM at 150 TB/s per chip. That's nearly 7x more bandwidth per chip, at the cost of vastly less storage. For token decode, which is bottlenecked by memory bandwidth rather than capacity, the LPU architecture makes more sense than throwing more GPU compute at the problem. [2] NVIDIA claims no CUDA changes are required. The LPU operates transparently as a decode accelerator within the existing software stack. Many of Groq's founders and engineers have joined NVIDIA, and early reports suggest the integration is going smoothly. [2]

The $1 trillion question: does cheaper compute mean less spending?

Here's where the story gets interesting for anyone thinking about the AI economy rather than individual chip specs. Jensen Huang projected that combined visible orders for Blackwell and Vera Rubin will exceed $1 trillion by 2027. Last year at GTC, that forecast was $500 billion through 2026. A year later, the number doubled with the time window extended by just one year. [4][5] This seems counterintuitive. If NVIDIA keeps making inference cheaper — 10x cheaper per generation, if you believe the marketing — shouldn't companies be spending less on compute? The answer is no, and it's one of the most important dynamics in the technology industry right now. In March 2023, when GPT-4 launched, API costs ran about $36 per million tokens. By mid-2024 with GPT-4o, the price dropped to around $7. By the end of 2025, the actual price had fallen below $2 per million tokens. That's a 94% decrease in two years. [4] Yet the combined annual capital expenditure of Amazon, Alphabet, Meta, and Microsoft increased from $154 billion in 2023 to $416 billion in 2025 — a 170% increase. Google alone surged from $32 billion to $91.5 billion. The four core cloud players could exceed $660 billion in 2026, a further 60% year-over-year jump. [4][5] This is the Jevons Paradox in real time. When the steam engine became more fuel-efficient in the 19th century, coal consumption didn't decrease — it exploded, because efficiency made steam power economical for applications that were previously too expensive. The same thing is happening with AI inference. As API prices plummeted, enterprises didn't save budget. They started deploying AI into customer service, code review, content generation, search re-ranking, ad bidding, and dozens of other use cases that were previously uneconomical. [4] Every 10x reduction in token cost opens up workloads that didn't exist before: longer reasoning chains, larger context windows, higher request volumes, applications requiring multiple model calls per user interaction. The expansion of demand far exceeds the rate of cost decline. [3][4] NVIDIA's data center revenue tells the story cleanly. From $10.6 billion in FY2022 to $115.2 billion in FY2025 — an 11x increase in three fiscal years. For comparison, after the iPhone launched in 2007, it took Apple about six years to achieve a similar order-of-magnitude revenue increase. [4]

What the fine print says

The 10x cost reduction claim needs careful reading, and NVIDIA isn't going out of its way to provide it. First, the 10x figure is specifically about NVFP4 inference at rack scale. NVFP4 is a format that not every model uses, requiring appropriate quantization pipelines. The standardized FP8 dense TFLOPS comparison shows Vera Rubin at 4x over Blackwell and 8x over H100 — impressive, but not 10x. The full cost reduction comes from system-level factors: Transformer Engine optimization, FP4 precision, larger batch inference, and architectural improvements. [2][4] Second, the NVL72 is a liquid-cooled, custom-networking rack. Operators without liquid-cooled data center infrastructure face significant retrofitting costs that aren't included in NVIDIA's calculations. The rack requires 45°C hot water cooling and operates in a completely fanless, tubeless design. That's elegant engineering, but it's also a barrier for operators who built their facilities around air-cooled or hybrid systems. [1][2] Third, Blackwell isn't old. It launched in 2025 and is still being deployed at scale. Organizations that committed to Blackwell purchases 12 months ago — and many did, given the GPU supply crunch — aren't in a position to swap when Rubin ships in the second half of 2026. The upgrade cycle creates real transition costs that the marketing materials don't address. [1] The Feynman preview that Huang tacked onto the end of the keynote — a 2028 architecture on TSMC's 1.6nm process, designed for massive key-value cache storage and long-term memory in reasoning models — was positioning, not product. No performance numbers were given. It served primarily to remind the audience that whatever they buy today will be superseded in two years. [1]

The real competitive picture

NVIDIA holds more than 75% of the AI chip market, and high pricing with a near-monopoly structure naturally pushes cloud providers to seek alternatives. Broadcom has secured large orders from Anthropic and OpenAI. Google continues investing in its custom TPU architecture. Multiple hyperscalers are pursuing in-house chip designs. Even with Vera Rubin, consensus expects NVIDIA's market share to drift lower over time. [5] NVIDIA's response has been strategic investment to lock in demand: a reported $30 billion investment in OpenAI and $10 billion in Anthropic, both tied to deployment commitments. These aren't passive financial bets — they're supply chain locks that ensure the largest AI labs remain NVIDIA customers through the next hardware cycle. [5] The stock tells an interesting story too. NVIDIA shares have been range-bound between $170 and $200 for the past six months despite rising downstream capex and repeated earnings beats. The market's concern isn't current performance — it's sustainability. With Meta's 2026 capex guided to $115-135 billion, pushing capex-to-revenue above 50%, there's limited room for hyperscalers to keep expanding AI infrastructure spending indefinitely. [5] On GTC's closing day, NVIDIA's stock rose 4.3%. The market chose to believe Huang's vision. Whether that belief holds through 2027 depends on something no chip specification can guarantee: whether the AI applications being built on this infrastructure actually generate enough revenue to justify the investment.

The bottom line

Vera Rubin is genuinely impressive hardware. A 5x inference improvement and 4x reduction in GPUs needed for training are real advances that will meaningfully change the economics of AI deployment. The Groq integration is architecturally clever and addresses a real bottleneck in token generation speed. The dedicated Vera CPU rack for reinforcement learning environments solves a compute problem that most coverage doesn't even mention. But the bigger story isn't the chip. It's the economic flywheel that NVIDIA has built. Cheaper inference doesn't shrink the market — it expands it. Every cost reduction unlocks new use cases, which drive more demand, which justifies more infrastructure spending, which buys more NVIDIA hardware. That loop has driven an 11x revenue increase in three years and shows no signs of breaking. The question for 2027 isn't whether Vera Rubin will be fast. It's whether the AI applications consuming all these tokens will generate enough value to keep the flywheel spinning — or whether at some point, the cost of building AI factories outpaces the revenue they produce. Jensen Huang is betting $1 trillion that it won't. The rest of the industry is betting alongside him.