When Nvidia confirmed a $20 billion licensing deal with Groq in late 2025, a lot of people treated it as a surprising price tag. It was not surprising at all. It was Nvidia executing the same playbook it has run for over a decade: identify a credible challenger at a specific layer of the AI stack, acquire the threat, and absorb the capability.
Nvidia does not just make GPUs. It controls every layer of the AI compute stack — silicon, interconnect, networking, software, and turnkey cluster delivery — and the Groq deal is the clearest recent proof of that strategy in action. This page gives you the strategic map; the seven articles give you the depth to act on it.
What did Nvidia actually buy from Groq — and why does it matter?
Nvidia did not acquire Groq outright. It entered a non-exclusive licensing agreement — reportedly worth $20 billion — for Groq’s LPU (Language Processing Unit) IP, compiler toolchain, and silicon design. Groq founder Jonathan Ross and president Sunny Madra joined Nvidia, and the LPU architecture was confirmed as the Groq 3 LPX chip in the Vera Rubin platform in March 2026. GroqCloud continues to operate independently, though its long-term strategic position is now uncertain.
The non-exclusive licence structure was deliberate — it avoided triggering a formal antitrust merger review. Analysts at Hedgeye called it “essentially an acquisition without being labelled one”. The Groq LPU’s static scheduling and SRAM-based design eliminated GPU inference bandwidth bottlenecks — that architecture is now part of Nvidia’s platform. The remaining inference-specialist challengers, Cerebras and SambaNova, are independent but the deal has weakened their positioning. Understanding what was licensed in the Groq deal and why it cost $20 billion is essential context for evaluating Nvidia’s inference roadmap.
The Nvidia Groq Deal Explained covers the deal mechanics, LPU architecture, antitrust structure, and GroqCloud’s future.
How does Nvidia control every layer of the AI stack?
Nvidia’s competitive position is not a single product — it is a stack. The company owns the GPU silicon (H100, H200, Blackwell, Vera Rubin), within-node interconnect (NVLink), multi-node networking (InfiniBand via the 2020 Mellanox acquisition), and software (CUDA, NCCL, cuDNN). It delivers the full stack as a turnkey cluster through DGX SuperPOD. Each layer reinforces the others, making partial replacement difficult and full replacement extraordinarily costly. The full picture of how Nvidia controls every layer of the AI stack — from the Mellanox acquisition through to the Groq deal — reveals a consistent competitive neutralisation playbook spanning more than a decade.
The software moat — CUDA — is the deepest and least visible part. More than 4 million developers and 40,000 companies are integrated into Nvidia’s CUDA ecosystem. Nineteen years of accumulated investment in cuDNN, cuBLAS, NCCL, and PyTorch optimisation means switching is not a hardware decision — it is an engineering and retraining problem. The NCCL networking layer compounds this by creating single-vendor pressure across multi-node clusters, making mixed-GPU deployments costly in ways that go well beyond hardware procurement.
Nvidia’s Vertical Integration Playbook maps the complete stack and walks through the competitive playbook connecting Mellanox to Groq.
That stack has cracks, though — and some of them are becoming visible on enterprise balance sheets.
DGX SuperPOD vs Huawei CloudMatrix 384 — who is winning the global AI infrastructure arms race?
Nvidia’s DGX SuperPOD — now powered by the Vera Rubin NVL72 — delivers 28.8 exaflops from 576 GPUs. Huawei’s CloudMatrix 384, powered by 384 Ascend 910C accelerators, is gaining traction in China — Baidu among the hyperscalers adopting it under export control restrictions. U.S. export controls have bifurcated the global AI infrastructure market into two distinct tiers, and the performance gap is real and widening. For enterprises making multi-year capex decisions, the DGX SuperPOD vs Huawei CloudMatrix 384 — the global arms race analysis provides the geopolitical and technical framing needed to evaluate Blackwell-to-Vera-Rubin upgrade timing.
The GB200 delivers over 3x the BF16 performance of the Ascend 910C. CFR analysis estimates Huawei delivers roughly 5% of Nvidia’s aggregate AI computing power globally even under optimistic assumptions. For enterprises in Western markets, the bifurcation matters indirectly — reduced global competition maintains Nvidia’s pricing leverage. For enterprises with China operations, it creates direct compliance and data residency implications.
DGX SuperPOD vs CloudMatrix 384 covers the technical comparison, export control mechanics, and the Blackwell-to-Vera-Rubin upgrade context.
Where is Nvidia’s moat weakest right now?
Nvidia’s moat shows its weakest points at the networking layer and in GPU utilisation efficiency. HetCCL — a cross-vendor collective communications library using RDMA — directly challenges NCCL’s role in enforcing single-vendor GPU deployments, achieving up to 97% efficiency relative to homogeneous NCCL runs in research testing. Separately, nearly half of enterprises are wasting millions on underutilised GPU capacity, a sign that infrastructure bloat is increasingly visible on balance sheets. The detailed analysis of GPU waste, HetCCL, and the cracks in Nvidia’s moat covers the audit methodology for identifying where your organisation is losing value right now.
HetCCL is not yet in production, but it represents the most technically grounded near-term challenge to networking-layer lock-in. If adopted at scale, it enables mixed Nvidia and AMD Instinct deployments without the communication overhead that currently makes heterogeneous clusters economically unattractive. GPU underutilisation is the more immediate problem: a single H100 costs upward of $40,000 and idle 25% of the time represents $10,000 in wasted value annually.
GPU Underutilisation, HetCCL, and the Emerging Cracks in Nvidia’s Hardware Moat covers the GPU waste audit methodology, HetCCL architecture, and how to build a heterogeneous Nvidia + AMD cluster.
What does AI inference actually cost per token — and what does Vera Rubin change?
Inference now represents 50–65% of enterprise AI compute volume and can account for 80–90% of lifetime AI costs in production deployments. Nvidia claims Vera Rubin delivers a 10x inference token cost reduction versus Blackwell. The independent benchmark picture is more nuanced: the cost reduction is real but varies significantly by workload, model architecture, and utilisation rate. LLMflation — the roughly 10x per year inference cost deflation trend coined by Guido Appenzeller at a16z — means the cost landscape shifts fast regardless of which platform you are on. For organisations managing monthly AI bills in the tens of millions, modelling the true cost per inference token across architectures is the financial analysis that separates informed procurement from vendor-led decision-making.
The “10x” claim compares against Blackwell under optimal workload conditions. The question for multi-year GPU commitments is whether to lock in capacity now or preserve flexibility as the next cost curve develops — because inference economics are shifting fast and monthly AI bills in the tens of millions are already common for high-volume deployments.
Inference Economics covers cost-per-token definitions, Nvidia’s 10x claim interrogated with benchmark data, and a CFO-ready inference ROI framework.
How deep is CUDA lock-in and what would it take to escape it?
CUDA lock-in operates at two distinct layers: the software/framework layer (CUDA libraries, toolchain, developer familiarity) and the networking layer (NCCL). They require different solutions. At the software layer, AMD ROCm 7 now delivers up to 3.5x better inference performance than previous versions, and vendor-agnostic frameworks like Triton and PyTorch’s hardware abstraction layer provide partial portability. At the networking layer, NCCL dependency only drops meaningfully when HetCCL or equivalent tooling reaches production adoption. The practical guide to understanding and reducing your CUDA lock-in starts with distinguishing which layer of dependency is actually constraining your stack — because the solutions are different.
The practical question for your production stack is not “is our code portable?” but “which CUDA libraries do we depend on, and what is the ROCm parity status for each?” CUDA typically outperforms ROCm by 10–30% in compute-intensive workloads, though auditing the actual dependency surface typically reveals it is more portable than the general “CUDA lock-in” label implies. At the networking layer, the constraint is separate: NCCL’s tight coupling with Nvidia hardware keeps cluster architecture single-vendor until cross-vendor tooling matures.
CUDA Lock-in Unpacked covers the two-layer distinction, switching cost breakdown, ROCm maturity, and a CUDA dependency audit framework.
Given all of this, what should your GPU strategy actually be?
GPU procurement decisions made in 2025–2026 carry significant weight. Vera Rubin represents peak Nvidia pricing power — the moment when demand most exceeds supply. The right starting point is an audit, not a purchase: run a GPU utilisation assessment before any new capex commitment. What you find will determine whether the real question is “what to buy” or “how to better use what you already have.” The structured approach in a practical GPU procurement decision framework sequences these steps — audit, analyse lock-in exposure, evaluate diversification paths, structure contracts — so the decision is based on data rather than vendor positioning.
Break-even analysis suggests purchasing makes sense only above 60–70% continuous utilisation. Cloud GPU pricing has also dropped substantially — AWS cut H100/H200 prices 44% in June 2025. Cloud access preserves optionality when inference economics are shifting and when multi-year on-premises commitments would lock in at peak pricing. Vendor diversification is viable but requires honest sequencing: audit utilisation first, identify your CUDA dependency surface, then evaluate whether AMD Instinct plus HetCCL makes sense for your workload mix.
GPU Procurement Strategy in 2025–2026 covers the utilisation audit, the stay-all-in versus diversify decision framework, contract optionality negotiation, and a board/CFO communication template.
Recommended Reading Order
This cluster works as a standalone strategic guide or as seven individual deep-dives. The recommended sequence follows the decision-support logic.
If you want immediate action first: Start with GPU Procurement Strategy in 2025–2026 for the decision framework, then return to the foundational articles as needed.
If you are building from context: Follow the full sequence below.
- The Nvidia Groq Deal Explained — the news hook; foundational context for the entire series
- GPU Underutilisation, HetCCL, and the Emerging Cracks in Nvidia’s Hardware Moat — the cost problem driving urgency; highest immediate actionability
- Nvidia’s Vertical Integration Playbook — the strategic logic of why Nvidia’s position is difficult to displace
- Inference Economics — the financial modelling layer; the CFO/board communication framework
- DGX SuperPOD vs CloudMatrix 384 — the competitive intelligence; geopolitical bifurcation context
- CUDA Lock-in Unpacked — the deepest technical treatment; the software moat audit
- GPU Procurement Strategy in 2025–2026 — the synthesis destination; the decision framework that draws on all of the above
