No frontier lab is running production training on GB200 NVL72 today. Reliability failures and immature software have kept Nvidia's H100, H200, and Google TPUs as the only hardware completing frontier-scale training runs at scale. SemiAnalysis benchmarked over 2,000 H100 GPUs, measuring model flops utilization, cost per 1M tokens trained, and energy consumption normalized to US household annual usage. The GB200 NVL72 rack costs $3.9M all-in for a typical hyperscaler, versus $250K per H100 server, and its per-GPU capital cost runs 1.6x to 1.7x higher than H100 once networking and storage are included.
The core question this report answers is whether GB200 NVL72's performance-per-dollar advantage survives after downtime and lost engineering time are factored into TCO. Spoiler: the math gets complicated fast. The benchmarks use Nvidia's DGXC reference scripts on the EOS cluster with 8x400 Gbit/s InfiniBand, the same numbers Neoclouds use to negotiate SLAs and qualify for Nvidia's Exemplar Cloud designation. GB200 results cover Llama 4 400B MoE and DeepSeek 670B MoE and are directly compared against the H100 baseline. The H100 scaling analysis runs from 128 to 2,048 GPUs across multiple Nvidia software versions, which is where the report gets genuinely granular.
SemiAnalysis expects GB200 NVL72 software to improve materially before year-end, and frontier model architectures are already being co-designed around its larger scale-up world size. That could shift the TCO calculus significantly. But the reliability problems are real and unresolved, and Nvidia needs closer coordination with partners to fix them. The full report is worth reading for the per-token energy and cost tables alone, and for understanding exactly how much reliability drag erases a hardware performance advantage before a single training run completes.
[READ ORIGINAL →]