Nvidia H100 NVL (94GB) AI and HPC GPU

The H100 NVL is NVIDIA's answer to the 80GB ceiling problem in their own H100 lineup. The H100 SXM delivers 80GB per card on an HGX baseboard, which is industry-leading for training. But for inference on 70B+ parameter models at FP16 (140GB+ memory footprint), a single 80GB card falls short. The NVL solves this by pairing two PCIe H100 cards via three NVLink bridges into a unified 94GB HBM3 pool that the software stack sees as a single memory space. Architecture per card: GH100 GPU at TSMC 4nm with 14,592 CUDA cores (132 SMs of the full 144 enabled), 456 fourth-gen Tensor Cores supporting FP64, TF32, FP16, BF16, FP8, and INT8 precision with the Transformer Engine that dynamically selects optimal precision per layer during inference. 47GB HBM3 per card at approximately 1,979 GB/s bandwidth. Per NVLink pair: 94GB unified at 3,938 GB/s combined, connected at 600 GB/s bidirectional NVLink bandwidth between the two cards. The Transformer Engine is the H100's defining feature versus the A100. It automatically manages mixed-precision computation across FP8 and FP16 per neural network layer, delivering up to 4x training throughput and 30x inference throughput on transformer-based models compared to the A100. Production LLM serving on H100 NVL pairs generates 2x to 3x more tokens per second per dollar than A100 80GB deployments for models in the 30B to 70B parameter range. TDP is 400W per card as the default power mode, with the PCIe 16-pin cable supporting configuration between 200W and 600W per card. At default 400W per pair (800W total), the NVL pair consumes roughly the same power as one H100 SXM at 700W but delivers 94GB versus 80GB with PCIe-slot infrastructure simplicity. MIG capability creates up to 7 fully isolated instances per card (14 total across the pair) with dedicated memory, cache, and compute. For multi-tenant inference serving, this granularity is valuable: serve different customers or models on isolated GPU slices with guaranteed QoS. The NVL fits standard PCIe server platforms. Any server with two adjacent PCIe Gen 5 x16 slots, physical clearance for the NVLink bridge assembly (three bridges spanning both cards), and adequate airflow for 800W combined passive cooling. Supermicro, Dell PowerEdge, HPE ProLiant, and Lenovo ThinkSystem platforms all document H100 NVL compatibility. No HGX baseboard required. Compared to MillionMiner's other offerings. Against the H100 SXM 80GB (separate listing): the SXM delivers higher per-GPU bandwidth (3,350 GB/s) and NVSwitch connectivity for multi-GPU training, but requires HGX baseboard infrastructure. Against the H200 NVL 141GB: the H200 doubles memory capacity with newer HBM3e for operators who need even more VRAM headroom. Against the RTX PRO 6000 96GB ($10,000 to $13,000): the RTX PRO 6000 offers newer Blackwell architecture with higher FP32 TFLOPS but lacks HBM bandwidth, the Transformer Engine, and the proven Hopper data center ecosystem.