NVIDIA H100 SXM5 80GB HBM3 Hopper Tensor Core GPU for AI Training

The H100 SXM exists because distributed AI training has a bandwidth problem that PCIe cannot solve. Training a 70B parameter model across multiple GPUs requires each GPU to exchange gradient updates with every other GPU after every forward and backward pass. On PCIe Gen 5 at 128 GB/s, those gradient exchanges become the bottleneck long before the GPUs run out of compute capacity. NVLink 4.0 at 900 GB/s per GPU (7x PCIe) and NVSwitch connecting all 8 GPUs in a node at full bandwidth eliminates that bottleneck. This is why every serious AI training deployment runs SXM, not PCIe. Full GH100 die specifications. 80 billion transistors at TSMC 4nm. 16,896 CUDA cores across 132 SMs (full die enablement on SXM versus 114 SMs on PCIe). 528 fourth-gen Tensor Cores supporting FP64, TF32, FP16, BF16, FP8, and INT8 with the Transformer Engine. 132 third-gen RT Cores. 80GB HBM3 on 5,120-bit bus at 3,350 GB/s bandwidth. 50MB L2 cache. The SXM5 form factor delivers approximately 30 percent more TFLOPS than the PCIe variant (67 versus 51 TFLOPS FP32) due to higher clock speeds enabled by the 700W power budget and HGX thermal infrastructure. Compute throughput at every precision tier. FP64: 34 TFLOPS (67 TFLOPS Tensor). FP32: 67 TFLOPS. TF32 Tensor: 989 TFLOPS with sparsity. FP16/BF16 Tensor: 1,979 TFLOPS with sparsity. FP8 Tensor: 3,958 TFLOPS with sparsity. INT8 Tensor: 3,958 TOPS with sparsity. The FP8 figure is the one that matters for transformer training: 3,958 TFLOPS with automatic precision management via the Transformer Engine means the H100 SXM delivers approximately 4x the training throughput of an A100 SXM on GPT-class models. NVLink and NVSwitch architecture. Each H100 SXM connects to the NVSwitch fabric via 18 NVLink 4.0 links providing 900 GB/s bidirectional bandwidth. NVSwitch provides all-to-all connectivity: any GPU can communicate with any other GPU in the same node at full 900 GB/s without routing through a CPU or PCIe bus. An 8-GPU HGX H100 node delivers 7.2 TB/s of aggregate NVLink bandwidth across all GPUs. For multi-node scaling, NVIDIA Quantum-2 NDR InfiniBand at 400 Gb/s per port extends the fabric beyond single nodes. DGX H100 versus HGX H100. DGX H100 is NVIDIA's turnkey 8-GPU system ($250,000 to $400,000 class) including CPUs, memory, storage, networking, and software stack. HGX H100 is the GPU baseboard module that server OEMs (Supermicro, Dell, HPE, Lenovo) integrate into their own server platforms. Both use the same 8x H100 SXM GPU configuration with NVSwitch. The HGX path offers more flexibility on CPU, storage, and networking choices. The SXM versus NVL versus PCIe decision framework. SXM (this product): maximum per-GPU performance, 8-GPU NVSwitch scaling, 700W TDP, requires HGX baseboard, optimized for distributed training. NVL (H100 NVL 94GB, separate MillionMiner listing): PCIe paired cards, 94GB unified memory, fits standard servers, optimized for large-model inference. PCIe (standard H100 80GB PCIe): single card at 350W, standard server slots, lower cost, limited to 2-GPU NVLink, suitable for single-GPU inference and fine-tuning. Choose SXM when training throughput and multi-GPU scaling efficiency are the priority. Choose NVL or PCIe when inference or infrastructure simplicity matters more. MIG on the H100 SXM creates up to 7 isolated instances at 10GB each. The most common production patterns per Spheron: 7x 1g.10gb for multi-tenant inference of small models, or 2x 3g.40gb for two simultaneous 13B model servers. Each MIG instance appears as a separate GPU device to the OS with hardware-enforced isolation. Confidential computing via Trusted Execution Environment (TEE) protects data and model weights during processing. This is a hardware-level security feature for compliance-sensitive AI deployments in healthcare (HIPAA), finance (SOC 2), and government (FedRAMP) where data cannot be exposed to the infrastructure operator. Up to 700W TDP configurable. Requires liquid cooling or high-airflow server chassis engineering. Standard air cooling is insufficient for sustained 700W operation. NVIDIA DGX H100 uses direct liquid cooling. HGX H100 configurations from Supermicro and Lenovo offer both air and liquid options depending on thermal budget.