一张清晰的表格,展示了当前GPU在AI推理速度、VRAM和训练吞吐量方面的性能表现——涵盖从工作站显卡到数据中心Blackwell硬件。参考编号,帮助您为您的工作负载选择合适的GPU。
在该领域内最高的AI推理指数。以下每个数字都是以RTX 3090为100进行的比例。
按 VRAM 过滤,排序任意列,点击任意两行的 +,即可将它们直接送入下方的对战场。
| 请选择 | GPU ↕ | VRAM ↕ | 人工智能指数 ▼ | FP16 TFLOPS ↕ | 培训 ↕ | 最佳选择 |
|---|---|---|---|---|---|---|
| 96 GB |
403
|
— | — | Largest LLMs without quantization, multi-tenant serving | ||
|
RTX Pro 5000 Blackwell
Flagship
|
48 GB |
285
|
— | — | High-throughput production inference | |
| 48 GB |
238
|
— | — | Best price-to-performance for serious inference | ||
| 32 GB |
207
|
419.1 | — | Fastest single-GPU inference & image generation | ||
| 40 GB |
152
|
312.0 | 1,396 img/s | Proven data-center training workhorse | ||
| 32 GB |
146
|
— | — | Balanced training + inference workstation | ||
|
RTX 4080 Super Pro
Mid
|
32 GB |
139
|
— | — | Mid-range inference & light fine-tuning | |
| 24 GB |
133
|
165.2 | 1,301 img/s | Best all-round price-to-performance card | ||
| 24 GB |
100
|
35.6 | 905 img/s | Budget-friendly entry into serious AI work | ||
|
V100 32GB
Entry
|
32 GB |
84
|
125.0 | — | Legacy data-center training | |
| 24 GB |
83
|
— | — | Compact workstation inference | ||
|
V100 16GB
Entry
|
16 GB |
62
|
125.0 | 833 img/s | Light training / dev environments | |
|
RTX 4070 Ti Super
Mid
|
16 GB |
56
|
— | — | Hobbyist projects & prototyping | |
|
RTX A4000
Entry
|
16 GB |
51
|
19.2 | — | Entry-level dev work, small models | |
| 96 GB | — | — | — | Power-efficient Max-Q variant for dense workstation builds | ||
| 96 GB | — | — | — | Boxed workstation edition, same silicon as Server Edition | ||
|
B200 SXM
Flagship
|
180 GB | — | 2,250.0 | — | Largest-scale multi-GPU training clusters | |
| 192 GB | — | 1,750.0 | — | Massive-context LLM training & serving | ||
|
Instinct MI350X
Flagship
|
288 GB | — | 2,306.9 | — | Massive VRAM pool for huge models, AMD ROCm stacks | |
|
Instinct MI325X
Flagship
|
256 GB | — | 1,307.4 | — | Large-model inference with huge memory headroom | |
| 141 GB | — | 989.5 | — | High-bandwidth-memory LLM serving at scale | ||
| 141 GB | — | 989.5 | — | PCIe/NVLink-bridge alternative to SXM for air-cooled racks | ||
|
Gaudi 3
Flagship
|
128 GB | — | 1,835.0 | — | Intel-based large-scale training alternative | |
|
Instinct MI300X
Flagship
|
192 GB | — | 1,307.4 | — | Single-GPU serving of very large open-weight models | |
| 80 GB | — | 989.5 | — | Industry-standard large-model training & inference | ||
|
RTX 6000 Ada
Flagship
|
48 GB | — | — | — | Top-tier workstation for training & rendering | |
| 32 GB | — | 419.1 | — | Blower-style RTX 5090 for dense multi-GPU builds | ||
| 24 GB | — | — | — | China-compliant RTX 5090 variant, cost-efficient rigs | ||
| 94 GB | — | 835.5 | — | Dual-GPU NVLink inference for very large models | ||
|
H100 PCIe
High-end
|
80 GB | — | 756.5 | — | PCIe-server LLM training & inference | |
| 80 GB | — | 756.5 | — | China-compliant H100 equivalent, reduced NVLink bandwidth | ||
| 80 GB | — | 312.0 | — | Export-compliant A100 alternative for restricted regions | ||
| 80 GB | — | 312.0 | — | Proven large-batch training workhorse | ||
| 40 GB | — | 312.0 | — | China-compliant A100 equivalent for large-scale training | ||
| 48 GB | — | 733.0 | — | Mixed training + inference + rendering server card | ||
| 32 GB | — | — | — | High-end workstation training & inference | ||
|
Instinct MI250X
High-end
|
128 GB | — | 383.0 | — | Large-memory AMD training clusters | |
|
RTX 4070 Ti
Mid
|
12 GB | — | 40.1 | — | Strong price-to-performance for local inference | |
|
L4
Mid
|
24 GB | — | 242.0 | — | Low-power inference & video AI at scale | |
| 48 GB | — | 181.1 | — | Graphics + AI mixed workloads on one card | ||
| 64 GB | — | 181.0 | — | PCIe AMD training/inference card | ||
|
Instinct MI100
Mid
|
32 GB | — | 184.6 | — | Earlier-gen AMD CDNA training/inference card | |
| 48 GB | — | 119.5 | — | Efficient server inference with large VRAM | ||
|
Radeon RX 7900 XTX
Mid
|
24 GB | — | 122.8 | — | AMD consumer flagship for local inference | |
|
RTX 4080 Super
Mid
|
16 GB | — | 104.4 | — | Strong mid-range inference & gaming/AI hybrid use | |
|
Radeon RX 7900 XT
Mid
|
20 GB | — | 103.2 | — | AMD mid-high consumer inference card | |
|
RTX 4080
Mid
|
16 GB | — | 97.5 | — | Solid mid-range inference workstation card | |
| 48 GB | — | 32.6 | — | Large-VRAM legacy workstation for bigger models | ||
|
A40
Mid
|
48 GB | — | 149.7 | — | Large-VRAM data-center inference card | |
| 24 GB | — | 165.0 | — | Efficient shared-server inference | ||
| 24 GB | — | 125.0 | — | Cost-efficient cloud inference card | ||
| 24 GB | — | 39.6 | — | Workstation training with large VRAM headroom | ||
|
RTX A6000
Mid
|
48 GB | — | 38.7 | — | Large-VRAM workstation for bigger models | |
|
RTX 3090 Ti
Mid
|
24 GB | — | 40.0 | — | High-VRAM consumer card for local models | |
|
RTX 3080 Ti
Mid
|
12 GB | — | 34.1 | — | Strong mid-range gaming/AI hybrid card | |
|
Arc A770
Mid
|
16 GB | — | 39.4 | — | Budget Intel card for OpenVINO/local inference | |
|
T4
Entry
|
16 GB | — | 65.0 | — | Very low-power cloud inference card | |
|
RTX A5000
Entry
|
24 GB | — | 27.8 | — | Reliable mid-VRAM workstation card | |
| 20 GB | — | 23.6 | — | Small-batch workstation training | ||
| 20 GB | — | 26.7 | — | Compact single-slot workstation inference | ||
| 20 GB | — | 38.4 | — | Small-form-factor workstation inference | ||
|
RTX 3080
Entry
|
10 GB | — | 29.8 | — | Affordable gaming/AI hybrid card | |
| 16 GB | — | 24.0 | — | Low-power small-form-factor inference | ||
| 16 GB | — | 22.3 | — | Legacy Turing workstation card, small models | ||
| 16 GB | — | 138.0 | — | Media/video-AI inference card | ||
|
RTX 3070
Entry
|
8 GB | — | 20.3 | — | Cheapest realistic entry point for small models | |
|
RTX 4060 Ti 16GB
Entry
|
16 GB | — | 22.1 | — | Budget 16GB card for local LLM experiments | |
| 12 GB | — | 8.0 | — | Ultra-low-power dev / edge inference | ||
|
RTX 4070
Entry
|
12 GB | — | 29.1 | — | Affordable current-gen dev/inference card | |
|
Radeon RX 7800 XT
Entry
|
16 GB | — | 74.4 | — | AMD mid-tier card with generous VRAM | |
| 16 GB | — | 18.7 | — | Legacy Pascal data-center training card | ||
|
RTX 4060
Entry
|
8 GB | — | 15.1 | — | Cheapest current-gen entry inference card | |
|
Instinct MI50
Entry
|
32 GB | — | 26.8 | — | Cheap secondhand AMD VRAM for local inference | |
|
Radeon RX 7600
Entry
|
8 GB | — | 43.5 | — | Budget AMD card for light local inference | |
| 8 GB | — | — | — | Ultra-low-power display/dev card, not AI-focused | ||
| 5 GB | — | — | — | Legacy budget workstation card, minimal AI use | ||
| 4 GB | — | 2.2 | — | Display-output card, not suitable for real AI workloads | ||
| 4 GB | — | — | — | Ultra-budget entry GPU, not recommended for AI workloads |
显示 78 张 / 78 张卡片
多GPU服务器的评判标准不同于单卡 —— 关键在于其内部每个GPU的总计算能力。以下的总计算能力是根据上表中每个卡的FP16 TFLOPS数字乘以GPU数量得出,以确保服务器在相同的基准范围内。 48 系统目前已列入我们的AI硬件目录 — 点击任何名称查看完整规格和价格。
| 服务器 | GPU 配置 | 总 VRAM | 总计算能力(FP16 TFLOPS) | 价格 |
|---|---|---|---|---|
|
NVIDIA DGX B200
DGX / Desktop
|
8x Blackwell GPUs, 1,440GB | 1,440 GB | 18,000.0 | $515,000 |
|
Gigabyte G893-SD1-AAX5
Rack Server
|
8x B200 SXM (HGX B200) | 1,440 GB | 18,000.0 | 索取报价 |
|
Gigabyte G893-ZD1-AAX5
Rack Server
|
8x B200 SXM (HGX B200) | 1,440 GB | 18,000.0 | 索取报价 |
|
HGX B200 8-GPU Baseboard
Baseboard
|
8x B200, 1,536GB HBM3e | 1,536 GB | 18,000.0 | 索取报价 |
|
NVIDIA DGX H100
DGX / Desktop
|
8x H100 SXM5, 640GB | 640 GB | 7,916.0 | $6,700 |
|
NVIDIA DGX H200
DGX / Desktop
|
8x H200 SXM5, 1,128GB | 1,128 GB | 7,916.0 | 索取报价 |
|
ASRock Rack 6U8X-EGS2
Rack Server
|
8x H200 SXM | 1,128 GB | 7,916.0 | $2,500 |
|
ASUS ESC N8-E11
Rack Server
|
8x H200 SXM (HGX) | 1,128 GB | 7,916.0 | $2,600 |
|
Dell PowerEdge XE9680
Rack Server
|
8x H100 SXM | 640 GB | 7,916.0 | $22,000 |
|
Gigabyte G593-SD2-AAX1
Rack Server
|
8x H100 SXM, 640GB | 640 GB | 7,916.0 | $5,600 |
|
Gigabyte G593-ZD2 (H100 Barebone)
Rack Server
|
8x H100 SXM5 (barebone) | 640 GB | 7,916.0 | $45,000 |
|
Gigabyte G593-ZD2 (H200 Barebone)
Rack Server
|
8x H200 SXM5 (barebone) | 1,128 GB | 7,916.0 | 索取报价 |
|
Lenovo ThinkSystem SR675 V3
Rack Server
|
8x H100 SXM | 640 GB | 7,916.0 | $300,000 |
|
Lenovo HGX H200 8-GPU Server
Rack Server
|
8x H200 SXM, 141GB each | 1,128 GB | 7,916.0 | 索取报价 |
|
Quanta S7PH H100
Rack Server
|
8x H100 SXM | 640 GB | 7,916.0 | $280,000 |
|
Quanta S7PH H200 (Barebone)
Rack Server
|
8x H200 SXM (barebone) | 1,128 GB | 7,916.0 | $20,000 |
|
HGX H100 Baseboard (Liquid)
Baseboard
|
8x H100 SXM5, 640GB, liquid-cooled | 640 GB | 7,916.0 | $3,400 |
|
HGX H100 8-GPU Baseboard
Baseboard
|
8x H100 SXM5, 640GB | 640 GB | 7,916.0 | $3,400 |
|
HGX H200 8-GPU Baseboard (Air)
Baseboard
|
8x H200 SXM, 1,128GB, air-cooled | 1,128 GB | 7,916.0 | 索取报价 |
|
HGX H200 8-GPU Baseboard (Liquid)
Baseboard
|
8x H200 SXM, 1,128GB, liquid-cooled | 1,128 GB | 7,916.0 | 索取报价 |
|
NVIDIA DGX H800
DGX / Desktop
|
8x H800 SXM5, 640GB | 640 GB | 6,052.0 | 索取报价 |
|
Supermicro HGX H800 SYS-821GE
Rack Server
|
8x H800 SXM | 640 GB | 6,052.0 | $5,499 |
|
HGX H200 4-GPU Baseboard
Baseboard
|
4x H200 SXM, 564GB | 564 GB | 3,958.0 | $180,000 |
|
Lenovo HGX H200 4-GPU Board
Baseboard
|
4x H200 SXM, liquid-cooled | 564 GB | 3,958.0 | $7,800 |
|
Supermicro SYS-741GE-TNRT
Compact
|
4x H100 PCIe | 320 GB | 3,026.0 | $5,999 |
|
NVIDIA DGX A100
DGX / Desktop
|
8x A100 SXM4, 640GB | 640 GB | 2,496.0 | 索取报价 |
|
Exeton Quasar 640X
Rack Server
|
8x A100 SXM, 640GB NVLink | 640 GB | 2,496.0 | 索取报价 |
|
Gigabyte G492-ZD0 (Used)
Rack Server
|
8x A100 SXM | 640 GB | 2,496.0 | 索取报价 |
|
Supermicro AS-4124GO-NART+
Rack Server
|
8x A100 HGX | 640 GB | 2,496.0 | $13,000 |
|
HGX A100 8-GPU Baseboard (640GB)
Baseboard
|
8x A100 SXM4, 640GB | 640 GB | 2,496.0 | $5,600 |
|
HGX A100 8-GPU Baseboard (320GB)
Baseboard
|
8x A100 SXM4 40GB, 320GB total | 320 GB | 2,496.0 | 索取报价 |
|
NVIDIA DGX Station A100
DGX / Desktop
|
4x A100, 160GB | 160 GB | 1,248.0 | $85,000 |
|
RTX 5090 Full System
Compact
|
Pre-built RTX 5090 AI workstation | 32 GB | 419.1 | $6,000 |
|
ASUS Ascent GX10
DGX / Desktop
|
GB10 Grace Blackwell Superchip | 128 GB | — | 索取报价 |
|
ASUS ESC8000A-E12P
Rack Server
|
Dual EPYC 9004, up to 8x GPU | — | — | 索取报价 |
|
ASUS XA NB3I-E12
Rack Server
|
8x B300 NVL (HGX B300 NVL8) | — | — | $7,999 |
|
Gigabyte H263-S67 (2U 4-Node)
Compact
|
2U 4-node high-density chassis | — | — | $18,000 |
|
NVIDIA DGX Spark
DGX / Desktop
|
GB10 Grace Blackwell Superchip | 128 GB | — | 索取报价 |
| Edge AI module, 64GB | 64 GB | — | 索取报价 | |
|
QuantaGrid D74H-7U
Rack Server
|
8-GPU chassis (barebone) | — | — | 索取报价 |
|
Supermicro AS-8125GS-TNHR (Refurb)
Rack Server
|
8-GPU server (refurbished) | — | — | 索取报价 |
|
Supermicro AS-8126GS-NB3RT
Rack Server
|
8x B300 NVL (HGX B300 NVL8) | — | — | $450,000 |
|
Supermicro GH200 SuperServer
DGX / Desktop
|
NVIDIA GH200 Grace Hopper Superchip (used) | — | — | 索取报价 |
|
Supermicro SYS-111C-NR-G1
Compact
|
1U GPU-ready server | — | — | $5,600 |
|
Supermicro SYS-212H-TN-G1
Compact
|
2U GPU-ready server | — | — | 索取报价 |
|
Supermicro SYS-511E-WR-G1
Compact
|
1U GPU-ready server | — | — | $3,500 |
|
Supermicro SYS-521C-NR-G1
Compact
|
2U GPU-ready server | — | — | $4,500 |
|
Supermicro SYS-522GA-NRT
Compact
|
RTX PRO 6000 / L40S multi-GPU | — | — | 索取报价 |
已经在上方发现了几个偏好的GPU?任选两个GPU,观察它们在VRAM、推理指数、FP16计算和训练吞吐量方面的对比——并用简明的语言得出哪个更胜一筹以及原因。
我们从公开发布的GPU云基准测试数据开始——可验证的数字,任何人都可以核实,没有内部猜测。

每张卡片都涵盖了LLM推理、图像生成和视觉等方面——这是AI硬件全天实际进行的工作。
在流行的开源模型中,Token 生成速度,从轻量级的 8B 助手到 70B+ 大型模型。
扩散模型的吞吐量和延迟,涵盖从高速涡轮管道到生产级渲染的一切。
在实际并发负载下的多模态图像理解和文档 OCR 处理能力。

所有评分都基于一个100分的评分标准,RTX 3090作为基准——因此,任何两张显卡一目了然地进行比较。
单GPU独立发布的图形(每秒图像数),显示在与该型号卡匹配的基准测试存在的地方。

仅供参考的基础数据——实际效果因软件堆栈、驱动程序、批处理大小和工作负载而异。不能作为性能保证。
不熟悉这些术语吗? 阅读关于选择AI GPU的简单指南
无论您是需要特定的GPU、完整的服务器配置,还是对本页上的数字有任何疑问——请随时联系我们,我们的工程师通常会在两小时内回复您。
GPU级别与您实际计划运行的内容的粗略对照指南。
绝对顶级水平——为同时服务数千用户的AI公司的企业量身打造。
为 demanding 团队和繁重的日常工作提供强大的生产能力。
理想的平衡点——以小团队能够承受的价格,提供卓越性能。
一种经济实惠的入门方式——本地学习、原型设计及运行较小的模型。
它是一个相对数值,显示GPU在LLM推理、图像生成和视觉工作负载中的整体性能,比例以RTX 3090始终等于100为基准。得分为200意味着大致是RTX 3090总吞吐量的两倍。
RTX Pro 6000 Blackwell 目前以 AI 推理指数 403 位列我们的排名第一——大致是 RTX 3090 基准的推理吞吐总量的四倍——配备 96 GB 显存,适用于无需量化的最大 LLMs。
RTX 5090 (AI Inference Index 207, 32 GB) 和 RTX 4090 (指数 133, 24 GB) 在单GPU推理和图像生成方面提供了最强的性价比。RTX 4090 Pro 变体增加了 48 GB VRAM,以支持更大模型,同时保持消费者级的价值。
对于未进行量化的最大LLMs,旗舰级数据中心卡如RTX Pro 6000 Blackwell(96 GB)、H200(141 GB)和Instinct MI300X(192 GB)为您提供了内存余量,能够让完整模型保持驻留。对于8B至30B模型,单个RTX 5090或RTX 4090可在极低成本下实现出色的吞吐量。
是的 — 当前一代的 NVIDIA GPU 和完整的 AI 服务器可以通过我们的 AI 硬件目录获得,提供全球 DDP 运输和按需的 B2B 价格。
它是最广泛部署的具有AI能力的GPU之一,成为比较新旧硬件的实际且广为人知的参考点。
VRAM 决定了哪些模型大小和批处理大小可以适配——它本身并不决定速度。即使显存较少的矿卡,如果其架构和内存带宽更强,仍然可以获得更高的推理指数。
AMD Instinct MI350X 以 288 GB 位居首位,其次是 MI325X,配备 256 GB。在 NVIDIA 显卡中,B100 SXM 提供 192 GB 高带宽内存,而 H200 则提供 141 GB。更多的 VRAM 使您可以加载更大的模型和批次,而无需在多个 GPU 之间拆分。
FP16 TFLOPS 是制造商公布的理论峰值计算指标——纯硬件的最高水平。AI 推理指数是一个基于工作负载的测量得分,由在大型语言模型、图像和视觉任务中的公开基准测试构建。当内存带宽或软件支持限制其性能时,即使显卡显示出高理论 TFLOPS,实际的指数也会较低。
两者均为Hopper架构的数据中心GPU,额定性能为989,5 FP16 TFLOPS。区别在于内存:H200配备了141 GB更快的HBM3e,而H100则为80 GB,因此它能支持更大的模型和更长的上下文窗口,并具有更高的持续吞吐量。在大多数新的LLM服务部署中,H200是更强的选择。
是的。请使用本页面的比较竞技场,选择我们列出的78个GPU中的任意两个,查看它们的AI推断指数、VRAM、FP16 TFLOPS和训练吞吐量,并突出显示每项指标的获胜者。
训练比推理对内存和计算的需求要高得多,因此更倾向于使用具有大显存和高FP16 TFLOPS的旗舰显卡,例如H100、H200、B200或Instinct MI300X系列。我们的训练吞吐量列显示在有公开基准测试的情况下,每秒单GPU处理的图像数量——例如,在A100 40GB上为1,396张/秒,而在RTX 3090上为905张/秒。
是的。除了78个单独的GPU之外,我们还列出了48台完整的多GPU AI服务器,包括基于H100、H200和Blackwell加速器的HGX和DGX级系统,适用于大规模训练和生产推理。请联系我们获取配置和B2B价格。
全部涵盖。除了完整的 NVIDIA 系列产品外,我们还包括 AMD Instinct 加速器(MI350X、MI325X、MI300X、MI250X)和 Radeon 显卡,以及 Intel Gaudi 3 和 Arc,方便您在一个统一的指数上进行供应商间的比较。
每个数据均基于公开发布的基准测试和制造商数据,汇总成一致的指数供参考。实际结果会因软件堆栈、驱动程序、批处理大小和工作负载而异,因此请将这些数字视为一个比较指南,而非性能保证。
我们会在新一代GPU发布时以及更多基准数据可用时,审查和更新数据。