AI: Nvidia, AMD, Intel Similar

Databricks tests have similar results

Databricks, a large service bureau, has the scale, expertise, and incentive to do things carefully. There are small differences that would change with a different set of tests. These numbers are strongly suggestive of very rough parity between the chips on the market in December 2023. Intel & AMD are not far behind Nvidia,

Here are their results.

 Intel Gaudi3AMD MI300XNVIDIA H100NVIDIA H200
 Single Card8x Gaudi3Single Card8x MI300XSingle Card8x H100Single Card8x H200
FP16 or BF16 TFLOP/s~1600 TFLOP/s~12800 TFLOP/s1307 TFLOP/s10456 TFLOP/s989.5 TFLOP/s7916 TFLOP/s989.5 TFLOP/s7916 TFLOP/s
HBM Memory (GB)N/AN/A192 GB1536 GB80GB640 GB141GB1128 GB
Memory Bandwidth~3675 GB/s~29.4 TB/s5300 GB/s42.4 TB/s3350 GB/s26.8 TB/s4800 GB/s38.4 TB/s
Peak Power ConsumptionN/AN/A750 WN/A700 W10200 W700 W10200 W
Rack Units (RU)N/AN/AN/A8UN/A8UN/A8U

PR types, reporters, and salesmen find small differences and shout them to the rooftops. In practice, the benchmarks have so many limits that small differences mean little. Extrapolating to real-world use adds further sources of error.

There are lies, damned lies, and benchmarks. They are notoriously unreliable. Companies choose the most favorable ones and the options that make them shine. Some train on the data in the benchmark.

Huawei has a high-performance chip which rarely is considered outside China.

Posted in Uncategorized.

Leave a Reply

Your email address will not be published. Required fields are marked *