AMD Introduces TAF Metric for Sustained GEMM Throughput on MI325X

AMD published a new performance metric called Typical Attained FLOPs (TAF) on October 6, 2026, designed to measure sustained GEMM throughput across a fixed set of workload-derived shapes. The metric joins Peak FLOPs and Max-Achievable FLOPs (MAF) in the company's GPU performance evaluation framework. TAF aims to answer a different question than the other two: what is a reasonable target for GEMM performance on a variety of GEMM sizes, rather than the theoretical ceiling or the single best-case sustained result.

The TAF methodology uses a predetermined set of 10 GEMM shapes derived from DeepSeek V3. Five (N,K) pairs are each evaluated at two M values, 32 and 32,768. Each shape runs for 30 seconds using mblas-bench to measure sustained performance. The final TAF value is a FLOP-weighted harmonic mean across all 10 shapes, so GEMMs with more mathematical work contribute proportionally more. This approach reduces the best-case shape-selection bias that can affect MAF, which reports only the highest result found across tested configurations.

AMD also changed the input initialization for both MAF and TAF tests. Earlier MAF results used trig_float initialization, which could lead to lower switching activity and power consumption, allowing the GPU to sustain higher frequencies. The current tests use normal-distribution initialization, with values generated directly in each target data format and scale factors set to one for unscaled FP8. Because of this change, earlier MAF results are not directly comparable with current results.

Results on One MI325X GPU

For one AMD Instinct MI325X GPU, the TAF results were 606 TFLOPs for FP16, 616 TFLOPs for BF16, and 1,143 TFLOPs for unscaled FP8. These values represent approximately 74 to 81 percent of the corresponding published MAF values. The published Peak FLOPs for the MI325X are 1,300 TFLOPs for FP16 and BF16, and 2,610 TFLOPs for FP8. The numerical differences illustrate how each metric answers a different question: Peak FLOPs define the theoretical compute ceiling, MAF measures the highest sustained throughput across tested GEMM sizes, and TAF measures sustained throughput across the complete representative set.

The lower TAF values compared to MAF do not indicate a loss of GPU performance. TAF is a measurement methodology, not an operating mode. The difference shows that the highest sustained throughput achieved on one favorable GEMM shape may not represent performance across a broader set of workload-derived shapes.

What TAF Means for Inference Engineers

TAF is a compute-side indicator for the GEMM portion of inference, not a direct measure of end-to-end application performance. Modern inference workloads, particularly the decode and token-generation stages of large language models, can be constrained by memory bandwidth, KV-cache access, communication overhead, latency, and sequential token dependencies rather than raw compute throughput. Consequently, differences in TAF may produce smaller improvements in end-to-end throughput or latency when another part of the pipeline is the bottleneck.

AMD states that TAF does not directly predict training time, inference latency, or tokens per second. Application-level benchmarks remain necessary to measure the performance of the complete hardware and software system. For engineers comparing GPUs, TAF provides a consistent basis for comparison using the same workload-derived GEMM set, but it should be interpreted alongside MAF and Peak FLOPs to get a complete picture of sustained compute performance.