ResearchTasksMulti-Head Attention (MHA)Multi-Head Attention (MHA)FollowBenchmarksDatasetSOTA methodSortMost resultsRecently updatedMost papersApplyDatasetSOTA methodMetricTrendResultsLast updatedMHA causal mask head dimension 128 FP8 on NVIDIA L40S GPU (test)LLM-TL257.9Performance (TFLOPS)6Feb 26, 2026NVIDIA A100 GPUDeepSeek-V3 + Ours207.2TFLOPS5Feb 26, 2026NVIDIA RTX8000 GPUDeepSeek-V3 + Ours49.9TFLOPS2Feb 26, 2026
MHA causal mask head dimension 128 FP8 on NVIDIA L40S GPU (test)LLM-TL257.9Performance (TFLOPS)6Feb 26, 2026