ResearchDatasetsMHAMHAFollowBenchmarksTaskDatasetSOTA methodSortMost resultsRecently updatedMost papersApplyTaskDatasetSOTA resultTrendResultsMulti-Head Attention (MHA)MHA causal mask head dimension 128 FP8 on NVIDIA L40S GPU (test)257.9Performance (TFLOPS)6Multi-Head Attention Operator DevelopmentMHA (hd=64, sl=1024) on A100 GPU175.6TFLOPS2
Multi-Head Attention (MHA)MHA causal mask head dimension 128 FP8 on NVIDIA L40S GPU (test)257.9Performance (TFLOPS)6