List your company
ResearchFormerly WizWand

Prefill Latency Measurement on LLaMA 8B Instruct 128K context length 3.1

24,055.4Attention Latency (ms)CompactAttention-FP
21,712.93637,524.56853,336.269,147.832May 16, 2026
Updated 5mo ago

Evaluation results

MethodLinks
CompactAttention-FPGPU=RTX PRO 6000, TP=2, Batch Size=4, Chunk Size=5122026.0524,055.454,130.6
CompactAttention-SAGPU=RTX PRO 6000, TP=2, Batch Size=4, Chunk Size=5122026.0532,84664,126.7
FlashPrefillGPU=RTX PRO 6000, TP=2, Batch Size=4, Chunk Size=5122026.0543,750.580,108.5
SeerAttentionGPU=RTX PRO 6000, TP=2, Batch Size=4, Chunk Size=5122026.0549,839.881,774.4
QUOKAGPU=RTX PRO 6000, TP=2, Batch Size=4, Chunk Size=5122026.0556,475.385,375.3
DenseGPU=RTX PRO 6000, TP=2, Batch Size=4, Chunk Size=5122026.0565,971.194,944.3
XAttentionGPU=RTX PRO 6000, TP=2, Batch Size=4, Chunk Size=5122026.0582,617113,016.5