List your company
ResearchFormerly WizWand

Prefill Latency Measurement on LLaMA 8B-Instruct 64K context length 3.1

8,945.9Attention LatencyCompactAttention-FP
8,310.96812,596.75916,882.5521,168.341May 16, 2026
Updated 5mo ago

Evaluation results

MethodLinks
CompactAttention-FPGPU=RTX PRO 6000, TP=2, Batch Size=4, Chunk Size=5122026.058,945.923,707.8
CompactAttention-SAGPU=RTX PRO 6000, TP=2, Batch Size=4, Chunk Size=5122026.0513,823.528,960.4
FlashPrefillGPU=RTX PRO 6000, TP=2, Batch Size=4, Chunk Size=5122026.0514,381.829,411.7
QUOKAGPU=RTX PRO 6000, TP=2, Batch Size=4, Chunk Size=5122026.0516,325.430,588
SeerAttentionGPU=RTX PRO 6000, TP=2, Batch Size=4, Chunk Size=5122026.0517,078.232,545.8
DenseGPU=RTX PRO 6000, TP=2, Batch Size=4, Chunk Size=5122026.0517,838.832,177.4
XAttentionGPU=RTX PRO 6000, TP=2, Batch Size=4, Chunk Size=5122026.0524,819.239,206.6