Latency Measurement on LLaMA-8B-Instruct Chunked Prefill 3.1 (inference)

423.1Attention Latency (ms)Dense

Evaluation Results

Method
Dense2026.05	423.1	2,299.7
Dense2026.05	429.8	1,363.2
FlashPrefill2026.05	464.5	1,431.9
FlashPrefill2026.05	496.6	2,294.2
CompactAttention-FP2026.05	531.5	1,476.1
QUOKA2026.05	582.3	1,580
CompactAttention-FP2026.05	596	2,345.1
QUOKA2026.05	674	2,470.4
CompactAttention-SA2026.05	780.2	1,788.3
XAttention2026.05	923.9	1,877
SeerAttention2026.05	1,002.5	1,991.7
SeerAttention2026.05	1,012.3	2,911.3
XAttention2026.05	1,095.5	2,874.8
FlashPrefill2026.05	1,262.5	3,212.2
Dense2026.05	1,272.9	3,129
CompactAttention-FP2026.05	1,274.7	3,165.8
CompactAttention-SA2026.05	1,303	3,139.8
QUOKA2026.05	1,516.6	3,571.4
CompactAttention-SA2026.05	1,775.7	3,745.1
XAttention2026.05	2,600.1	4,536.7
SeerAttention2026.05	3,073.4	5,029.2
CompactAttention-FP2026.05	3,292	7,006.8
FlashPrefill2026.05	3,438.8	7,276.7
CompactAttention-FP2026.05	3,474.8	10,691.7
Dense2026.05	4,173.8	7,911.3
CompactAttention-SA2026.05	4,393.8	8,125.8
FlashPrefill2026.05	4,547.7	12,088.4
QUOKA2026.05	4,629.7	8,482.5
QUOKA2026.05	5,196.5	13,116.8
Dense2026.05	5,213.3	12,421.8
CompactAttention-SA2026.05	5,584.4	12,706.7
SeerAttention2026.05	6,171.8	13,620
XAttention2026.05	7,729.7	11,649.8
XAttention2026.05	7,770.5	14,896.8
CompactAttention-FP2026.05	8,292	15,990
SeerAttention2026.05	9,478.1	13,671
FlashPrefill2026.05	9,489.7	17,355.7
CompactAttention-SA2026.05	10,895.6	18,453.3
Dense2026.05	14,810.9	22,264
QUOKA2026.05	15,012.2	22,451.8
CompactAttention-FP2026.05	20,489.7	36,265.4
XAttention2026.05	24,777.8	32,093.5
FlashPrefill2026.05	24,886.5	41,152.3
SeerAttention2026.05	27,632.5	36,406.8
CompactAttention-SA2026.05	29,468.1	45,366.8
QUOKA2026.05	51,292.4	66,029.1
Dense2026.05	55,678.4	71,015.1
SeerAttention2026.05	85,852	105,207.6
XAttention2026.05	86,821.4	103,870.2