List your company
ResearchFormerly WizWand

End-to-end decode throughput on 8K Context Length

268.5Throughput (tok/s)Llama-3.1-8B
146.924178.487210.05241.613May 13, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Llama-3.1-8BKV Representation=SphKV2026.05268.5
GPT-ossKV Representation=SphKV2026.05261.2
Qwen2.5-14BKV Representation=SphKV2026.05239.4
Llama-3.1-8BKV Representation=Dense2026.05173.4
GPT-ossKV Representation=Dense2026.05163.7
Qwen2.5-14BKV Representation=Dense2026.05151.6