List your company
ResearchFormerly WizWand

Inference Efficiency on DeepScaleR-40k (1,024 mathematical problems)

760.74Throughput (tokens/s)G-KV
1.8936198.9018395.91592.9182Nov 29, 2025
Updated 8mo ago

Evaluation results

MethodLinks
G-KVModel=DeepSeek-Qwen-Distill-7B, Batch Size=128, KV Cache Budget=5122025.11760.74
G-KVModel=DeepSeek-Llama-Distill-8B, Batch Size=64, KV Cache Budget=5122025.11612.6
G-KVModel=DeepSeek-Llama-Distill-8B, Batch Size=32, KV Cache Budget=5122025.11517.3
G-KVModel=DeepSeek-Qwen-Distill-7B, Batch Size=64, KV Cache Budget=5122025.11475.59
G-KVModel=DeepSeek-Qwen-Distill-7B, Batch Size=128, KV Cache Budget=10242025.11448.35
G-KVModel=DeepSeek-Qwen-Distill-7B, Batch Size=64, KV Cache Budget=10242025.11367.96
G-KVModel=DeepSeek-Qwen-Distill-7B, Batch Size=32, KV Cache Budget=5122025.11261.32
G-KVModel=DeepSeek-Llama-Distill-8B, Batch Size=64, KV Cache Budget=10242025.11258.18
G-KVModel=DeepSeek-Qwen-Distill-7B, Batch Size=128, KV Cache Budget=20482025.11248.22
R-KVModel=DeepSeek-Qwen-Distill-7B, Batch Size=128, KV Cache Budget=20482025.11238.43
G-KVModel=DeepSeek-Qwen-Distill-7B, Batch Size=64, KV Cache Budget=20482025.11221.23
G-KVModel=DeepSeek-Qwen-Distill-7B, Batch Size=32, KV Cache Budget=10242025.11212.93
R-KVModel=DeepSeek-Qwen-Distill-7B, Batch Size=64, KV Cache Budget=20482025.11203.66
G-KVModel=DeepSeek-Llama-Distill-8B, Batch Size=32, KV Cache Budget=10242025.11193.56
R-KVModel=DeepSeek-Qwen-Distill-7B, Batch Size=32, KV Cache Budget=20482025.11172.44
G-KVModel=DeepSeek-Qwen-Distill-7B, Batch Size=32, KV Cache Budget=20482025.11170.64
G-KVModel=DeepSeek-Llama-Distill-8B, Batch Size=16, KV Cache Budget=5122025.11158.43
G-KVModel=DeepSeek-Llama-Distill-8B, Batch Size=64, KV Cache Budget=20482025.11154.52
G-KVModel=DeepSeek-Llama-Distill-8B, Batch Size=32, KV Cache Budget=20482025.11118.82
G-KVModel=DeepSeek-Llama-Distill-8B, Batch Size=16, KV Cache Budget=10242025.11118.1
R-KVModel=DeepSeek-Llama-Distill-8B, Batch Size=64, KV Cache Budget=20482025.11111.89
R-KVModel=DeepSeek-Llama-Distill-8B, Batch Size=32, KV Cache Budget=20482025.1198.01
G-KVModel=DeepSeek-Llama-Distill-8B, Batch Size=16, KV Cache Budget=20482025.1193.91
R-KVModel=DeepSeek-Llama-Distill-8B, Batch Size=16, KV Cache Budget=20482025.1182.33
Full-KVModel=DeepSeek-Qwen-Distill-7B, Batch Size=322025.1162.41
Full-KVModel=DeepSeek-Llama-Distill-8B, Batch Size=162025.1131.08