Mathematical Reasoning on GSM8K (Inference Latency & Throughput)
1,163Latency (NoKV) (ms)NoKV
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| NoKVModel=Qwen2.5-7B Instruct2026.06 | 1,163 | — | — | — | — | — | |
| NoKVModel=Falcon3-7B Instruct2026.06 | 1,170 | — | — | — | — | — | |
| NoKVModel=Mistral-7B Instruct2026.06 | 1,262 | — | — | — | — | — | |
| NoKVModel=Llama-3-8B Instruct2026.06 | 1,283 | — | — | — | — | — | |
| NoKVModel=Falcon3-10B Instruct2026.06 | 1,634 | — | — | — | — | — | |
| RKSCModel=Qwen2.5-7B Instruct2026.06 | — | — | 515 | — | 2.26 | 32 | |
| RKSCModel=Mistral-7B Instruct2026.06 | — | — | 581 | — | 2.17 | 30 | |
| RKSCModel=Falcon3-7B Instruct2026.06 | — | — | 376 | — | 3.11 | 50 | |
| RKSCModel=Falcon3-10B Instruct2026.06 | — | — | 473 | — | 3.46 | 57 | |
| RKSCModel=Llama-3-8B Instruct2026.06 | — | — | 464 | — | 2.76 | 65 | |
| vLLMModel=Qwen2.5-7B Instruct2026.06 | — | 652 | — | 1.78 | — | — | |
| vLLMModel=Mistral-7B Instruct2026.06 | — | 689 | — | 1.83 | — | — | |
| vLLMModel=Falcon3-7B Instruct2026.06 | — | 650 | — | 1.8 | — | — | |
| vLLMModel=Falcon3-10B Instruct2026.06 | — | 898 | — | 1.82 | — | — | |
| vLLMModel=Llama-3-8B Instruct2026.06 | — | 710 | — | 1.81 | — | — |