General Knowledge Question Answering on MMLU (Latency and Throughput)
1,160Latency (NoKV, ms)NoKV
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| NoKVModel=Qwen2.5-7B Instruct2026.06 | 1,160 | — | — | — | — | — | |
| NoKVModel=Falcon3-7B Instruct2026.06 | 1,175 | — | — | — | — | — | |
| NoKVModel=Mistral-7B Instruct2026.06 | 1,260 | — | — | — | — | — | |
| NoKVModel=Llama-3-8B Instruct2026.06 | 1,284 | — | — | — | — | — | |
| NoKVModel=Falcon3-10B Instruct2026.06 | 1,634 | — | — | — | — | — | |
| RKSCModel=Qwen2.5-7B Instruct2026.06 | — | — | 371 | — | 3.13 | 70 | |
| RKSCModel=Mistral-7B Instruct2026.06 | — | — | 534 | — | 2.36 | 47 | |
| RKSCModel=Falcon3-7B Instruct2026.06 | — | — | 395 | — | 2.98 | 39 | |
| RKSCModel=Falcon3-10B Instruct2026.06 | — | — | 467 | — | 3.5 | 66 | |
| RKSCModel=Llama-3-8B Instruct2026.06 | — | — | 322 | — | 3.99 | 92 | |
| vLLMModel=Qwen2.5-7B Instruct2026.06 | — | 649 | — | 1.79 | — | — | |
| vLLMModel=Mistral-7B Instruct2026.06 | — | 692 | — | 1.82 | — | — | |
| vLLMModel=Falcon3-7B Instruct2026.06 | — | 651 | — | 1.8 | — | — | |
| vLLMModel=Falcon3-10B Instruct2026.06 | — | 898 | — | 1.82 | — | — | |
| vLLMModel=Llama-3-8B Instruct2026.06 | — | 710 | — | 1.81 | — | — |