Reasoning Question Answering on GPQA (Latency/Efficiency)
1,158Latency (NoKV, ms)NoKV
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| NoKVModel=Falcon3-7B Instruct, Batch size=8, Prefix length=~1,024 tok, Branching factor (t)=8, N=502026.06 | 1,158 | — | — | — | — | — | |
| NoKVModel=Qwen2.5-7B Instruct, Batch size=8, Prefix length=~1,024 tok, Branching factor (t)=8, N=502026.06 | 1,159 | — | — | — | — | — | |
| NoKVModel=Mistral-7B Instruct, Batch size=8, Prefix length=~1,024 tok, Branching factor (t)=8, N=502026.06 | 1,257 | — | — | — | — | — | |
| NoKVModel=Llama-3-8B Instruct, Batch size=8, Prefix length=~1,024 tok, Branching factor (t)=8, N=502026.06 | 1,280 | — | — | — | — | — | |
| NoKVModel=Falcon3-10B Instruct, Batch size=8, Prefix length=~1,024 tok, Branching factor (t)=8, N=502026.06 | 1,629 | — | — | — | — | — | |
| RKSCModel=Qwen2.5-7B Instruct, Batch size=8, Prefix length=~1,024 tok, Branching factor (t)=8, N=502026.06 | — | — | 598 | — | 1.94 | 3 | |
| RKSCModel=Mistral-7B Instruct, Batch size=8, Prefix length=~1,024 tok, Branching factor (t)=8, N=502026.06 | — | — | 590 | — | 2.13 | 23 | |
| RKSCModel=Falcon3-7B Instruct, Batch size=8, Prefix length=~1,024 tok, Branching factor (t)=8, N=502026.06 | — | — | 360 | — | 3.21 | 54 | |
| RKSCModel=Falcon3-10B Instruct, Batch size=8, Prefix length=~1,024 tok, Branching factor (t)=8, N=502026.06 | — | — | 450 | — | 3.63 | 72 | |
| RKSCModel=Llama-3-8B Instruct, Batch size=8, Prefix length=~1,024 tok, Branching factor (t)=8, N=502026.06 | — | — | 509 | — | 2.51 | 57 | |
| vLLMModel=Qwen2.5-7B Instruct, Batch size=8, Prefix length=~1,024 tok, Branching factor (t)=8, N=502026.06 | — | 653 | — | 1.78 | — | — | |
| vLLMModel=Mistral-7B Instruct, Batch size=8, Prefix length=~1,024 tok, Branching factor (t)=8, N=502026.06 | — | 690 | — | 1.82 | — | — | |
| vLLMModel=Falcon3-7B Instruct, Batch size=8, Prefix length=~1,024 tok, Branching factor (t)=8, N=502026.06 | — | 637 | — | 1.82 | — | — | |
| vLLMModel=Falcon3-10B Instruct, Batch size=8, Prefix length=~1,024 tok, Branching factor (t)=8, N=502026.06 | — | 895 | — | 1.82 | — | — | |
| vLLMModel=Llama-3-8B Instruct, Batch size=8, Prefix length=~1,024 tok, Branching factor (t)=8, N=502026.06 | — | 708 | — | 1.81 | — | — |