List your company
ResearchFormerly WizWand

General Knowledge Question Answering on MMLU (Latency and Throughput)

1,160Latency (NoKV, ms)NoKV
1,141.041,269.021,3971,524.98Jun 7, 2026
Updated 4mo ago

Evaluation results

MethodLinks
NoKVModel=Qwen2.5-7B Instruct2026.061,160—————
NoKVModel=Falcon3-7B Instruct2026.061,175—————
NoKVModel=Mistral-7B Instruct2026.061,260—————
NoKVModel=Llama-3-8B Instruct2026.061,284—————
NoKVModel=Falcon3-10B Instruct2026.061,634—————
RKSCModel=Qwen2.5-7B Instruct2026.06——371—3.1370
RKSCModel=Mistral-7B Instruct2026.06——534—2.3647
RKSCModel=Falcon3-7B Instruct2026.06——395—2.9839
RKSCModel=Falcon3-10B Instruct2026.06——467—3.566
RKSCModel=Llama-3-8B Instruct2026.06——322—3.9992
vLLMModel=Qwen2.5-7B Instruct2026.06—649—1.79——
vLLMModel=Mistral-7B Instruct2026.06—692—1.82——
vLLMModel=Falcon3-7B Instruct2026.06—651—1.8——
vLLMModel=Falcon3-10B Instruct2026.06—898—1.82——
vLLMModel=Llama-3-8B Instruct2026.06—710—1.81——