LLM Serving and Question Answering on MultiHopRAG hot0.7 (test)
49.27KV-Cache Hit RatePRISM
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| PRISMModel=Qwen3-4B-Instruct-2507, QPS=802026.05 | 49.27 | 40.28 | |
| PRISMModel=Qwen3-4B-Instruct-2507, QPS=602026.05 | 49.25 | 40.29 | |
| PRISMModel=Qwen3-4B-Instruct-2507, QPS=702026.05 | 49.1 | 40.44 | |
| PRISMModel=Qwen3-4B-Instruct-2507, QPS=402026.05 | 48.98 | 40.61 | |
| PRISMModel=Qwen3-4B-Instruct-2507, QPS=502026.05 | 48.9 | 40.33 | |
| ContextPilotModel=Qwen3-4B-Instruct-2507, QPS=602026.05 | 43.76 | 39.36 | |
| ContextPilotModel=Qwen3-4B-Instruct-2507, QPS=402026.05 | 43.64 | 40.33 | |
| ContextPilotModel=Qwen3-4B-Instruct-2507, QPS=502026.05 | 43.51 | 39.85 | |
| ContextPilotModel=Qwen3-4B-Instruct-2507, QPS=802026.05 | 43.24 | 40.56 | |
| ContextPilotModel=Qwen3-4B-Instruct-2507, QPS=702026.05 | 42.04 | 39.48 | |
| PRISMModel=Llama2-13B, QPS=302026.05 | 40.15 | 21.88 | |
| PRISMModel=Llama2-13B, QPS=102026.05 | 39.79 | 21.69 | |
| PRISMModel=Llama2-13B, QPS=252026.05 | 39.22 | 21.79 | |
| PRISMModel=Llama2-13B, QPS=202026.05 | 39.15 | 21.36 | |
| PRISMModel=Llama2-13B, QPS=152026.05 | 39.02 | 21.58 | |
| ContextPilotModel=Llama2-13B, QPS=252026.05 | 28.14 | 20.67 | |
| ContextPilotModel=Llama2-13B, QPS=302026.05 | 27.5 | 20.58 | |
| ContextPilotModel=Llama2-13B, QPS=202026.05 | 27.06 | 20.48 | |
| ContextPilotModel=Llama2-13B, QPS=152026.05 | 27.01 | 20.98 | |
| ContextPilotModel=Llama2-13B, QPS=102026.05 | 26.51 | 20.97 | |
| k-LPMModel=Llama2-13B, QPS=302026.05 | 20.86 | 20.66 | |
| k-LPMModel=Qwen3-4B-Instruct-2507, QPS=702026.05 | 20.39 | 39.76 | |
| k-LPMModel=Llama2-13B, QPS=202026.05 | 20.34 | 20.56 | |
| k-LPMModel=Qwen3-4B-Instruct-2507, QPS=602026.05 | 20.26 | 39.76 | |
| k-LPMModel=Llama2-13B, QPS=152026.05 | 20.14 | 20.45 | |
| k-LPMModel=Llama2-13B, QPS=102026.05 | 20.1 | 21.01 | |
| k-LPMModel=Qwen3-4B-Instruct-2507, QPS=402026.05 | 19.94 | 39.81 | |
| k-LPMModel=Llama2-13B, QPS=252026.05 | 19.89 | 20.68 | |
| k-LPMModel=Qwen3-4B-Instruct-2507, QPS=802026.05 | 19.86 | 39.81 | |
| k-LPMModel=Qwen3-4B-Instruct-2507, QPS=502026.05 | 16.93 | 39.81 | |
| native SGLangModel=Qwen3-4B-Instruct-2507, QPS=802026.05 | 16.06 | 39.48 | |
| native SGLangModel=Qwen3-4B-Instruct-2507, QPS=402026.05 | 15.17 | 39.76 | |
| native SGLangModel=Qwen3-4B-Instruct-2507, QPS=602026.05 | 15.14 | 39.81 | |
| native SGLangModel=Qwen3-4B-Instruct-2507, QPS=502026.05 | 15.07 | 39.81 | |
| native SGLangModel=Qwen3-4B-Instruct-2507, QPS=702026.05 | 14.98 | 39.53 | |
| native SGLangModel=Llama2-13B, QPS=302026.05 | 11.06 | 20.98 | |
| native SGLangModel=Llama2-13B, QPS=102026.05 | 10.17 | 20.78 | |
| native SGLangModel=Llama2-13B, QPS=202026.05 | 10.14 | 20.74 | |
| native SGLangModel=Llama2-13B, QPS=152026.05 | 10.07 | 20.88 | |
| native SGLangModel=Llama2-13B, QPS=252026.05 | 9.98 | 20.96 |