Mathematical Reasoning on MATH 500 (Accuracy, Memory Usage)
81.2AccuracyFullKV
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FullKVModel=Qwen-2.5-7B, rho=1.002026.05 | 81.2 | 23.1 | |
| FullKVModel=Llama-3.1-8B, rho=1.002026.05 | 79.8 | 23.5 | |
| ArborKVModel=Qwen-2.5-7B, rho=0.502026.05 | 77.6 | 10.9 | |
| StreamingLLMModel=Qwen-2.5-7B, rho=0.502026.05 | 76.4 | 11.6 | |
| ThinKVModel=Qwen-2.5-7B, rho=0.502026.05 | 76 | 10.6 | |
| ArborKVModel=Llama-3.1-8B, rho=0.502026.05 | 75.8 | 12.3 | |
| H2OModel=Qwen-2.5-7B, rho=0.502026.05 | 74.8 | 10.7 | |
| StreamingLLMModel=Llama-3.1-8B, rho=0.502026.05 | 74.2 | 13.1 | |
| ThinKVModel=Llama-3.1-8B, rho=0.502026.05 | 73.8 | 11.5 | |
| H2OModel=Llama-3.1-8B, rho=0.502026.05 | 72.2 | 12.4 | |
| ArborKVModel=Qwen-2.5-7B, rho=0.252026.05 | 67.4 | 6.1 | |
| ThinKVModel=Qwen-2.5-7B, rho=0.252026.05 | 65.4 | 6 | |
| ArborKVModel=Llama-3.1-8B, rho=0.252026.05 | 64.8 | 5.9 | |
| StreamingLLMModel=Qwen-2.5-7B, rho=0.252026.05 | 62.8 | 5.9 | |
| StreamingLLMModel=Llama-3.1-8B, rho=0.252026.05 | 62.4 | 5.8 | |
| ThinKVModel=Llama-3.1-8B, rho=0.252026.05 | 61.6 | 5.4 | |
| H2OModel=Qwen-2.5-7B, rho=0.252026.05 | 61.6 | 5.4 | |
| H2OModel=Llama-3.1-8B, rho=0.252026.05 | 60.2 | 6 |