Mathematical Reasoning on MATH500 (Accuracy and Solution Component Metrics)
91.2AccuracyFASA
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| FASABackbone=DeepSeek-R1-Distill-Qwen-14B, Fixed Budget=10002026.02 | 91.2 | 127 | 3,139 | 3,266 | |
| FASABackbone=DeepSeek-R1-Distill-Qwen-32B, Fixed Budget=10002026.02 | 91.2 | 127 | 2,887 | 3,014 | |
| FASABackbone=DeepSeek-R1-Distill-Qwen-14B, Fixed Budget=7002026.02 | 90.2 | 127 | 3,139 | 3,266 | |
| FASABackbone=DeepSeek-R1-Distill-Qwen-32B, Fixed Budget=5002026.02 | 90.2 | 127 | 2,887 | 3,014 | |
| FASABackbone=DeepSeek-R1-Distill-Qwen-32B, Fixed Budget=7002026.02 | 90.2 | 127 | 2,887 | 3,014 | |
| FASABackbone=DeepSeek-R1-Distill-Qwen-14B, Fixed Budget=5002026.02 | 88.8 | 127 | 3,139 | 3,266 | |
| FASABackbone=DeepSeek-R1-Distill-Qwen-14B, Fixed Budget=3002026.02 | 86.6 | 127 | 3,139 | 3,266 | |
| FASABackbone=DeepSeek-R1-Distill-Qwen-32B, Fixed Budget=3002026.02 | 86.4 | 127 | 2,887 | 3,014 | |
| OracleBackbone=DeepSeek-R1-Distill-Llama-8B, Fixed Budget=7002026.02 | 74.2 | 127 | 3,195 | 3,321 | |
| OracleBackbone=DeepSeek-R1-Distill-Llama-8B, Fixed Budget=5002026.02 | 72.6 | 127 | 3,195 | 3,321 | |
| FKVBackbone=DeepSeek-R1-Distill-Llama-8B, Fixed Budget=3002026.02 | 72.4 | 127 | 2,977 | 3,104 | |
| FKVBackbone=DeepSeek-R1-Distill-Llama-8B, Fixed Budget=10002026.02 | 72.4 | 127 | 2,977 | 3,104 | |
| OracleBackbone=DeepSeek-R1-Distill-Llama-8B, Fixed Budget=10002026.02 | 71.8 | 127 | 3,195 | 3,321 | |
| FASABackbone=DeepSeek-R1-Distill-Llama-8B, Fixed Budget=10002026.02 | 71.8 | 127 | 3,171 | 3,298 | |
| OracleBackbone=DeepSeek-R1-Distill-Llama-8B, Fixed Budget=3002026.02 | 70.4 | 127 | 3,195 | 3,321 | |
| FASABackbone=DeepSeek-R1-Distill-Llama-8B, Fixed Budget=7002026.02 | 69.4 | 127 | 3,171 | 3,298 | |
| FASABackbone=DeepSeek-R1-Distill-Llama-8B, Fixed Budget=5002026.02 | 68.8 | 127 | 3,171 | 3,298 | |
| FASABackbone=DeepSeek-R1-Distill-Llama-8B, Fixed Budget=3002026.02 | 62.2 | 127 | 3,171 | 3,298 | |
| RKVBackbone=DeepSeek-R1-Distill-Llama-8B, Fixed Budget=10002026.02 | 57 | 127 | 7,005 | 7,132 | |
| SnapKVBackbone=DeepSeek-R1-Distill-Llama-8B, Fixed Budget=10002026.02 | 54.6 | 127 | 7,047 | 7,174 | |
| H2OBackbone=DeepSeek-R1-Distill-Llama-8B, Fixed Budget=7002026.02 | 53.87 | 127 | 8,244 | 8,370 | |
| RKVBackbone=DeepSeek-R1-Distill-Llama-8B, Fixed Budget=7002026.02 | 49.2 | 127 | 7,005 | 7,132 | |
| StreamBackbone=DeepSeek-R1-Distill-Llama-8B, Fixed Budget=10002026.02 | 47.4 | 127 | 3,520 | 3,647 | |
| SnapKVBackbone=DeepSeek-R1-Distill-Llama-8B, Fixed Budget=7002026.02 | 46.8 | 127 | 7,047 | 7,174 | |
| H2OBackbone=DeepSeek-R1-Distill-Llama-8B, Fixed Budget=10002026.02 | 42.8 | 127 | 8,244 | 8,370 | |
| StreamBackbone=DeepSeek-R1-Distill-Llama-8B, Fixed Budget=7002026.02 | 40.4 | 127 | 3,520 | 3,647 | |
| RKVBackbone=DeepSeek-R1-Distill-Llama-8B, Fixed Budget=5002026.02 | 39.4 | 127 | 7,005 | 7,132 | |
| H2OBackbone=DeepSeek-R1-Distill-Llama-8B, Fixed Budget=5002026.02 | 33 | 127 | 8,244 | 8,370 | |
| SnapKVBackbone=DeepSeek-R1-Distill-Llama-8B, Fixed Budget=5002026.02 | 32.6 | 127 | 7,047 | 7,174 | |
| StreamBackbone=DeepSeek-R1-Distill-Llama-8B, Fixed Budget=5002026.02 | 24.6 | 127 | 3,520 | 3,647 | |
| RKVBackbone=DeepSeek-R1-Distill-Llama-8B, Fixed Budget=3002026.02 | 24 | 127 | 7,005 | 7,132 | |
| SnapKVBackbone=DeepSeek-R1-Distill-Llama-8B, Fixed Budget=3002026.02 | 21.6 | 127 | 7,047 | 7,174 | |
| StreamBackbone=DeepSeek-R1-Distill-Llama-8B, Fixed Budget=3002026.02 | 9.6 | 127 | 3,520 | 3,647 | |
| H2OBackbone=DeepSeek-R1-Distill-Llama-8B, Fixed Budget=3002026.02 | 6.8 | 127 | 8,244 | 8,370 |