Mathematical Reasoning on OlympiadBench (Accuracy, Latency, % B)
0.44AccuracySPECS
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| SPECSModel Family=Qwen2.5 Family, n=162025.06 | 0.44 | 24.6 | 2.56 | 0.615 | |
| BS (B)Model Family=Qwen2.5 Family, n=162025.06 | 0.437 | 25.4 | 2.57 | 1 | |
| SPECSModel Family=Qwen2.5 Family, n=82025.06 | 0.417 | 16.9 | 1.8 | 0.803 | |
| SPECSModel Family=Qwen2.5 Family, n=42025.06 | 0.413 | 13.1 | 1.3 | 0.831 | |
| BS (B)Model Family=Qwen2.5 Family, n=82025.06 | 0.413 | 17.9 | 1.81 | 1 | |
| BS (B)Model Family=Qwen2.5 Family, n=42025.06 | 0.393 | 13.1 | 1.32 | 1 | |
| RSD++Model Family=Qwen2.5 Family, n=162025.06 | 0.335 | 28.7 | 2.58 | 0.612 | |
| RSD++Model Family=Qwen2.5 Family, n=82025.06 | 0.333 | 21.7 | 1.96 | 0.653 | |
| RSD++Model Family=Qwen2.5 Family, n=42025.06 | 0.33 | 17.3 | 1.6 | 0.663 | |
| BS (S)Model Family=Qwen2.5 Family, n=162025.06 | 0.313 | 21.1 | 2.02 | 0 | |
| BS (S)Model Family=Qwen2.5 Family, n=82025.06 | 0.31 | 13.4 | 1.22 | 0 | |
| BS (S)Model Family=Qwen2.5 Family, n=42025.06 | 0.25 | 7.9 | 0.81 | 0 | |
| BS (B)Model Family=Llama3 Family, n=162025.06 | 0.22 | 30.1 | 1.2 | 1 | |
| SPECSModel Family=Llama3 Family, n=162025.06 | 0.2 | 31.7 | 1.21 | 0.911 | |
| SPECSModel Family=Llama3 Family, n=82025.06 | 0.185 | 19.7 | 0.78 | 0.946 | |
| BS (B)Model Family=Llama3 Family, n=82025.06 | 0.183 | 18.9 | 0.83 | 1 | |
| SPECSModel Family=Llama3 Family, n=42025.06 | 0.18 | 14.9 | 0.62 | 0.981 | |
| BS (B)Model Family=Llama3 Family, n=42025.06 | 0.17 | 13 | 0.59 | 1 | |
| BS (S)Model Family=Llama3 Family, n=162025.06 | 0.113 | 23.17 | 1.06 | 0 | |
| RSD++Model Family=Llama3 Family, n=162025.06 | 0.095 | 25.8 | 1.18 | 0.932 | |
| BS (S)Model Family=Llama3 Family, n=42025.06 | 0.08 | 5.58 | 0.35 | 0 | |
| BS (S)Model Family=Llama3 Family, n=82025.06 | 0.08 | 11.09 | 0.59 | 0 | |
| RSD++Model Family=Llama3 Family, n=82025.06 | 0.08 | 18.9 | 0.83 | 0.929 | |
| RSD++Model Family=Llama3 Family, n=42025.06 | 0.073 | 13.1 | 0.61 | 0.952 |