Mathematical Reasoning on MATH500 (Accuracy and Latency)
81AccuracySPECS
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| SPECSModel Family=Qwen2.5 Family, n=162025.06 | 81 | 15.8 | 1.31 | 45.5 | |
| SPECSModel Family=Qwen2.5 Family, n=82025.06 | 79.5 | 11.3 | 1.31 | 45.5 | |
| RSD++Model Family=Qwen2.5 Family, n=162025.06 | 79 | 16.3 | 1.84 | 26.8 | |
| RSD++Model Family=Qwen2.5 Family, n=82025.06 | 78.3 | 11.8 | 1.37 | 28.4 | |
| RSD++Model Family=Qwen2.5 Family, n=42025.06 | 77.8 | 10.1 | 1.14 | 32.3 | |
| SPECSModel Family=Qwen2.5 Family, n=42025.06 | 77 | 9.83 | 1.11 | 49 | |
| BS (B)Model Family=Qwen2.5 Family, n=162025.06 | 75.5 | 21.3 | 2.15 | 100 | |
| BS (S)Model Family=Qwen2.5 Family, n=162025.06 | 74.5 | 16.8 | 1.57 | 0 | |
| BS (B)Model Family=Qwen2.5 Family, n=82025.06 | 71.3 | 13.6 | 1.51 | 100 | |
| BS (B)Model Family=Qwen2.5 Family, n=42025.06 | 69.3 | 11.2 | 1.13 | 100 | |
| BS (S)Model Family=Qwen2.5 Family, n=82025.06 | 66.7 | 10.7 | 1.01 | 0 | |
| BS (S)Model Family=Qwen2.5 Family, n=42025.06 | 64.3 | 7.9 | 0.65 | 0 | |
| SPECSModel Family=Llama3 Family, n=162025.06 | 44 | 22.9 | 0.86 | 78.1 | |
| SPECSModel Family=Llama3 Family, n=82025.06 | 40.7 | 15.2 | 0.57 | 85.2 | |
| RSD++Model Family=Llama3 Family, n=162025.06 | 40 | 23 | 1.02 | 70.3 | |
| RSD++Model Family=Llama3 Family, n=82025.06 | 39.3 | 15.2 | 0.71 | 73.9 | |
| BS (B)Model Family=Llama3 Family, n=162025.06 | 39 | 18.9 | 0.79 | 100 | |
| RSD++Model Family=Llama3 Family, n=42025.06 | 35.7 | 11.2 | 0.55 | 72.7 | |
| SPECSModel Family=Llama3 Family, n=42025.06 | 35.3 | 10.8 | 0.4 | 91.1 | |
| BS (S)Model Family=Llama3 Family, n=162025.06 | 34.3 | 17.42 | 0.78 | 0 | |
| BS (B)Model Family=Llama3 Family, n=82025.06 | 33 | 12.1 | 0.52 | 100 | |
| BS (B)Model Family=Llama3 Family, n=42025.06 | 32 | 8.4 | 0.39 | 100 | |
| BS (S)Model Family=Llama3 Family, n=82025.06 | 28.3 | 9.55 | 0.45 | 0 | |
| BS (S)Model Family=Llama3 Family, n=42025.06 | 23.1 | 4.82 | 0.27 | 0 |