Mathematical Reasoning on AIME 24 (SC Accuracy, Efficiency, Overhead)
80SC AccuracySelf-Consistency
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Self-ConsistencyModel=QWQ32B, Sample size N=512026.01 | 80 | — | — | — | |
| PoLRModel=QWQ32B, Sample size N=512026.01 | 80 | 0 | 59.7 | 10.8 | |
| Self-ConsistencyModel=QWQ32B, Sample size N=312026.01 | 78.3 | — | — | — | |
| PoLRModel=QWQ32B, Sample size N=312026.01 | 78.3 | 0.7 | 61.6 | 6.2 | |
| Self-ConsistencyModel=QWQ32B, Sample size N=112026.01 | 78.3 | — | — | — | |
| PoLRModel=QWQ32B, Sample size N=112026.01 | 78.3 | -2 | 66.6 | 12.8 | |
| Self-ConsistencyModel=DSQ7B, Sample size N=112026.01 | 54.3 | — | — | — | |
| PoLRModel=DSQ7B, Sample size N=112026.01 | 54.3 | -3.3 | 49.8 | 2 | |
| Self-ConsistencyModel=DSQ7B, Sample size N=512026.01 | 53.3 | — | — | — | |
| PoLRModel=DSQ7B, Sample size N=512026.01 | 53.3 | -6.7 | 50.9 | 6.3 | |
| Self-ConsistencyModel=DSQ7B, Sample size N=312026.01 | 53.3 | — | — | — | |
| PoLRModel=DSQ7B, Sample size N=312026.01 | 53.3 | -3 | 51.5 | 3.3 | |
| Self-ConsistencyModel=Phi-4-15B, Sample size N=112026.01 | 50.3 | — | — | — | |
| PoLRModel=Phi-4-15B, Sample size N=112026.01 | 50.3 | -5.3 | 58.6 | 2.2 | |
| Self-ConsistencyModel=Phi-4-15B, Sample size N=512026.01 | 50 | — | — | — | |
| PoLRModel=Phi-4-15B, Sample size N=512026.01 | 50 | 3.3 | 49.5 | 12.1 | |
| Self-ConsistencyModel=Phi-4-15B, Sample size N=312026.01 | 49.7 | — | — | — | |
| PoLRModel=Phi-4-15B, Sample size N=312026.01 | 49.7 | 0.3 | 51.5 | 5.3 |