Mathematical Reasoning on GSM8K (SC Accuracy & Efficiency Metrics)
90.9SC AccuracySelf-Consistency
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Self-ConsistencyModel=QWQ32B, Sample size N=312026.01 | 90.9 | — | — | — | |
| PoLRModel=QWQ32B, Sample size N=312026.01 | 90.9 | -0.6 | 48.6 | 5.8 | |
| Self-ConsistencyModel=QWQ32B, Sample size N=512026.01 | 90.8 | — | — | — | |
| PoLRModel=QWQ32B, Sample size N=512026.01 | 90.8 | -0.3 | 47.6 | 11.2 | |
| Self-ConsistencyModel=QWQ32B, Sample size N=112026.01 | 90.6 | — | — | — | |
| PoLRModel=QWQ32B, Sample size N=112026.01 | 90.6 | -1.3 | 54.2 | 2.4 | |
| Self-ConsistencyModel=DSQ7B, Sample size N=312026.01 | 80 | — | — | — | |
| PoLRModel=DSQ7B, Sample size N=312026.01 | 80 | -0.4 | 27.2 | 4.3 | |
| Self-ConsistencyModel=DSQ7B, Sample size N=512026.01 | 79.8 | — | — | — | |
| PoLRModel=DSQ7B, Sample size N=512026.01 | 79.8 | 0.2 | 26.5 | 5.9 | |
| Self-ConsistencyModel=DSQ7B, Sample size N=112026.01 | 79.7 | — | — | — | |
| PoLRModel=DSQ7B, Sample size N=112026.01 | 79.7 | 0 | 28.1 | 1.9 | |
| Self-ConsistencyModel=DSQ1.5B, Sample size N=512026.01 | 73.2 | — | — | — | |
| PoLRModel=DSQ1.5B, Sample size N=512026.01 | 73.2 | 0 | 40.1 | 11.3 | |
| Self-ConsistencyModel=DSQ1.5B, Sample size N=312026.01 | 73.2 | — | — | — | |
| PoLRModel=DSQ1.5B, Sample size N=312026.01 | 73.2 | 0 | 41.1 | 6 | |
| Self-ConsistencyModel=DSQ1.5B, Sample size N=112026.01 | 72.6 | — | — | — | |
| PoLRModel=DSQ1.5B, Sample size N=112026.01 | 72.6 | 0 | 43.7 | 2.3 |