Mathematical Reasoning on BRUMO (Accuracy, Consistency, # traces)
826Trace CountUniform
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| UniformBackbone=Qwen3-4B, Confidence-weighted=false2026.02 | 826 | 0.94 | 73.2 | |
| UniformBackbone=GPT-20B, Confidence-weighted=false2026.02 | 776 | 0.94 | 90.1 | |
| UniformBackbone=GPT-120B, Confidence-weighted=false2026.02 | 776 | 0.94 | 90.1 | |
| UniformBackbone=Qwen3-4B, Confidence-weighted=true2026.02 | 589 | 0.94 | 75.7 | |
| UniformBackbone=GPT-20B, Confidence-weighted=true2026.02 | 574 | 0.95 | 90.7 | |
| UniformBackbone=GPT-120B, Confidence-weighted=true2026.02 | 574 | 0.95 | 90.7 | |
| UniformBackbone=Qwen-Long, Confidence-weighted=true2026.02 | 421 | 0.97 | 87.2 | |
| UniformBackbone=Qwen-Long, Confidence-weighted=false2026.02 | 320 | 0.97 | 87.3 | |
| UniformBackbone=Qwen3-30B, Confidence-weighted=true2026.02 | 310 | 0.95 | 83.6 | |
| PETS-OfflineBackbone=Qwen3-4B, Confidence-weighted=true2026.02 | 292 | 0.98 | 77.3 | |
| PETS-OfflineBackbone=Qwen3-4B, Confidence-weighted=false2026.02 | 280 | 1 | 75.4 | |
| PETS-OfflineBackbone=GPT-20B, Confidence-weighted=false2026.02 | 253 | 1 | 92.1 | |
| PETS-OfflineBackbone=GPT-120B, Confidence-weighted=false2026.02 | 253 | 1 | 92.1 | |
| UniformBackbone=Qwen3-30B, Confidence-weighted=false2026.02 | 237 | 0.96 | 83.7 | |
| PETS-OfflineBackbone=GPT-20B, Confidence-weighted=true2026.02 | 217 | 0.98 | 92.4 | |
| PETS-OfflineBackbone=GPT-120B, Confidence-weighted=true2026.02 | 217 | 0.98 | 92.4 | |
| PETS-OfflineBackbone=Qwen3-30B, Confidence-weighted=true2026.02 | 162 | 0.98 | 86.1 | |
| PETS-OfflineBackbone=Qwen-Long, Confidence-weighted=true2026.02 | 150 | 0.98 | 87.8 | |
| PETS-OfflineBackbone=Qwen-Long, Confidence-weighted=false2026.02 | 149 | 1 | 86.7 | |
| PETS-OfflineBackbone=Qwen3-30B, Confidence-weighted=false2026.02 | 148 | 1 | 86.7 |