Mathematical Reasoning on Minerva (Accuracy, #Tokens)
54.53AccuracyPass@k
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Pass@kModel=Phi-4-mini-Instruct, N=322026.05 | 54.53 | 17,119 | |
| Pass@kModel=Qwen2.5-7B-Instruct, N=322026.05 | 52.33 | 19,765 | |
| Pass@kModel=Qwen2.5-3B-Instruct, N=322026.05 | 46.2 | 19,765 | |
| SPSModel=Qwen2.5-7B-Instruct, N=82026.05 | 43.26 | 9,387 | |
| PB-SMCModel=Qwen2.5-7B-Instruct, N=82026.05 | 41.79 | 13,216 | |
| SMCModel=Qwen2.5-7B-Instruct, N=322026.05 | 40.32 | 21,642 | |
| Beam SearchModel=Qwen2.5-7B-Instruct, N=322026.05 | 40.2 | 23,900 | |
| DVTSModel=Qwen2.5-7B-Instruct, N=322026.05 | 39.22 | 23,549 | |
| Self-ConsistencyModel=Qwen2.5-7B-Instruct, N=322026.05 | 38.85 | 19,765 | |
| SPSModel=Qwen2.5-3B-Instruct, N=82026.05 | 38.73 | 10,800 | |
| Best-of-NModel=Qwen2.5-7B-Instruct, N=322026.05 | 38.48 | 19,765 | |
| PB-SMCModel=Qwen2.5-3B-Instruct, N=82026.05 | 38.23 | 15,450 | |
| PB-SMCModel=Phi-4-mini-Instruct, N=82026.05 | 37.87 | 13,198 | |
| SMCModel=Qwen2.5-3B-Instruct, N=322026.05 | 36.52 | 25,143 | |
| MCTSModel=Qwen2.5-7B-Instruct, N=322026.05 | 36.27 | 25,208 | |
| Beam SearchModel=Qwen2.5-3B-Instruct, N=322026.05 | 35.78 | 23,131 | |
| SPSModel=Phi-4-mini-Instruct, N=82026.05 | 35.42 | 9,832 | |
| DVTSModel=Qwen2.5-3B-Instruct, N=322026.05 | 35.05 | 24,971 | |
| Best-of-NModel=Phi-4-mini-Instruct, N=322026.05 | 34.8 | 17,119 | |
| Base ModelModel=Qwen2.5-7B-Instruct, N=12026.05 | 34.68 | 617 | |
| SMCModel=Phi-4-mini-Instruct, N=322026.05 | 34.44 | 24,907 | |
| DVTSModel=Phi-4-mini-Instruct, N=322026.05 | 33.82 | 22,140 | |
| Best-of-NModel=Qwen2.5-3B-Instruct, N=322026.05 | 33.7 | 19,765 | |
| Self-ConsistencyModel=Phi-4-mini-Instruct, N=322026.05 | 33.7 | 17,119 | |
| MCTSModel=Qwen2.5-3B-Instruct, N=322026.05 | 33.58 | 26,121 | |
| Beam SearchModel=Phi-4-mini-Instruct, N=322026.05 | 33.46 | 23,768 | |
| Self-ConsistencyModel=Qwen2.5-3B-Instruct, N=322026.05 | 33.21 | 19,765 | |
| MCTSModel=Phi-4-mini-Instruct, N=322026.05 | 32.75 | 23,108 | |
| Base ModelModel=Qwen2.5-3B-Instruct, N=12026.05 | 29.53 | 617 | |
| Base ModelModel=Phi-4-mini-Instruct, N=12026.05 | 23.9 | 535 |