Mathematical Reasoning on GSM8K (Custom Sampling Metrics)
66.4m1@t1Self-Refine
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Self-RefineBackbone=GPT-3.52024.07 | 66.4 | 80.2 | — | — | 61 | 71.6 | |
| GLoREComponent=ORM2024.07 | 48.2 | — | — | — | 49.5 | 57.1 | |
| GLoREComponent=SORM2024.07 | 48.2 | — | — | — | 51.6 | 59.7 | |
| GLoREComponent=Direct2024.07 | 48.2 | — | — | — | 47.4 | 59.2 | |
| RISEBackbone=Llama2 Base, Iteration=22024.07 | 37.3 | 51 | 55 | 68.4 | — | — | |
| Self-RefineBackbone=Llama2 Base, Iteration=22024.07 | 37.3 | 50.5 | — | — | 33.3 | 44.5 | |
| Eurus-7B-SFTBackbone=Eurus-7B2024.07 | 36.3 | 66.3 | 47.9 | 53.1 | — | — | |
| Self-RefineBackbone=Eurus-7B-SFT2024.07 | 36.3 | 65.9 | — | — | 26.2 | 42.8 | |
| RISEBackbone=Llama2 Base, Iteration=12024.07 | 35.6 | 49.7 | 50.7 | 63.9 | — | — | |
| RISEBackbone=Mistral-7B, Iteration=12024.07 | 35.3 | 50.6 | 59.2 | 68.6 | — | — | |
| RISEBackbone=Mistral-7B2024.07 | 33.7 | 49.4 | 39 | 46.9 | — | — | |
| Self-RefineBackbone=Mistral-7B2024.07 | 33.7 | 48.5 | — | — | 21.2 | 37.9 | |
| RISEBackbone=Llama2 Base, Stage=Boost2024.07 | 32.9 | 45.4 | 39.2 | 55.5 | — | — | |
| SFT on oracle dataBackbone=Llama2 Base, Data=Only correct data2024.07 | 27.4 | 42.2 | 34 | 43.6 | — | — | |
| SFT on oracle dataBackbone=Llama2 Base, Data=Correct and incorrect2024.07 | 25.7 | 41.8 | 31.2 | 41.5 | — | — | |
| RISEBackbone=Llama2 Base2024.07 | 10.5 | 22.8 | 11.1 | 13.9 | — | — | |
| Self-RefineBackbone=Llama2 Base2024.07 | 10.5 | 22.4 | — | — | 7.1 | 13 |