Mathematical Reasoning on GSM8K 10 (test)
66.4m1@t1GPT-3.5
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| GPT-3.52024.07 | 66.4 | 80.6 | 71 | 74.7 | — | — | — | |
| Self-RefineBackbone=GPT-3.52024.07 | 66.4 | — | — | — | 61 | 71.6 | — | |
| GLoREVariant=+ORM2024.07 | 48.2 | — | — | — | 49.5 | 57.1 | — | |
| GLoREVariant=+SORM2024.07 | 48.2 | — | — | — | 51.6 | 59.7 | — | |
| GLoREVariant=+Direct2024.07 | 48.2 | — | — | — | 47.4 | 59.2 | — | |
| RISEBackbone=Llama2 Base, Iteration=+Iteration 22024.07 | 37.3 | 51 | 55 | 68.4 | — | — | — | |
| Self-RefineBackbone=Llama2 Base, Iteration=+Iteration 22024.07 | 37.3 | — | — | — | 33.3 | 44.5 | — | |
| Eurus-7b-SFT2024.07 | 36.3 | 66.3 | 47.9 | 53.1 | — | — | — | |
| Self-RefineBackbone=Eurus-7b-SFT2024.07 | 36.3 | — | — | — | 26.2 | 42.8 | — | |
| RISEBackbone=Llama2 Base, Iteration=+Iteration 12024.07 | 35.6 | 49.7 | 50.7 | 63.9 | — | — | — | |
| Self-RefineBackbone=Llama2 Base, Iteration=+Iteration 12024.07 | 35.6 | — | — | — | 31.7 | 43.7 | — | |
| RISEBackbone=Mistral-7B, Iteration=+Iteration 12024.07 | 35.3 | 50.6 | 59.2 | 68.6 | — | — | — | |
| RISEBackbone=Mistral-7B, Iteration=Base2024.07 | 33.7 | 49.4 | 39 | 46.9 | — | — | — | |
| Mistral-7B2024.07 | 33.7 | 49.4 | 39 | 46.9 | — | — | — | |
| Self-RefineBackbone=Mistral-7B2024.07 | 33.7 | — | — | — | 21.2 | 37.9 | — | |
| RISEBackbone=Llama2 Base, Iteration=+Boost2024.07 | 32.9 | 45.4 | 39.2 | 55.5 | — | — | — | |
| Self-RefineBackbone=Llama2 Base, Iteration=+Boost2024.07 | 32.9 | — | — | — | 26.5 | 40.9 | — | |
| V-STARVariant=+STaR2024.07 | 28 | — | — | — | — | — | 46.1 | |
| V-STARVariant=+Verification2024.07 | 28 | — | — | — | — | — | 56.2 | |
| V-STARVariant=+V-STAR2024.07 | 28 | — | — | — | — | — | 63.2 | |
| SFT on oracle dataData Type=Only correct data2024.07 | 27.4 | 42.2 | 34 | 43.6 | — | — | — | |
| SFT on oracle dataData Type=Correct and incorrect2024.07 | 25.7 | 41.8 | 31.2 | 41.5 | — | — | — | |
| RISEBackbone=Llama2 Base, Iteration=Base2024.07 | 10.5 | 22.8 | 11.1 | 13.9 | — | — | — | |
| Self-RefineBackbone=Llama2 Base, Iteration=Base2024.07 | 10.5 | — | — | — | 7.1 | 13 | — |