Mathematical Reasoning on MATH 18 (test)
39.7m1@t1GPT-3.5
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| GPT-3.52024.07 | 39.7 | 47.8 | 45.1 | 54.3 | — | — | |
| Self-RefineBackbone=GPT-3.52024.07 | 39.7 | 46.5 | — | — | 36.5 | 46.7 | |
| Eurus-7b-SFT2024.07 | 12.3 | 19.8 | 16.3 | 22.9 | — | — | |
| Self-RefineBackbone=Eurus-7b-SFT2024.07 | 12.3 | 19.4 | — | — | 9 | 15.1 | |
| RISEBackbone=Mistral-7B, Iteration=Base2024.07 | 7.5 | 13 | 8.4 | 13 | — | — | |
| Mistral-7B2024.07 | 7.5 | 13 | 8.4 | 13 | — | — | |
| Self-RefineBackbone=Mistral-7B2024.07 | 7.5 | 12.3 | — | — | 7.1 | 11.4 | |
| RISEBackbone=Mistral-7B, Iteration=+Iteration 12024.07 | 6.7 | 9.5 | 18.4 | 29.7 | — | — | |
| RISEBackbone=Llama2 Base, Iteration=+Iteration 12024.07 | 6.3 | 8.8 | 9.7 | 19.4 | — | — | |
| Self-RefineBackbone=Llama2 Base, Iteration=+Iteration 12024.07 | 6.3 | 8.7 | — | — | 5.9 | 9.9 | |
| RISEBackbone=Llama2 Base, Iteration=+Iteration 22024.07 | 5.8 | 10.4 | 10.4 | 19.8 | — | — | |
| SFT on oracle dataData Type=Only correct data2024.07 | 5.8 | 7.9 | 5.5 | 12.1 | — | — | |
| Self-RefineBackbone=Llama2 Base, Iteration=+Iteration 22024.07 | 5.8 | 9.4 | — | — | 5.7 | 9.5 | |
| RISEBackbone=Llama2 Base, Iteration=+Boost2024.07 | 5.5 | 6.8 | 5.5 | 14.6 | — | — | |
| Self-RefineBackbone=Llama2 Base, Iteration=+Boost2024.07 | 5.5 | 6.5 | — | — | 2.9 | 7.2 | |
| SFT on oracle dataData Type=Correct and incorrect2024.07 | 5 | 5.2 | 5 | 13.1 | — | — | |
| RISEBackbone=Llama2 Base, Iteration=Base2024.07 | 1.9 | 5.1 | 1.4 | 2.3 | — | — | |
| Self-RefineBackbone=Llama2 Base, Iteration=Base2024.07 | 1.9 | 5.1 | — | — | 1.9 | 3.1 |