Mathematical Problem Solving on MATH (Acc@t1/t2 Evaluation)
47.4Accuracy @ t1Prompt based
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Prompt basedBackbone=Mistral-22B-Small-Instruct, Variant=w/o reflection2025.05 | 47.4 | — | — | |
| Prompt basedBackbone=Mistral-22B-Small-Instruct, Variant=w/ reflection (variant 1)2025.05 | 47.4 | 62.8 | 15.4 | |
| Prompt basedBackbone=Mistral-22B-Small-Instruct, Variant=w/ reflection (variant 2)2025.05 | 47.4 | 62 | 14.6 | |
| Prompt basedBackbone=Meta-Llama-3-70B-Instruct, Variant=w/o reflection2025.05 | 40.8 | — | — | |
| Prompt basedBackbone=Meta-Llama-3-70B-Instruct, Variant=w/ reflection (variant 1)2025.05 | 40.8 | 49.6 | 8.8 | |
| Prompt basedBackbone=Meta-Llama-3-70B-Instruct, Variant=w/ reflection (variant 2)2025.05 | 40.8 | 48.6 | 7.8 | |
| Prompt basedBackbone=Meta-Llama-3-8B-Instruct, Variant=w/o reflection2025.05 | 14.4 | — | — | |
| Prompt basedBackbone=Meta-Llama-3-8B-Instruct, Variant=w/ reflection2025.05 | 14.4 | 15 | 0.6 | |
| ReflectEvoBackbone=Meta-Llama-3-8B-Instruct, Variant=one-stage w/ D+2025.05 | 14.4 | 23.6 | 9.2 | |
| ReflectEvoBackbone=Meta-Llama-3-8B-Instruct, Variant=two-stage w/ D+2025.05 | 14.4 | 14.5 | 0.1 | |
| ReflectEvoBackbone=Meta-Llama-3-8B-Instruct, Variant=w/ D+/-2025.05 | 14.4 | 15.2 | 0.8 | |
| ReflectEvoBackbone=Meta-Llama-3-8B-Instruct, Variant=w/ Dpref2025.05 | 14.4 | 14.8 | 0.4 | |
| SFT basedBackbone=Meta-Llama-3-8B-Instruct, Variant=w/ SFT qa pairs2025.05 | 10 | — | — | |
| Prompt basedBackbone=Mistral-7B-Instruct-v0.2, Variant=w/o reflection2025.05 | 9.2 | — | — | |
| Prompt basedBackbone=Mistral-7B-Instruct-v0.2, Variant=w/ reflection2025.05 | 9.2 | 10.6 | 1.4 | |
| ReflectEvoBackbone=Mistral-7B-Instruct-v0.2, Variant=one-stage w/ D+2025.05 | 9.2 | 10.2 | 1 | |
| ReflectEvoBackbone=Mistral-7B-Instruct-v0.2, Variant=two-stage w/ D+2025.05 | 9.2 | 10 | 0.8 | |
| SFT basedBackbone=Mistral-7B-Instruct-v0.2, Variant=w/ SFT qa pairs2025.05 | 7.6 | — | — |