STEM Task Evaluation on MMLU Math
51.81AccuracyRe-TASK (Lite)
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Re-TASK (Lite)Model=Llama3-8B2024.08 | 51.81 | 7.61 | |
| One-shot CoTModel=Llama3-8B2024.08 | 49.42 | 2.51 | |
| Zero-shot CoT+SCModel=Llama3-8B2024.08 | 48.19 | 4.23 | |
| Step-BackModel=Llama3-8B2024.08 | 45.65 | -0.6 | |
| Re-TASK (Lite)Model=Qwen1.5-7B2024.08 | 43.84 | 7.61 | |
| Zero-shot CoT+SCModel=Qwen1.5-7B2024.08 | 41.67 | 4.23 | |
| Zero-shot CoTModel=Llama3-8B2024.08 | 40.58 | — | |
| Plan-and-SolveModel=Llama3-8B2024.08 | 40.58 | -2.3 | |
| Zero-shot CoTModel=Qwen1.5-7B2024.08 | 36.96 | — | |
| One-shot CoTModel=Qwen1.5-7B2024.08 | 36.52 | 2.51 | |
| Step-BackModel=Mistral-7B2024.08 | 34.42 | -0.6 | |
| Plan-and-SolveModel=Mistral-7B2024.08 | 30.43 | -2.3 | |
| Re-TASK (Lite)Model=Mistral-7B2024.08 | 28.99 | 7.61 | |
| Zero-shot CoT+SCModel=Mistral-7B2024.08 | 24.64 | 4.23 | |
| Zero-shot CoTModel=Mistral-7B2024.08 | 24.28 | — | |
| Plan-and-SolveModel=Qwen1.5-7B2024.08 | 23.91 | -2.3 | |
| One-shot CoTModel=Mistral-7B2024.08 | 23.41 | 2.51 | |
| Step-BackModel=Qwen1.5-7B2024.08 | 19.93 | -0.6 |