STEM Task Evaluation on MMLU Physics
60.78AccuracyOne-shot CoT
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| One-shot CoTModel=Llama3-8B2024.08 | 60.78 | 3.59 | |
| Re-TASK (Lite)Model=Llama3-8B2024.08 | 60.78 | 6.21 | |
| Zero-shot CoT+SCModel=Llama3-8B2024.08 | 58.82 | -0.65 | |
| Zero-shot CoTModel=Llama3-8B2024.08 | 57.84 | — | |
| Plan-and-SolveModel=Llama3-8B2024.08 | 55.88 | 0.65 | |
| Re-TASK (Lite)Model=Qwen1.5-7B2024.08 | 50.98 | 6.21 | |
| One-shot CoTModel=Mistral-7B2024.08 | 45.1 | 3.59 | |
| Re-TASK (Lite)Model=Mistral-7B2024.08 | 44.12 | 6.21 | |
| Zero-shot CoTModel=Qwen1.5-7B2024.08 | 42.16 | — | |
| One-shot CoTModel=Qwen1.5-7B2024.08 | 42.16 | 3.59 | |
| Plan-and-SolveModel=Mistral-7B2024.08 | 42.16 | 0.65 | |
| Plan-and-SolveModel=Qwen1.5-7B2024.08 | 41.18 | 0.65 | |
| Zero-shot CoT+SCModel=Mistral-7B2024.08 | 39.22 | -0.65 | |
| Zero-shot CoTModel=Mistral-7B2024.08 | 37.25 | — | |
| Zero-shot CoT+SCModel=Qwen1.5-7B2024.08 | 37.25 | -0.65 | |
| Step-BackModel=Mistral-7B2024.08 | 34.31 | -14.05 | |
| Step-BackModel=Llama3-8B2024.08 | 30.39 | -14.05 | |
| Step-BackModel=Qwen1.5-7B2024.08 | 30.39 | -14.05 |